{"paper_id":"0ff3138f-4e49-49de-9dda-8e85dfffa974","body_text":"Hierarchical Vision–Language-Aware Product Detection in Dense Retail Shelves Using Enhanced YOLOv12 with Hybrid OCR-LLM Refinement | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Hierarchical Vision–Language-Aware Product Detection in Dense Retail Shelves Using Enhanced YOLOv12 with Hybrid OCR-LLM Refinement Kiran Muraleedharan, B. S. Revathi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9540033/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Automated product identification from densely packed retail shelf images remains a central challenge for intelligent inventory management, afflicted by occlusion, clutter, and the high visual similarity of competing stock-keeping units (SKUs). This journal article extends and significantly deepens a prior conference work by integrating an improved multi-stage inference pipeline that fuses YOLOv12-based detection with an adaptive OCR-LLM refinement layer. The full methodology retains all previously validated preprocessing and augmentation components— Adaptive Shelf Segmentation and Perspective Normalisation (ASSPN), Context-Aware Normalisation (CAN), and the domain-specific ShelfMix augmentation strategy—and introduces three novel contributions at the final inference stage: (i) a confidence-stratified routing mechanism that directs each detection through direct, validated, or OCR-LLM pathways according to YOLO confidence; (ii) a semantic similarity guard using sentence-transformer embeddings that resolves conflicts between YOLO class labels and EasyOCR extractions; and (iii) a Phi-3 Mini language model post-processor with a structured noise filter for clean product name generation. Comprehensive experiments on a 45,000-image, 120-category benchmark demonstrate a mean Average Precision ( [email protected] ) of 92.8 % at 66 FPS, outperforming YOLOv8, YOLOv7, and ConvNeXt-YOLO. Statistical significance tests (paired t-test, p < 0.01) confirm the reliability of gains. Ablation studies validate the independent contribution of each proposed module. The resulting system offers a practical, scalable solution for real-time retail analytics in resource-constrained deployments. Retail shelf detection YOLOv12 attention mechanism cross-modal fusion data augmentation vision-language classification OCR large language model inventory management Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 15 May, 2026 Reviewers invited by journal 11 May, 2026 Editor assigned by journal 28 Apr, 2026 Submission checks completed at journal 28 Apr, 2026 First submitted to journal 27 Apr, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-9540033\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":false,\"archivedVersions\":[],\"articleType\":\"Research Article\",\"associatedPublications\":[],\"authors\":[{\"id\":638615792,\"identity\":\"032f312b-1d21-4062-9438-c3ddfe763b31\",\"order_by\":0,\"name\":\"Kiran Muraleedharan\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA80lEQVRIiWNgGAWjYBACCWYQWWEjJ4EQSwAjAlrOpBmToAVEMLYdTpyBogUfkGznffi5gI05fWb72WfShW12DPzsOQYMD3fg1iLNzG4sPYOHLXc2T7qZ9My2ZAbJnjcGDIlncGuRY2ZjkOaR4Mmdx5DGJs3bxsxgcANoS2IbXi3Mv3kMJNLl+J+BtNQz2BPSIs3MxibNk2CQIC0BtuUwg4EEAS2SzWxs1jwHEgxnznjGbM1z7jiPxJlnBQfwaZE4f4z5Nu+///IS59MYb/OUVcvxtydvfPgTjxZUwMjGwAOiDxCrAQj+kKB2FIyCUTAKRgwAAPpZQPaGMHiPAAAAAElFTkSuQmCC\",\"orcid\":\"\",\"institution\":\"Karunya Institute of Technology and Sciences\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Kiran\",\"middleName\":\"\",\"lastName\":\"Muraleedharan\",\"suffix\":\"\"},{\"id\":638615793,\"identity\":\"64c07d2e-0ea9-492e-aaef-4d6b217429b1\",\"order_by\":1,\"name\":\"B. S. Revathi\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Karunya Institute of Technology and Sciences\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"B.\",\"middleName\":\"S.\",\"lastName\":\"Revathi\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2026-04-27 10:11:06\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-9540033/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-9540033/v1\",\"draftVersion\":[],\"editorialEvents\":[],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":109181429,\"identity\":\"b61fb954-6083-47e4-9f3c-f35803574d55\",\"added_by\":\"auto\",\"created_at\":\"2026-05-13 10:15:25\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":245853,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"journal.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-9540033/v1_covered_65208c67-44cb-4f41-a12e-80dd14d78573.pdf\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Hierarchical Vision–Language-Aware Product Detection in Dense Retail Shelves Using Enhanced YOLOv12 with Hybrid OCR-LLM Refinement\",\"fulltext\":[],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":false,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":false,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":false,\"isAuthorSuppliedPdf\":true,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":true,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"signal-image-and-video-processing\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"sivp\",\"sideBox\":\"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)\",\"snPcode\":\"11760\",\"submissionUrl\":\"https://submission.nature.com/new-submission/11760/3\",\"title\":\"Signal, Image and Video Processing\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"em\",\"reportingPortfolio\":\"Springer Hybrid\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":false},\"keywords\":\"Retail shelf detection, YOLOv12, attention mechanism, cross-modal fusion, data augmentation, vision-language classification, OCR, large language model, inventory management\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-9540033/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-9540033/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"Automated product identification from densely packed retail shelf images remains a central challenge for intelligent inventory management, afflicted by occlusion, clutter, and the high visual similarity of competing stock-keeping units (SKUs). This journal article extends and significantly deepens a prior conference work by integrating an improved multi-stage inference pipeline that fuses YOLOv12-based detection with an adaptive OCR-LLM refinement layer. The full methodology retains all previously validated preprocessing and augmentation components— Adaptive Shelf Segmentation and Perspective Normalisation (ASSPN), Context-Aware Normalisation (CAN), and the domain-specific ShelfMix augmentation strategy—and introduces three novel contributions at the final inference stage: (i) a confidence-stratified routing mechanism that directs each detection through direct, validated, or OCR-LLM pathways according to YOLO confidence; (ii) a semantic similarity guard using sentence-transformer embeddings that resolves conflicts between YOLO class labels and EasyOCR extractions; and (iii) a Phi-3 Mini language model post-processor with a structured noise filter for clean product name generation. Comprehensive experiments on a 45,000-image, 120-category benchmark demonstrate a mean Average Precision (mAP@0.5) of 92.8 % at 66 FPS, outperforming YOLOv8, YOLOv7, and ConvNeXt-YOLO. Statistical significance tests (paired t-test, p \\u003c 0.01) confirm the reliability of gains. Ablation studies validate the independent contribution of each proposed module. The resulting system offers a practical, scalable solution for real-time retail analytics in resource-constrained deployments.\",\"manuscriptTitle\":\"Hierarchical Vision–Language-Aware Product Detection in Dense Retail Shelves Using Enhanced YOLOv12 with Hybrid OCR-LLM Refinement\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2026-05-13 10:13:56\",\"doi\":\"10.21203/rs.3.rs-9540033/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0},{\"type\":\"reviewerAgreed\",\"content\":\"338427398603599141908564234197011822605\",\"date\":\"2026-05-15T08:55:10+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewersInvited\",\"content\":\"\",\"date\":\"2026-05-11T19:29:35+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorAssigned\",\"content\":\"\",\"date\":\"2026-04-28T09:39:29+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"checksComplete\",\"content\":\"\",\"date\":\"2026-04-28T09:38:49+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"submitted\",\"content\":\"Signal, Image and Video Processing\",\"date\":\"2026-04-27T09:53:00+00:00\",\"index\":\"\",\"fulltext\":\"\"}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"signal-image-and-video-processing\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"sivp\",\"sideBox\":\"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)\",\"snPcode\":\"11760\",\"submissionUrl\":\"https://submission.nature.com/new-submission/11760/3\",\"title\":\"Signal, Image and Video Processing\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"em\",\"reportingPortfolio\":\"Springer Hybrid\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":false}}],\"origin\":\"\",\"ownerIdentity\":\"1e3927e8-880e-40f9-9255-f7e39c5cfe0c\",\"owner\":[],\"postedDate\":\"May 13th, 2026\",\"published\":true,\"recentEditorialEvents\":[{\"type\":\"reviewerAgreed\",\"content\":\"338427398603599141908564234197011822605\",\"date\":\"2026-05-15T08:55:10+00:00\",\"index\":17,\"fulltext\":\"\"},{\"type\":\"reviewersInvited\",\"content\":\"12\",\"date\":\"2026-05-11T19:29:35+00:00\",\"index\":\"\",\"fulltext\":\"\"}],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"under-review\",\"subjectAreas\":[],\"tags\":[],\"updatedAt\":\"2026-05-13T10:13:56+00:00\",\"versionOfRecord\":[],\"versionCreatedAt\":\"2026-05-13 10:13:56\",\"video\":\"\",\"vorDoi\":\"\",\"vorDoiUrl\":\"\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-9540033\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-9540033\",\"identity\":\"rs-9540033\",\"version\":[\"v1\"]},\"buildId\":\"XKTyCvWXoU3ODBz1xrDgd\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}