Vi-DiSC: A novel dataset and framework for extracting information from Vietnamese signposts images

Vi-DiSC: A novel dataset and framework for extracting information from Vietnamese signposts images | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Vi-DiSC: A novel dataset and framework for extracting information from Vietnamese signposts images Minh Tam Nguyen, Hoang Anh Tran, Truong Thinh Nguyen, Gia Phu Pham Tran, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3793638/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract With the development of autonomous vehicle technologies, a vision-based vehicle guidance system should be able to understand traffic signs, especially directional signposts. Directional signposts are not as specific as the warning and compulsory signs since they include many different variations. More specifically, they have special arrangements, but still follow some principles. It is difficult for captioning models to tell precisely what information the sign road includes. From the above difficulty, we propose to build a framework that simulates the process of reading a human traffic sign, which we named Vi-DiSC. The framework includes a directional sign recognition model, an optical character recognition system in parallel with a directional arrow recognition model, and finally a rule-based model based on the information extracted from the above models. We also built two Deep Learning models for the Image Captioning task to compare the generated results with the rule-based system. Our team also collects, filters, labels, and augments the directional signposts images to create the dataset for the image captioning task. In addition, we also propose TRID, a suitable metric for evaluating the directional signposts image captioning task. The framework is trained and evaluated on our dataset using various metrics, including our proposed metric TRID, and achieved promising resutls. The dataset is available at \url{ https://github.com/thinhnt19393/ViDirectionSignpostCaptioning} . Framework Image Captioning Signpost OCR Object Detection Rule-based Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3793638","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":262681769,"identity":"d82aae53-0553-4668-ad96-4b3a6d1f0d6c","order_by":0,"name":"Minh Tam Nguyen","email":"","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Minh","middleName":"Tam","lastName":"Nguyen","suffix":""},{"id":262681770,"identity":"8f2b983e-b08f-41a6-8232-a171208d9ef7","order_by":1,"name":"Hoang Anh Tran","email":"","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Hoang","middleName":"Anh","lastName":"Tran","suffix":""},{"id":262681771,"identity":"63bc4c54-ce22-4b59-bd9e-5b51bede4113","order_by":2,"name":"Truong Thinh Nguyen","email":"","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Truong","middleName":"Thinh","lastName":"Nguyen","suffix":""},{"id":262681772,"identity":"ed01d52c-5a87-4e84-8bd0-dd69279b80ef","order_by":3,"name":"Gia Phu Pham Tran","email":"","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Gia","middleName":"Phu Pham","lastName":"Tran","suffix":""},{"id":262681773,"identity":"480b8a38-3c3d-49f5-b692-9c6e7ae243d7","order_by":4,"name":"Co Thai Quach","email":"","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Co","middleName":"Thai","lastName":"Quach","suffix":""},{"id":262681774,"identity":"e0ba4c03-10b7-4e5d-9d2c-aa58e4a8c1b2","order_by":5,"name":"Trong-Hop Do","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYBACAyCWYKgAMRkbgAQzkVoOnAFraWwgXsvBNgaYNURoMWfvPXj747zaaAaJ5PYHDBXWiQ0SuQfwarHsOZdscXDb8dwGiUSgw86kA7XkJeB32I0cM4mD245BtDC2HU5s4DljQISWOTAt/4jW0lAD1dIA1MLeQ0DLGaBfzhw7kNvG87BxRsKxdOM2glqO9x68UVFTl9vPnv7gw4caa9l+Zh78WhgYeEDEYQY2EJUAxGwE1MO01BFWNwpGwSgYBSMXAABeLky+QGE+QwAAAABJRU5ErkJggg==","orcid":"","institution":"University of Information Technology, Vietnam National University Ho Chi Minh City","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Trong-Hop","middleName":"","lastName":"Do","suffix":""}],"badges":[],"createdAt":"2023-12-22 19:44:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3793638/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3793638/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":50461799,"identity":"34b80742-c7e9-4024-907a-cf9ae4242a04","added_by":"auto","created_at":"2024-01-31 21:22:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1053918,"visible":true,"origin":"","legend":"","description":"","filename":"TrafficSign2columns.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3793638/v1_covered_04efdc8f-715b-4365-97fa-f75a85795caf.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Vi-DiSC: A novel dataset and framework for extracting information from Vietnamese signposts images","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Framework, Image Captioning, Signpost, OCR, Object Detection, Rule-based","lastPublishedDoi":"10.21203/rs.3.rs-3793638/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3793638/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"With the development of autonomous vehicle technologies, a vision-based vehicle guidance system should be able to understand traffic signs, especially directional signposts. Directional signposts are not as specific as the warning and compulsory signs since they include many different variations. More specifically, they have special arrangements, but still follow some principles. It is difficult for captioning models to tell precisely what information the sign road includes. From the above difficulty, we propose to build a framework that simulates the process of reading a human traffic sign, which we named Vi-DiSC. The framework includes a directional sign recognition model, an optical character recognition system in parallel with a directional arrow recognition model, and finally a rule-based model based on the information extracted from the above models. We also built two Deep Learning models for the Image Captioning task to compare the generated results with the rule-based system. Our team also collects, filters, labels, and augments the directional signposts images to create the dataset for the image captioning task. In addition, we also propose TRID, a suitable metric for evaluating the directional signposts image captioning task. The framework is trained and evaluated on our dataset using various metrics, including our proposed metric TRID, and achieved promising resutls. The dataset is available at \\url{https://github.com/thinhnt19393/ViDirectionSignpostCaptioning}.\n","manuscriptTitle":"Vi-DiSC: A novel dataset and framework for extracting information from Vietnamese signposts images","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-12-28 02:55:53","doi":"10.21203/rs.3.rs-3793638/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"3a7903cd-96f5-4f0f-917f-9d68bddf2267","owner":[],"postedDate":"December 28th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-01-31T21:14:19+00:00","versionOfRecord":[],"versionCreatedAt":"2023-12-28 02:55:53","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3793638","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3793638","identity":"rs-3793638","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}