End-to-End Semantically Aware Tactile Generation

End-to-End Semantically Aware Tactile Generation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article End-to-End Semantically Aware Tactile Generation Mohammad Mahdi Heydari Dastjerdi, Abbas Akkasi, Hilaire Djani, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5338871/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Tactile graphics are an essential tool for conveying visual information to visually impaired individuals. However, translating 2D plots, such as B´ezier curves, polygons, and bar charts, into an effective tactile format remains a challenge. This paper presents a novel, two-stage deep learning pipeline for automating this conversion process.Our method leverages a Pix2Pix architecture, employing a U-Net++ generatornetwork for robust image generation. To improve the perceptual quality of the tactilerepresentations, we incorporate an adversarial perceptual loss function alongside agradient penalty. The pipeline operates in a sequential manner: firstly, convertingthe source plot into a grayscale tactile representation, followed by a transformationinto a channel-wise equivalent.We evaluate the performance of our model on a comprehensive synthetic datasetconsisting of 20,000 source-target pairs encompassing various 2D plot types. Toquantify performance, we utilize fuzzy versions of established metrics like pixel accuracy, Dice coefficient, and Jaccard index. Additionally, a human study is conductedto assess the visual quality of the generated tactile graphics.The proposed approach demonstrates promising results, significantly streamliningthe conversion of 2D plots into tactile graphics. This paves the way for the development of fully automated systems, enhancing accessibility of visual information forvisually impaired individuals. Tactile Generation U-Net++ Perceptual Loss Gradient Penalty Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5338871","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":373491561,"identity":"c4b2183f-5974-47d0-bdd6-89102b2e23c9","order_by":0,"name":"Mohammad Mahdi Heydari Dastjerdi","email":"","orcid":"","institution":"Carleton University","correspondingAuthor":false,"prefix":"","firstName":"Mohammad","middleName":"Mahdi Heydari","lastName":"Dastjerdi","suffix":""},{"id":373491562,"identity":"bf5c1911-a05b-4f70-ad59-cf78cfa9b00c","order_by":1,"name":"Abbas Akkasi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAElEQVRIie2Qv0vDQBTHXzm4Lg+zpgSSf+GFg0Axxn9FOdDlRMdCC82ki8U14D/T46DZ6hqxgyWDi0Pd4tYjzgfN5nCf4Xvw4HPvB4DH8x8JsX8otLE+2DgbpOjKBgc2QGF4ihK8rvRPB4WYPNV7ky+Kez5+0e0DFImzyW4rIwSZRajIqI2cPqJhogKZli6nURTZkfIYFJi7khEPJbefrEcuJWmU+O1gmcfB16eZlkviSdsrly6FGpXZC5gsCq/IjEpju7BeuXYp6W57c45Ui0n1Tfp5UxNHKQSSlC4l/liZ9242T6u32/bQLeYUjPW+xVlx4Vz/b7wTKh6Px+MZwBEuPE0JVLOXuwAAAABJRU5ErkJggg==","orcid":"","institution":"Carleton University","correspondingAuthor":true,"prefix":"","firstName":"Abbas","middleName":"","lastName":"Akkasi","suffix":""},{"id":373491563,"identity":"da845b15-3e1d-4f29-b854-1f585cf61032","order_by":2,"name":"Hilaire Djani","email":"","orcid":"","institution":"Carleton University","correspondingAuthor":false,"prefix":"","firstName":"Hilaire","middleName":"","lastName":"Djani","suffix":""},{"id":373491564,"identity":"53d13180-873a-404d-8885-0fba704c312d","order_by":3,"name":"Aatreyi Pranavbhai Mehta","email":"","orcid":"","institution":"Carleton University","correspondingAuthor":false,"prefix":"","firstName":"Aatreyi","middleName":"Pranavbhai","lastName":"Mehta","suffix":""},{"id":373491565,"identity":"e537a4d2-9b55-447e-a821-2223e06acf6f","order_by":4,"name":"Majid Komeili","email":"","orcid":"","institution":"Carleton University","correspondingAuthor":false,"prefix":"","firstName":"Majid","middleName":"","lastName":"Komeili","suffix":""}],"badges":[],"createdAt":"2024-10-26 19:23:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5338871/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5338871/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":76499899,"identity":"18000bfc-fd34-4167-b85c-9f6acdbc3dd4","added_by":"auto","created_at":"2025-02-17 20:01:41","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":499852,"visible":true,"origin":"","legend":"","description":"","filename":"Visualcomputer.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5338871/v1_covered_64ea35ef-4a5b-450d-8851-54078e212185.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"End-to-End Semantically Aware Tactile Generation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Tactile Generation, U-Net++, Perceptual Loss, Gradient Penalty","lastPublishedDoi":"10.21203/rs.3.rs-5338871/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5338871/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTactile graphics are an essential tool for conveying visual information to visually impaired individuals. However, translating 2D plots, such as B\u0026acute;ezier curves, polygons, and bar charts, into an effective tactile format remains a challenge. This paper presents a novel, two-stage deep learning pipeline for automating this conversion process.Our method leverages a Pix2Pix architecture, employing a U-Net++ generatornetwork for robust image generation. To improve the perceptual quality of the tactilerepresentations, we incorporate an adversarial perceptual loss function alongside agradient penalty. The pipeline operates in a sequential manner: firstly, convertingthe source plot into a grayscale tactile representation, followed by a transformationinto a channel-wise equivalent.We evaluate the performance of our model on a comprehensive synthetic datasetconsisting of 20,000 source-target pairs encompassing various 2D plot types. Toquantify performance, we utilize fuzzy versions of established metrics like pixel accuracy, Dice coefficient, and Jaccard index. Additionally, a human study is conductedto assess the visual quality of the generated tactile graphics.The proposed approach demonstrates promising results, significantly streamliningthe conversion of 2D plots into tactile graphics. This paves the way for the development of fully automated systems, enhancing accessibility of visual information forvisually impaired individuals.\u003c/p\u003e","manuscriptTitle":"End-to-End Semantically Aware Tactile Generation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-06 02:56:48","doi":"10.21203/rs.3.rs-5338871/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ab713cd2-424a-4ea4-9d2f-a3fb3b343190","owner":[],"postedDate":"November 6th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-02-17T19:53:37+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-06 02:56:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5338871","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5338871","identity":"rs-5338871","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}