ViT with diamond patches for lane line detection under occlusion

ViT with diamond patches for lane line detection under occlusion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article ViT with diamond patches for lane line detection under occlusion Xianrang Shi, Rong Wang, Hengyu Zhang, Zezhi Li, Yang Su, Yan Ti, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3179683/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Lane line detection based on deep learning has achieved good results in common scenarios. However, it is challenging to detect lane lines in extreme occlusion scenes where the visual clues are severely missing. To address this problem, we propose a novel method that leverages Vision Transformer (ViT) for de-occlusion and feature fusion. Specifically, we first design a ViT-based model to reconstruct the occluded lane lines from the input image. Then, we extract the feature map of the model and fuse it with the original image feature map. Finally, we use the fused feature map to detect the lane lines in a robust manner. Moreover, we introduce a sensitivity loss function that measures the error of each pixel and considers the coordinate difference between pixels. Experiments show that our sensitivity loss function can improve the performance of lane line detection. We evaluate our method on three benchmark datasets: TuSimple, CULane and CurveLanes. The results demonstrate that our method outperforms the existing methods in terms of accuracy and F1-score on all these datasets. lane line detection Transformer deep learning image reconstruction loss function. Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3179683","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":219549143,"identity":"024e897b-4abf-40ed-acca-0caddfd66eb9","order_by":0,"name":"Xianrang Shi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAp0lEQVRIiWNgGAWjYBADOTb29gOkaTHm4zmTQJqWxHkSDgbEKTWXSH748WfbvfQ2CYYEhh8V2whrsZyRZizNc6Y4t0268QBjz5nbhLUY3EgwkGaoSMhtkzmQwMzYRpSW9M8/fxgkpLNJJBgQqyXHTIKnIiGBBC1n3pRZA8PXsA1IHCTOL8fTN9/82ZYgL9/efvDBjwoitKCAAySqHwWjYBSMglGACwAA4R86PJwcLdYAAAAASUVORK5CYII=","orcid":"","institution":"Nanjing University of Aeronautics and Astronautics","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Xianrang","middleName":"","lastName":"Shi","suffix":""},{"id":219549144,"identity":"9a089ee8-cb84-46b4-b31d-e8a4ab92f8db","order_by":1,"name":"Rong Wang","email":"","orcid":"","institution":"Nanjing University of Aeronautics and Astronautics","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Rong","middleName":"","lastName":"Wang","suffix":""},{"id":219549145,"identity":"c0780981-c5cf-4f12-b31e-80037505cc3e","order_by":2,"name":"Hengyu Zhang","email":"","orcid":"","institution":"Nanjing University of Aeronautics and Astronautics","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Hengyu","middleName":"","lastName":"Zhang","suffix":""},{"id":219549146,"identity":"39b6267e-9970-41db-8063-235f02cf410e","order_by":3,"name":"Zezhi Li","email":"","orcid":"","institution":"Nanjing University of Aeronautics and Astronautics","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zezhi","middleName":"","lastName":"Li","suffix":""},{"id":219549147,"identity":"8591580f-967f-4092-95a8-e051dbddf14f","order_by":4,"name":"Yang Su","email":"","orcid":"","institution":"Nanjing University of Aeronautics and Astronautics","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yang","middleName":"","lastName":"Su","suffix":""},{"id":219549148,"identity":"31e55db7-39a6-4834-8405-e17f800c4460","order_by":5,"name":"Yan Ti","email":"","orcid":"","institution":"Jiangsu University of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yan","middleName":"","lastName":"Ti","suffix":""},{"id":219549149,"identity":"1bde7f1f-9759-4b6e-a599-32c3219f93de","order_by":6,"name":"Tinglun Song","email":"","orcid":"","institution":"Chery Automobile CO., Ltd","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Tinglun","middleName":"","lastName":"Song","suffix":""}],"badges":[],"createdAt":"2023-07-18 02:29:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3179683/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3179683/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":46708899,"identity":"5171817e-2917-4dd9-a6c7-d5275a2d4aca","added_by":"auto","created_at":"2023-11-18 17:52:34","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":763432,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3179683/v1_covered_a6904634-f3f2-4cb6-ad38-530129b2fa50.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"ViT with diamond patches for lane line detection under occlusion","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":" lane line detection, Transformer, deep learning, image reconstruction, loss function.","lastPublishedDoi":"10.21203/rs.3.rs-3179683/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3179683/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Lane line detection based on deep learning has achieved good results in common scenarios. However, it is challenging to detect lane lines in extreme occlusion scenes where the visual clues are severely missing. To address this problem, we propose a novel method that leverages Vision Transformer (ViT) for de-occlusion and feature fusion. Specifically, we first design a ViT-based model to reconstruct the occluded lane lines from the input image. Then, we extract the feature map of the model and fuse it with the original image feature map. Finally, we use the fused feature map to detect the lane lines in a robust manner. Moreover, we introduce a sensitivity loss function that measures the error of each pixel and considers the coordinate difference between pixels. Experiments show that our sensitivity loss function can improve the performance of lane line detection. We evaluate our method on three benchmark datasets: TuSimple, CULane and CurveLanes. The results demonstrate that our method outperforms the existing methods in terms of accuracy and F1-score on all these datasets.","manuscriptTitle":"ViT with diamond patches for lane line detection under occlusion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-07-21 02:40:09","doi":"10.21203/rs.3.rs-3179683/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"279eb606-fc5b-49e4-bcc8-623987ca3369","owner":[],"postedDate":"July 21st, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2023-11-18T17:44:14+00:00","versionOfRecord":[],"versionCreatedAt":"2023-07-21 02:40:09","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3179683","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3179683","identity":"rs-3179683","version":["v1"]},"buildId":"WrCJVZZCHTDjtuVLN7oU0","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}