Enhanced 3D Object Detection using 4D Radar and Vision Fusion with Segmentation Assistance

Enhanced 3D Object Detection using 4D Radar and Vision Fusion with Segmentation Assistance | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Enhanced 3D Object Detection using 4D Radar and Vision Fusion with Segmentation Assistance Xuemei Chen, Yaohan Jia, Pengfei Ren, Zeyuan Xu, Wenzhe shan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5358941/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract 4D radar exhibits robustness to complex lighting and adverse weather conditions, providing unique data characteristics compared to LiDAR for 3D target detection. However, due to the sparsity of 4D radar point clouds, the performance of most 3D target detection algorithms is limited. To address this, this paper propose a 3D object detection model based on fine-grained point cloud segmentation. Our approach first enriches the point cloud data using a radar reference point module to compensate for its sparsity. The point cloud is then pillarized, and semantic information is extracted through a simple segmentation network. Finally, 3D object detection is achieved by fusing point cloud features and semantic information using an attention mechanism. Extensive experiments conducted on the VoD dataset demonstrate that our model achieves a mean average precision (mAP) that is 5% higher than the baseline on the validation set, with notable improvements of 4% for bicycles and 8% for pedestrians. These results narrow the performance gap with LiDAR-based models, highlighting the effectiveness of our segmentation-assisted detection approach.the source codes are released at https://github.com/Huniki/RVASANET.git Autonomous driving 4D radar object detection Semantic segmentation Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5358941","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":375273984,"identity":"119f8009-ee5d-497a-8a63-d1dd4f21aaaa","order_by":0,"name":"Xuemei Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAz0lEQVRIiWNgGAWjYDACCQhV38/MfODAhwoStDDObGdLPDjjDClaNpznMT7M20KEDv7Zzcce87bZMEs283w4wNvAIM8vdoCAJXeOpRvObEtj42fm3XBAcgeD4czZCfi1GEjkmEl8bDvMI9kM1GJ4hiHB4DZBLfnfJBLb/ksYHOZ5cCCxjSgtOWxAWw4YALUwHDhIjBaJG2lmkjPOJSdINrMZHGw4I0HYL/wzkp9J85TZJfDzH378+U+FjTy/NAEtGLaSpnwUjIJRMApGAXYAAMMrRLxYILPLAAAAAElFTkSuQmCC","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Xuemei","middleName":"","lastName":"Chen","suffix":""},{"id":375273985,"identity":"26fbd161-6db9-438f-af3b-e6bee1c69f93","order_by":1,"name":"Yaohan Jia","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Yaohan","middleName":"","lastName":"Jia","suffix":""},{"id":375273986,"identity":"af94ebd3-a19b-46cd-bb95-4adc2ff1cc68","order_by":2,"name":"Pengfei Ren","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Pengfei","middleName":"","lastName":"Ren","suffix":""},{"id":375273987,"identity":"e679defa-4062-486e-8107-37035062f308","order_by":3,"name":"Zeyuan Xu","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Zeyuan","middleName":"","lastName":"Xu","suffix":""},{"id":375273988,"identity":"84b1d678-6dc1-4b11-b7d1-93082d26505e","order_by":4,"name":"Wenzhe shan","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Wenzhe","middleName":"","lastName":"shan","suffix":""}],"badges":[],"createdAt":"2024-10-30 06:53:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5358941/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5358941/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":79971256,"identity":"fbd545b7-8735-4ff9-9b99-652beeadca11","added_by":"auto","created_at":"2025-04-05 16:46:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1281898,"visible":true,"origin":"","legend":"","description":"","filename":"RVSADNet.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5358941/v1_covered_62c61c71-04b5-43fa-8672-3138d12c54ae.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhanced 3D Object Detection using 4D Radar and Vision Fusion with Segmentation Assistance","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Autonomous driving, 4D radar, object detection, Semantic segmentation","lastPublishedDoi":"10.21203/rs.3.rs-5358941/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5358941/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"4D radar exhibits robustness to complex lighting and adverse weather conditions, providing unique data characteristics compared to LiDAR for 3D target detection. However, due to the sparsity of 4D radar point clouds, the performance of most 3D target detection algorithms is limited. To address this, this paper propose a 3D object detection model based on fine-grained point cloud segmentation. Our approach first enriches the point cloud data using a radar reference point module to compensate for its sparsity. The point cloud is then pillarized, and semantic information is extracted through a simple segmentation network. Finally, 3D object detection is achieved by fusing point cloud features and semantic information using an attention mechanism. Extensive experiments conducted on the VoD dataset demonstrate that our model achieves a mean average precision (mAP) that is 5\\% higher than the baseline on the validation set, with notable improvements of 4\\% for bicycles and 8\\% for pedestrians. These results narrow the performance gap with LiDAR-based models, highlighting the effectiveness of our segmentation-assisted detection approach.the source codes are released at https://github.com/Huniki/RVASANET.git","manuscriptTitle":"Enhanced 3D Object Detection using 4D Radar and Vision Fusion with Segmentation Assistance","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-12 06:35:47","doi":"10.21203/rs.3.rs-5358941/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e3a19022-1673-4346-a80a-5e7c711a13a2","owner":[],"postedDate":"November 12th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-04-05T16:38:20+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-12 06:35:47","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5358941","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5358941","identity":"rs-5358941","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}