Multimodal UAV Object Detection Based on Progressive Bi-directional Attention and Relational Bayesian Constraints

Multimodal UAV Object Detection Based on Progressive Bi-directional Attention and Relational Bayesian Constraints | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Multimodal UAV Object Detection Based on Progressive Bi-directional Attention and Relational Bayesian Constraints Zihan Wu, Fei Zhang, Bin Yang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9380305/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract To address the inadequate robustness of single-modality detection caused by severe scale variations and complex illumination in UAV imagery, as well as the feature degradation induced by forced alignment in existing cross-modal fusion networks, this paper proposes a multimodal object detection network based on progressive bi-directional attention and relational Bayesian constraints. Based on YOLOv12, a dual-stream architecture is constructed to decouple RGB and infrared (IR) features. We design a Progressive Design with Bi-Attention (PDBA) to capture multi-scale complementary information via adaptive down-sampled cross-modal interactions. Additionally, a Multi-modal Relational Bayesian Loss (MRBLoss) is introduced to dynamically model inter-modal uncertainty using posterior probabilities, guiding reasonable semantic alignment in the deep latent space. Experimental results show that under the Medium scale, our model achieves an mAP 50 of 75.1% on the VEDAI dataset at a resolution of 640×640. Furthermore, cross-capacity (Nano vs. Medium) ablation studies on the DroneVehicle dataset reveal that model scaling yields noticeably different performance gains depending on feature richness (+30.8% vs. +1.6%). This paper analyzes the feature homogenization phenomenon in multimodal fusion, demonstrating that the effectiveness of cross-modal joint optimization strategies is constrained by the base network's capacity and the richness of target prior features. This finding provides a theoretical reference for the design of multimodal alignment in small object detection and lightweight networks. UAV object detection YOLOv12 multimodal fusion progressive bi-directional attention relational Bayesian constraint feature homogenization Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9380305","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":624995287,"identity":"ce30bc74-a6bd-47ef-9316-73b58773ff5f","order_by":0,"name":"Zihan Wu","email":"","orcid":"","institution":"Nanfang College, Guangzhou","correspondingAuthor":false,"prefix":"","firstName":"Zihan","middleName":"","lastName":"Wu","suffix":""},{"id":624995288,"identity":"9ec5b493-a0d3-4c94-8e6c-88296b12a251","order_by":1,"name":"Fei Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIiWNgGAWjYDACCRBhw8DDz8D4AMhiJlZLGgOPZAOzAWlaGAwOEKtFfnbzs4dfEmxkjG83s0kwVFgnNrCfPYBXC+OcY+bGMglpPGZ3DgO1nElPbODJS8CrhVkiwUxa8sdhHrMb+cckGNsOJzZI8Bjg1cImkf5NWiLhP4/xjGQ2CcZ/RGjhkcgxk/yQcIDHQAKkpYEILRISOWXSDAnJPBI3kpktEo6lG7fx5ODXIj8jfZvkjwQ7e/4ZyYw3PtRYy/azn8GvBQSYeWCsBJDvCKoHAsYfxKgaBaNgFIyCkQsAV4o7dEqfe4cAAAAASUVORK5CYII=","orcid":"","institution":"Nanfang College, Guangzhou","correspondingAuthor":true,"prefix":"","firstName":"Fei","middleName":"","lastName":"Zhang","suffix":""},{"id":624995289,"identity":"07aaaa7d-dffa-493a-89c5-e4e5dcdc5d70","order_by":2,"name":"Bin Yang","email":"","orcid":"","institution":"Nanfang College, Guangzhou","correspondingAuthor":false,"prefix":"","firstName":"Bin","middleName":"","lastName":"Yang","suffix":""}],"badges":[],"createdAt":"2026-04-10 13:53:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9380305/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9380305/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":109240026,"identity":"c74a201b-01db-4d29-9b59-fa26d22a26f7","added_by":"auto","created_at":"2026-05-14 06:25:31","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1016669,"visible":true,"origin":"","legend":"","description":"","filename":"SpringerLaTex.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9380305/v1_covered_a5904bcd-77c3-4321-b789-ba8e50cbff8b.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Multimodal UAV Object Detection Based on Progressive Bi-directional Attention and Relational Bayesian Constraints","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"UAV object detection, YOLOv12, multimodal fusion, progressive bi-directional attention, relational Bayesian constraint, feature homogenization","lastPublishedDoi":"10.21203/rs.3.rs-9380305/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9380305/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTo address the inadequate robustness of single-modality detection caused by severe scale variations and complex illumination in UAV imagery, as well as the feature degradation induced by forced alignment in existing cross-modal fusion networks, this paper proposes a multimodal object detection network based on progressive bi-directional attention and relational Bayesian constraints. Based on YOLOv12, a dual-stream architecture is constructed to decouple RGB and infrared (IR) features. We design a Progressive Design with Bi-Attention (PDBA) to capture multi-scale complementary information via adaptive down-sampled cross-modal interactions. Additionally, a Multi-modal Relational Bayesian Loss (MRBLoss) is introduced to dynamically model inter-modal uncertainty using posterior probabilities, guiding reasonable semantic alignment in the deep latent space. Experimental results show that under the Medium scale, our model achieves an mAP\u003csub\u003e\u003cstrong\u003e50\u003c/strong\u003e\u003c/sub\u003e of 75.1% on the VEDAI dataset at a resolution of \u003cstrong\u003e640×640.\u003c/strong\u003e \u0026nbsp;Furthermore, cross-capacity (Nano vs. Medium) ablation studies on the DroneVehicle dataset reveal that model scaling yields noticeably different performance gains depending on feature richness (+30.8% vs. +1.6%). This paper analyzes the feature homogenization phenomenon in multimodal fusion, demonstrating that the effectiveness of cross-modal joint optimization strategies is constrained by the base network's capacity and the richness of target prior features. This finding provides a theoretical reference for the design of multimodal alignment in small object detection and lightweight networks.\u003c/p\u003e","manuscriptTitle":"Multimodal UAV Object Detection Based on Progressive Bi-directional Attention and Relational Bayesian Constraints","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-24 04:47:26","doi":"10.21203/rs.3.rs-9380305/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cc3d617b-ccd7-48f7-a192-297ce70c7e02","owner":[],"postedDate":"April 24th, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Rejected","date":"2026-05-14T06:08:18+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-05T02:23:26+00:00","index":18,"fulltext":""},{"type":"reviewerAgreed","content":"278597047324131923090306941087266666175","date":"2026-04-29T13:17:16+00:00","index":16,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-14T06:24:25+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-24 04:47:26","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9380305","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9380305","identity":"rs-9380305","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}