MvDeDiffusion: Multi-view Consistent Generation via Cross-view Deformable Attention for Denoising Diffusion Models

MvDeDiffusion: Multi-view Consistent Generation via Cross-view Deformable Attention for Denoising Diffusion Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article MvDeDiffusion: Multi-view Consistent Generation via Cross-view Deformable Attention for Denoising Diffusion Models Bin Lu, Qing Li, Yanju Liang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7322439/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Denoising diffusion models have demonstrated remarkable success in image generation, with numerous approaches achieving state-of-the-art synthesis quality. For autonomous driving applications, there is a critical need to extend these capabilities to multi-view image generation. However, achieving precise multi-view-consistent generation with 3D geometric awareness, critical for 3D perception tasks, remains challenging. Current approaches predominantly rely on overhead layout guidance, yet they frequently fail to maintain cross-view geometric coherence. This limitation manifests as misaligned object structures, discontinuous occlusions, and inconsistent depth relationships when synthesizing scenes from multiple angles. In this paper, we propose MvDeDiffusion, a diffusion-based framework for 3D-consistent multi-view image synthesis, which introduces two key innovations: (1) a cross-view deformable attention mechanism that explicitly enforces geometric and appearance consistency between adjacent viewpoints by adaptively aligning features domain in the denoising process, (2) a 3D-aware conditioning pipeline that integrates camera poses, foreground positional information, adjacent-view overlap to enable fine-grained control over scene structure while preserving photorealistic details. Our framework ensures view-consistent generation through explicit modeling of inter-perspective correlations during the diffusion process, overcoming the inherent limitations of independent per-view synthesis. Comprehensive experiments demonstrate that our model achieves:(1) superior multi-view continuity through geometrically coherent image synthesis,(2) maximizing controllability while preserving the richness of generated scenes.These advancements are quantitatively verified to significantly outperform existing approaches in both cross-view alignment fidelity and scene variation richness. Autonomous driving Denoising diffusion Deformable attention Cross-view consistency Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 19 Aug, 2025 Reviewers invited by journal 19 Aug, 2025 Editor assigned by journal 12 Aug, 2025 Submission checks completed at journal 12 Aug, 2025 First submitted to journal 07 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7322439","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":502336016,"identity":"4d4625ac-a23b-42b4-ba3a-454c4051334a","order_by":0,"name":"Bin Lu","email":"","orcid":"","institution":"*Institute of Microelectronics of the Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Bin","middleName":"","lastName":"Lu","suffix":""},{"id":502336018,"identity":"d26875b0-71b4-4f6b-aefd-d6f2716183fc","order_by":1,"name":"Qing Li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtElEQVRIiWNgGAWjYBACPmYGhgMMDDY8xGthg2hJI0ULhDpMgsPY2Jk3Hi74dV5Gt72B8cMPBrs8IhzGVnB4Zt9tHrMzB5glexiSi4nQwmNwmLcHqOVGAoM00F+JDURqOcdjdv8B82/itfD8OAC0hYGNWFuAfuFtSAb6JbHNsscgmbAWfv7Dmz/z/LGzNzt++PCNHxV2hLUAgQEDYxuIZmwAsYkCQGV/iFM5CkbBKBgFIxQAAKoRNqM7ALj6AAAAAElFTkSuQmCC","orcid":"","institution":"*Institute of Microelectronics of the Chinese Academy of Sciences","correspondingAuthor":true,"prefix":"","firstName":"Qing","middleName":"","lastName":"Li","suffix":""},{"id":502336019,"identity":"4060dc85-8fe7-4d2f-96f0-9c7e4806df69","order_by":2,"name":"Yanju Liang","email":"","orcid":"","institution":"*Institute of Microelectronics of the Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Yanju","middleName":"","lastName":"Liang","suffix":""}],"badges":[],"createdAt":"2025-08-08 01:23:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7322439/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7322439/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":89969028,"identity":"c9848f7b-5ba3-49fd-8e70-5d3354423bcc","added_by":"auto","created_at":"2025-08-27 04:44:18","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1181631,"visible":true,"origin":"","legend":"","description":"","filename":"MvDeDiffusion.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7322439/v1_covered_47c45b77-423c-4864-899c-69cbbe55b830.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"MvDeDiffusion: Multi-view Consistent Generation via Cross-view Deformable Attention for Denoising Diffusion Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Autonomous driving, Denoising diffusion, Deformable attention, Cross-view consistency","lastPublishedDoi":"10.21203/rs.3.rs-7322439/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7322439/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eDenoising diffusion models have demonstrated remarkable success in image generation, with numerous approaches achieving state-of-the-art synthesis quality. For autonomous driving applications, there is a critical need to extend these capabilities to multi-view image generation. However, achieving precise multi-view-consistent generation with 3D geometric awareness, critical for 3D perception tasks, remains challenging. Current approaches predominantly rely on overhead layout guidance, yet they frequently fail to maintain cross-view geometric coherence. This limitation manifests as misaligned object structures, discontinuous occlusions, and inconsistent depth relationships when synthesizing scenes from multiple angles. In this paper, we propose MvDeDiffusion, a diffusion-based framework for 3D-consistent multi-view image synthesis, which introduces two key innovations: (1) a cross-view deformable attention mechanism that explicitly enforces geometric and appearance consistency between adjacent viewpoints by adaptively aligning features domain in the denoising process, (2) a 3D-aware conditioning pipeline that integrates camera poses, foreground positional information, adjacent-view overlap to enable fine-grained control over scene structure while preserving photorealistic details. Our framework ensures view-consistent generation through explicit modeling of inter-perspective correlations during the diffusion process, overcoming the inherent limitations of independent per-view synthesis. Comprehensive experiments demonstrate that our model achieves:(1) superior multi-view continuity through geometrically coherent image synthesis,(2) maximizing controllability while preserving the richness of generated scenes.These advancements are quantitatively verified to significantly outperform existing approaches in both cross-view alignment fidelity and scene variation richness.\u003c/p\u003e","manuscriptTitle":"MvDeDiffusion: Multi-view Consistent Generation via Cross-view Deformable Attention for Denoising Diffusion Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-27 04:20:12","doi":"10.21203/rs.3.rs-7322439/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-08-19T06:06:52+00:00","index":"","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-08-19T06:06:08+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-12T09:40:17+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-12T09:38:40+00:00","index":"","fulltext":""},{"type":"submitted","content":"Signal, Image and Video Processing","date":"2025-08-08T01:14:45+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"92346f3b-455b-4263-80c0-66a3cd4d1d3d","owner":[],"postedDate":"August 27th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-08-31T23:53:09+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-27 04:20:12","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7322439","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7322439","identity":"rs-7322439","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}