Towards Human-Centered and Efficient Video Synthesis: A Survey of Multimodal Diffusion Models

Towards Human-Centered and Efficient Video Synthesis: A Survey of Multimodal Diffusion Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Towards Human-Centered and Efficient Video Synthesis: A Survey of Multimodal Diffusion Models Alaa Abdullah Albaghdadi, Ahmad R. Naghsh-Nilchi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7533477/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Multimodal video diffusion models have emerged as transformative tools for controlled video synthesis, integrating text, images, audio, and pose sequences to generate semantically meaningful content. Despite significant advances, critical gaps persist in temporal consistency, multimodal alignment, and human-centric motion generation. Existing surveys have not addressed clearly the complex interplay between these components, particularly physiological constraints and identity preservation in human motion synthesis. This survey provides a comprehensive analysis through a unified architectural framework, examining spatial-temporal representations and multimodal conditioning mechanisms. We present the first systematic evaluation of human-centric motion modeling, addressing physiological plausibility and identity consistency challenges. Our analysis reveals fundamental trade-offs between computational efficiency and generation quality, demonstrating that specialized techniques like temporal block pruning achieve 523× computational savings with minimal quality degradation. Key findings indicate that current approaches struggle with seamless multimodal integration, human-centric applications face "uncanny valley" effects when physics constraints are too rigid, and identity preservation conflicts with motion dynamics. We introduce MIME-Vid (Multi-modal Integration with Motion Enhancement for Video Generation), a novel framework that integrates advanced Kalman filtering techniques with multi-modal architecture for enhanced temporal consistency and motion realism. Furthermore, we propose novel evaluation paradigms and identify future research directions for advancing multimodal video generation Diffusion models multimodal synthesis video generation human-centric AI temporal modeling generative models controllability reinforcement learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 26 Mar, 2026 Reviews received at journal 26 Dec, 2025 Reviewers agreed at journal 09 Dec, 2025 Reviewers agreed at journal 04 Dec, 2025 Reviews received at journal 25 Oct, 2025 Reviewers agreed at journal 10 Oct, 2025 Reviewers agreed at journal 27 Sep, 2025 Reviewers invited by journal 25 Sep, 2025 Editor assigned by journal 18 Sep, 2025 Submission checks completed at journal 16 Sep, 2025 First submitted to journal 04 Sep, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7533477","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":524801877,"identity":"15aab66b-591a-4e04-9143-0fcd476b4981","order_by":0,"name":"Alaa Abdullah Albaghdadi","email":"","orcid":"","institution":"Isfahan University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Alaa","middleName":"Abdullah","lastName":"Albaghdadi","suffix":""},{"id":524801878,"identity":"4e5085c8-9b25-406b-b598-84d2540ce777","order_by":1,"name":"Ahmad R. Naghsh-Nilchi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAx0lEQVRIiWNgGAWjYDACZuYGZgYDGwgnASxCUAsjSEsaKVoYQFoYDpPgLnN2xsbPBQXn5cz5F7BJPGCwk2dg532AV4tlM2Oz9AyD28aWMx6wSSQwJBs2MLMb4NVicJixjZnH4HbihhsHQFqYgYgNv8OgWs7VQ7XUE63lQILB+QaQlsNEaWmW5jFINtxwg7HZIsHguGEbQS3nDx/8zPPHTh7EuPmjolqen/8Yfi0IIJHYADSBgYGAHciA/wDxakfBKBgFo2BkAQDHdDhA/hnIpgAAAABJRU5ErkJggg==","orcid":"","institution":"Isfahan University of Technology","correspondingAuthor":true,"prefix":"","firstName":"Ahmad","middleName":"R.","lastName":"Naghsh-Nilchi","suffix":""}],"badges":[],"createdAt":"2025-09-04 07:53:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7533477/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7533477/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92923864,"identity":"996404ad-9c5d-40c7-99c3-3fb194a2de7c","added_by":"auto","created_at":"2025-10-07 07:32:24","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5020,"visible":true,"origin":"","legend":"","description":"","filename":"7bff36620d6f491bb58e445235d8669f.json","url":"https://assets-eu.researchsquare.com/files/rs-7533477/v1/6ca66351f8fcb7b11e9060b8.json"},{"id":92924976,"identity":"9734e43f-f9da-4894-ba92-802388f76809","added_by":"auto","created_at":"2025-10-07 07:40:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5636189,"visible":true,"origin":"","legend":"","description":"","filename":"videodiffusionmodelsurvey.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7533477/v1_covered_b2022f83-de0e-4a06-8dca-62f17eda4d67.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Towards Human-Centered and Efficient Video Synthesis: A Survey of Multimodal Diffusion Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"artificial-intelligence-review","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"aire","sideBox":"Learn more about [Artificial Intelligence Review](http://link.springer.com/journal/10462)","snPcode":"10462","submissionUrl":"https://submission.nature.com/new-submission/10462/3","title":"Artificial Intelligence Review","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Diffusion models, multimodal synthesis, video generation, human-centric AI, temporal modeling, generative models, controllability, reinforcement learning","lastPublishedDoi":"10.21203/rs.3.rs-7533477/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7533477/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Multimodal video diffusion models have emerged as transformative tools for controlled video synthesis, integrating text, images, audio, and pose sequences to generate semantically meaningful content. Despite significant advances, critical gaps persist in temporal consistency, multimodal alignment, and human-centric motion generation. Existing surveys have not addressed clearly the complex interplay between these components, particularly physiological constraints and identity preservation in human motion synthesis. This survey provides a comprehensive analysis through a unified architectural framework, examining spatial-temporal representations and multimodal conditioning mechanisms. We present the first systematic evaluation of human-centric motion modeling, addressing physiological plausibility and identity consistency challenges. Our analysis reveals fundamental trade-offs between computational efficiency and generation quality, demonstrating that specialized techniques like temporal block pruning achieve 523× computational savings with minimal quality degradation. Key findings indicate that current approaches struggle with seamless multimodal integration, human-centric applications face \"uncanny valley\" effects when physics constraints are too rigid, and identity preservation conflicts with motion dynamics. We introduce MIME-Vid (Multi-modal Integration with Motion Enhancement for Video Generation), a novel framework that integrates advanced Kalman filtering techniques with multi-modal architecture for enhanced temporal consistency and motion realism. Furthermore, we propose novel evaluation paradigms and identify future research directions for advancing multimodal video generation","manuscriptTitle":"Towards Human-Centered and Efficient Video Synthesis: A Survey of Multimodal Diffusion Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-07 07:32:19","doi":"10.21203/rs.3.rs-7533477/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-26T07:53:26+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-26T12:41:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"141608433062850552834444955934803440138","date":"2025-12-09T18:54:34+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"251646907163279652013105499333177015721","date":"2025-12-04T11:54:41+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-26T01:49:23+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"159039539871478837508717636555808248320","date":"2025-10-10T06:54:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"128966461046323930301604651925108785655","date":"2025-09-27T18:57:16+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-25T04:22:28+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-18T13:04:54+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-16T12:52:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"Artificial Intelligence Review","date":"2025-09-04T07:48:02+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"artificial-intelligence-review","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"aire","sideBox":"Learn more about [Artificial Intelligence Review](http://link.springer.com/journal/10462)","snPcode":"10462","submissionUrl":"https://submission.nature.com/new-submission/10462/3","title":"Artificial Intelligence Review","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"a5f8c62b-b776-4d12-955b-272e0e0dfb6f","owner":[],"postedDate":"October 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-25T08:08:35+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-07 07:32:19","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7533477","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7533477","identity":"rs-7533477","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}