A Dual-Branch Approach with Multi-Stage Semantic Integration and Dual Optical Flow for Micro- Expression Recognition

A Dual-Branch Approach with Multi-Stage Semantic Integration and Dual Optical Flow for Micro- Expression Recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Dual-Branch Approach with Multi-Stage Semantic Integration and Dual Optical Flow for Micro- Expression Recognition Shuhuan Zhao, Peijing Zhao, Zixin Hao, Shuaiqi Liu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5801228/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Aug, 2025 Read the published version in Multimedia Systems → Version 1 posted 10 You are reading this latest preprint version Abstract Micro-expression can reveal a person's true feelings and possesses significant importance in fields such as police interrogation and psychological counseling. However, due to the subtlety and complexity of micro-expression, comprehensively understanding its features remains a considerable challenge. To address this challenge, this paper proposes a dual-branch network structure that integrates traditional optical flow with deep learning-based optical flow. The micro-expression features are extracted and processed in separate branches, thereby harnessing the complementary advantages of the two optical flow methods. The first branch employs the proposed Multi-Scale Patch Attention Convolution Network (MPACNet), which is designed to process Farneback optical flow by capturing local details. The second branch utilizes the Swin Transformer network with FlowNet2 optical flow, demonstrating outstanding performance in extracting global dynamic information. In addition, this framework effectively combines local information from traditional convolutional networks with both local and global information from the Swin Transformer, achieving multi-level feature fusion. Following the standards of Comprehensive Database Evaluation (CDE) and Single Database Evaluation (SDE), extensive experiments have been conducted on four datasets—SMIC-HS, CASME II, SAMM, and CAS(ME) 3 . The results demonstrate that the proposed method outperforms other state-of-the-art approaches across various evaluation metrics. Micro-expression recognition deep learning dual-stream network attention mechanism multi-stage fusion Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 21 Aug, 2025 Read the published version in Multimedia Systems → Version 1 posted Editorial decision: Revision requested 04 May, 2025 Reviews received at journal 19 Apr, 2025 Reviewers agreed at journal 23 Mar, 2025 Reviews received at journal 22 Mar, 2025 Reviewers agreed at journal 21 Mar, 2025 Reviewers agreed at journal 21 Mar, 2025 Reviewers invited by journal 21 Mar, 2025 Editor assigned by journal 01 Feb, 2025 Submission checks completed at journal 10 Jan, 2025 First submitted to journal 10 Jan, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5801228","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":400562642,"identity":"42095220-e442-4e24-99ff-dff28f1a1eb4","order_by":0,"name":"Shuhuan Zhao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAArElEQVRIiWNgGAWjYFACHgZmxgYGOTb29gOkaTHm4zmTQJqWxHkSDgbEaZDvP3uAuXDH4fQ2CYYEhh8V2whrYZyRl8A888zh3DbpxgOMPWduE9bCLMFjwMzbBtQicyCBmbGNCC1s/GfAWtLZJBIMiNPCw5AD1pJAvBYJCZCWM+mGbcBAPkiUX+T7QQ7bYS0v395+8MGPCiK0AAH7DxjrAFHqR8EoGAWjYBQQBgAE+jUfCsI6mAAAAABJRU5ErkJggg==","orcid":"","institution":"Hebei University","correspondingAuthor":true,"prefix":"","firstName":"Shuhuan","middleName":"","lastName":"Zhao","suffix":""},{"id":400562643,"identity":"a1239838-b5e8-4f48-959f-1628f8989acd","order_by":1,"name":"Peijing Zhao","email":"","orcid":"","institution":"Hebei University","correspondingAuthor":false,"prefix":"","firstName":"Peijing","middleName":"","lastName":"Zhao","suffix":""},{"id":400562644,"identity":"ba9271d8-519f-4e1a-9418-385f90bc2685","order_by":2,"name":"Zixin Hao","email":"","orcid":"","institution":"Hebei University","correspondingAuthor":false,"prefix":"","firstName":"Zixin","middleName":"","lastName":"Hao","suffix":""},{"id":400562645,"identity":"2604f4bf-8cb3-4974-8fd4-034372ed8157","order_by":3,"name":"Shuaiqi Liu","email":"","orcid":"","institution":"Hebei University","correspondingAuthor":false,"prefix":"","firstName":"Shuaiqi","middleName":"","lastName":"Liu","suffix":""}],"badges":[],"createdAt":"2025-01-10 06:53:39","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5801228/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5801228/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s00530-025-01933-z","type":"published","date":"2025-08-21T16:28:55+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":89847155,"identity":"25231650-92cb-44fe-98e9-728d90b2ff28","added_by":"auto","created_at":"2025-08-25 16:41:22","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":640393,"visible":true,"origin":"","legend":"","description":"","filename":"25.1.10.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5801228/v1_covered_1a4a4da0-22e7-4888-b131-254856fb686c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Dual-Branch Approach with Multi-Stage Semantic Integration and Dual Optical Flow for Micro- Expression Recognition","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"multimedia-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mmsj","sideBox":"Learn more about [Multimedia Systems](http://link.springer.com/journal/530)","snPcode":"530","submissionUrl":"https://submission.nature.com/new-submission/530/3","title":"Multimedia Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Micro-expression recognition, deep learning, dual-stream network, attention mechanism, multi-stage fusion","lastPublishedDoi":"10.21203/rs.3.rs-5801228/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5801228/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eMicro-expression can reveal a person's true feelings and possesses significant importance in fields such as police interrogation and psychological counseling. However, due to the subtlety and complexity of micro-expression, comprehensively understanding its features remains a considerable challenge. To address this challenge, this paper proposes a dual-branch network structure that integrates traditional optical flow with deep learning-based optical flow. The micro-expression features are extracted and processed in separate branches, thereby harnessing the complementary advantages of the two optical flow methods. The first branch employs the proposed Multi-Scale Patch Attention Convolution Network (MPACNet), which is designed to process Farneback optical flow by capturing local details. The second branch utilizes the Swin Transformer network with FlowNet2 optical flow, demonstrating outstanding performance in extracting global dynamic information. In addition, this framework effectively combines local information from traditional convolutional networks with both local and global information from the Swin Transformer, achieving multi-level feature fusion. Following the standards of Comprehensive Database Evaluation (CDE) and Single Database Evaluation (SDE), extensive experiments have been conducted on four datasets\u0026mdash;SMIC-HS, CASME II, SAMM, and CAS(ME)\u003csup\u003e3\u003c/sup\u003e. The results demonstrate that the proposed method outperforms other state-of-the-art approaches across various evaluation metrics.\u003c/p\u003e","manuscriptTitle":"A Dual-Branch Approach with Multi-Stage Semantic Integration and Dual Optical Flow for Micro- Expression Recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-13 16:59:10","doi":"10.21203/rs.3.rs-5801228/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-05-04T09:58:06+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-19T16:46:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"48465019394418395769701386321754128659","date":"2025-03-24T01:42:46+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-03-22T08:07:45+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"163467988181840516775393825277127003430","date":"2025-03-22T01:46:11+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"241364127091948689354769851324936535766","date":"2025-03-22T01:43:48+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-22T01:39:55+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-02-01T14:46:33+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-01-10T15:35:34+00:00","index":"","fulltext":""},{"type":"submitted","content":"Multimedia Systems","date":"2025-01-10T06:51:18+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"multimedia-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mmsj","sideBox":"Learn more about [Multimedia Systems](http://link.springer.com/journal/530)","snPcode":"530","submissionUrl":"https://submission.nature.com/new-submission/530/3","title":"Multimedia Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"3574b301-6b22-45b0-b71b-a16a9c9689a7","owner":[],"postedDate":"January 13th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-08-25T16:31:29+00:00","versionOfRecord":{"articleIdentity":"rs-5801228","link":"https://doi.org/10.1007/s00530-025-01933-z","journal":{"identity":"multimedia-systems","isVorOnly":false,"title":"Multimedia Systems"},"publishedOn":"2025-08-21 16:28:55","publishedOnDateReadable":"August 21st, 2025"},"versionCreatedAt":"2025-01-13 16:59:10","video":"","vorDoi":"10.1007/s00530-025-01933-z","vorDoiUrl":"https://doi.org/10.1007/s00530-025-01933-z","workflowStages":[]},"version":"v1","identity":"rs-5801228","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5801228","identity":"rs-5801228","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}