A deep learning framework for micro-expression recognition via multi-feature fusion

A deep learning framework for micro-expression recognition via multi-feature fusion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A deep learning framework for micro-expression recognition via multi-feature fusion Yupeng Liu, Weinan Zheng, Ying Du, Yuehui Wang, Jian Jin, Miao Yu This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6620715/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In order to enhance the quality of inclusive education and promptly assess the knowledge comprehension of special education students through their facial expression changes during lectures, it is essential to recognize micro-expressions. However, the subtle variations in micro-expressions pose challenges to detection. To enhance micro-expression recognition, a learning method combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks for spatiotemporal feature extraction and recognition is proposed. Initially, CNN extracts spatial features from the dataset using a dual-channel configuration, followed by LSTM to process temporal features. The sequence is looped with a fixed frame count. Experiments are performed on a test dataset to compare the recognition efficiency of the proposed algorithm with several other algorithms using the existing database. The results demonstrate that the proposed method achieves a comprehensive micro-expression recognition efficiency of 76.24%, representing a significant improvement. Physical sciences/Engineering Physical sciences/Engineering/Mechanical engineering Facial micro-expressions Convolutional neural networks Long short-term memory Spatiotemporal feature learning Affective computing Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6620715","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":474704024,"identity":"bd1ac46b-8738-4ec9-b154-cbeb3cae1553","order_by":0,"name":"Yupeng Liu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1ElEQVRIiWNgGAWjYBACNvnDBx9+/GPDwy//+ABxWvgk2JKNJRvSZIA4gTgtchI8ZhK8DYdsDBpyDIh0mHRbmoTkjgM8BgxnPt54w2Anp9tASIvM4cMWhWfu8Jgz9m62nMOQbGx2gJAWhrTEGxJsz3gsm3m3SfMwHEjcRlhLjoEED9thHoNjPM+I1CKRYyTB2wbUcoaHjUgtPMeSjSXOpPFIzmAztpxjQIRf5NubDz78UGFjzy/B/PDGmwo7OYJaUIAED5FRg6yFVB2jYBSMglEwIgAAEJA/wap8C3oAAAAASUVORK5CYII=","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":true,"prefix":"","firstName":"Yupeng","middleName":"","lastName":"Liu","suffix":""},{"id":474704025,"identity":"7700068b-efa1-4c01-9a18-56dc1fca6505","order_by":1,"name":"Weinan Zheng","email":"","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":false,"prefix":"","firstName":"Weinan","middleName":"","lastName":"Zheng","suffix":""},{"id":474704027,"identity":"d06518ae-e0a1-42c3-ab3b-f1ec30817aaa","order_by":2,"name":"Ying Du","email":"","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":false,"prefix":"","firstName":"Ying","middleName":"","lastName":"Du","suffix":""},{"id":474704028,"identity":"886e6379-7a5f-4c14-a0f0-a7ef8107e467","order_by":3,"name":"Yuehui Wang","email":"","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":false,"prefix":"","firstName":"Yuehui","middleName":"","lastName":"Wang","suffix":""},{"id":474704029,"identity":"ed2ffcb1-d0d6-49c6-a41f-e6165ec66628","order_by":4,"name":"Jian Jin","email":"","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jian","middleName":"","lastName":"Jin","suffix":""},{"id":474704030,"identity":"cdb367dd-651b-4f6c-a70a-8aaffd126fa9","order_by":5,"name":"Miao Yu","email":"","orcid":"","institution":"Jilin University of Architecture and Technology","correspondingAuthor":false,"prefix":"","firstName":"Miao","middleName":"","lastName":"Yu","suffix":""}],"badges":[],"createdAt":"2025-05-08 12:38:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6620715/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6620715/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":86225821,"identity":"186bea22-da5c-4400-a02a-d321278569b3","added_by":"auto","created_at":"2025-07-08 08:02:32","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":609703,"visible":true,"origin":"","legend":"","description":"","filename":"Adeeplearningframeworkformicroexpressionrecognitionviamultifeaturefusion.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6620715/v1_covered_6d07fa80-8549-41f2-b610-4d142d19ea3e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eA deep learning framework for micro-expression recognition via multi-feature fusion\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Facial micro-expressions, Convolutional neural networks, Long short-term memory, Spatiotemporal feature learning, Affective computing","lastPublishedDoi":"10.21203/rs.3.rs-6620715/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6620715/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn order to enhance the quality of inclusive education and promptly assess the knowledge comprehension of special education students through their facial expression changes during lectures, it is essential to recognize micro-expressions. However, the subtle variations in micro-expressions pose challenges to detection. To enhance micro-expression recognition, a learning method combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks for spatiotemporal feature extraction and recognition is proposed. Initially, CNN extracts spatial features from the dataset using a dual-channel configuration, followed by LSTM to process temporal features. The sequence is looped with a fixed frame count. Experiments are performed on a test dataset to compare the recognition efficiency of the proposed algorithm with several other algorithms using the existing database. The results demonstrate that the proposed method achieves a comprehensive micro-expression recognition efficiency of 76.24%, representing a significant improvement.\u003c/p\u003e","manuscriptTitle":"A deep learning framework for micro-expression recognition via multi-feature fusion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-23 15:27:28","doi":"10.21203/rs.3.rs-6620715/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"35d2fd83-c724-467d-8cb9-3d14e1088554","owner":[],"postedDate":"June 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":50466984,"name":"Physical sciences/Engineering"},{"id":50466985,"name":"Physical sciences/Engineering/Mechanical engineering"}],"tags":[],"updatedAt":"2025-07-08T07:54:19+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-23 15:27:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6620715","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6620715","identity":"rs-6620715","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}