Multimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning

Multimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Multimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning Shital R.Shegokar This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8966083/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Emotion recognition based on physiological activity has become a vital component of affective computing and human–machine interaction. Electroencephalography (EEG) offers direct neural insight into emotional processing but suffers from noise sensitivity, motion artifacts, and high inter-subject variability. Peripheral physiological signals such as electrocardiography (ECG), galvanic skin response (GSR), photoplethysmography (PPG), and respiration provide stable indicators of autonomic changes but lack the specificity necessary for distinguishing subtle emotional states. Multimodal fusion has emerged as a promising solution by integrating complementary bio signals to build more reliable and generalizable affect-recognition models. This study presents a comprehensive review of multimodal EEG–physiology fusion systems, examining fusion strategies, representation-learning approaches, robustness challenges, and cross-subject generalization. Evidence from recent literature shows that unimodal EEG systems typically achieve 60–72% accuracy, while multimodal fusion methods reach 82–90%, and advanced hybrid deep-fusion architectures exceed 92% in subject-independent evaluations. Key limitations persist, including heterogeneous feature spaces, misaligned modalities, sensor noise, and dataset inconsistencies. The synthesis identifies hybrid deep-learning fusion as the most resilient and effective strategy, establishing multimodal physiological learning as a foundational pathway toward future real-time, noise-robust, and generalizable emotion-recognition technologies. Biotechnology and Bioengineering Bioinformatics Biomedical Engineering Multimodal Fusion EEG Emotion Recognition Machine Learning Affective Computing Bio signal Processing. Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8966083","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":596830302,"identity":"578d760d-3989-4e68-9cdc-001e12f91a50","order_by":0,"name":"Shital R.Shegokar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA8klEQVRIiWNgGAWjYDACCcYGBoYCifo2/uYDIK4MMVoaGxgMLBj7JY4lgLg8RGhhAFpjUME4syHHAMQnrEV+dnP7gx8GEswGB858fnWjxoKHgf3w0Q34tBjcOdjY2GMgwWZwuHebdc4xoMN40tJu4NUikdjYwGMgwWNw4Ow24xw2oBYJHjO8WuRnJDY2/jGQkDA4kPPMOOcfEVoYbiQ2NgNtMZBsyGF+nNtGhBYDoJbZMgYSCfwSx8yYc/skeNgI+UV+RvqDj28q6hLY+Jsff875VifHz374GH6HIQE2CTBJrHIQYP5AiupRMApGwSgYOQAAndVJIjoukfkAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0003-9284-9334","institution":"Poornima University","correspondingAuthor":true,"prefix":"","firstName":"Shital","middleName":"","lastName":"R.Shegokar","suffix":""}],"badges":[],"createdAt":"2026-02-25 09:43:32","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":true,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":true},"doi":"10.21203/rs.3.rs-8966083/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8966083/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103507617,"identity":"f80797f5-c72c-4542-8313-4fdb21794842","added_by":"auto","created_at":"2026-02-26 13:42:27","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":461622,"visible":true,"origin":"","legend":"","description":"","filename":"MultimodalFusionofEEGandPhysiologicalSignalsforRobustEmotionRecognitionviaMachineLearning.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8966083/v1_covered_c0e6c60c-0b6d-4ad7-81a9-d410b2a9563a.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eMultimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Poornima University ","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Multimodal Fusion, EEG, Emotion Recognition, Machine Learning, Affective Computing, Bio signal Processing.","lastPublishedDoi":"10.21203/rs.3.rs-8966083/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8966083/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEmotion recognition based on physiological activity has become a vital component of affective computing and human\u0026ndash;machine interaction. Electroencephalography (EEG) offers direct neural insight into emotional processing but suffers from noise sensitivity, motion artifacts, and high inter-subject variability. Peripheral physiological signals such as electrocardiography (ECG), galvanic skin response (GSR), photoplethysmography (PPG), and respiration provide stable indicators of autonomic changes but lack the specificity necessary for distinguishing subtle emotional states. Multimodal fusion has emerged as a promising solution by integrating complementary bio signals to build more reliable and generalizable affect-recognition models. This study presents a comprehensive review of multimodal EEG\u0026ndash;physiology fusion systems, examining fusion strategies, representation-learning approaches, robustness challenges, and cross-subject generalization. Evidence from recent literature shows that unimodal EEG systems typically achieve 60\u0026ndash;72% accuracy, while multimodal fusion methods reach 82\u0026ndash;90%, and advanced hybrid deep-fusion architectures exceed 92% in subject-independent evaluations. Key limitations persist, including heterogeneous feature spaces, misaligned modalities, sensor noise, and dataset inconsistencies. The synthesis identifies hybrid deep-learning fusion as the most resilient and effective strategy, establishing multimodal physiological learning as a foundational pathway toward future real-time, noise-robust, and generalizable emotion-recognition technologies.\u003c/p\u003e","manuscriptTitle":"Multimodal Fusion of EEG and Physiological Signals for Robust Emotion Recognition via Machine Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-26 03:19:01","doi":"10.21203/rs.3.rs-8966083/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"63a78299-fa1f-4d3d-84ce-e80414842799","owner":[],"postedDate":"February 26th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":63503883,"name":"Biotechnology and Bioengineering"},{"id":63503884,"name":"Bioinformatics"},{"id":63503885,"name":"Biomedical Engineering"}],"tags":[],"updatedAt":"2026-02-26T03:19:01+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-26 03:19:01","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8966083","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8966083","identity":"rs-8966083","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}