Hierarchical Fusion with Decision Enhancement for Human Activity Recognition Using Deep Learning Framework

Hierarchical Fusion with Decision Enhancement for Human Activity Recognition Using Deep Learning Framework | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Hierarchical Fusion with Decision Enhancement for Human Activity Recognition Using Deep Learning Framework Samatha R. Swamy, Nandini K Prasad This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7949557/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Human Activity Recognition (HAR) is emerging as a critical enabler of context-aware applications in healthcare, fitness, and smart environments. In this research we present an approach that involves the Hierarchical Fusion with Decision Enhancement for Human Activity Recognition (Hi-FuseDE-HAR) framework. It contains four sequential hierarchical levels that transform raw sensor signals into a reliable HAR decision. At the input level, heterogeneous data streams are obtained from multiple wearable and ambient sensors. Applying Level 0 build the discriminative latent embedding for each sensor modality separately. This involves a number of CNN-based or Transformer-based encoders to transform each sensor dimension from raw to latent embedding. Level 1 fuses across sensors in groupings and determines the value of using each modality and determines its relative contribution to the overall feature importance. Level 2 applies a Graph Cross-Modal Transformer that learns relationship between sensors groups producing a globally consistent fused representation. Level 3 provides decision enhancement through uncertainty calibration and utility aware optimization to ensure the final estimates based. Experimental results indicate that the proposed framework achieves 97.6% accuracy and 96.7% F1-score on the PAMAP2 dataset, 95.5% accuracy and 93.2% F1-score on the OPPORTUNITY dataset, and 96.5% accuracy and 95.2% F1-score on the MHEALTH dataset respectively. Notably, Hi-FuseDE-HAR retains strong performance confirming its capability to generalize across varied sensor contexts and complex activity patterns. Biological sciences/Computational biology and bioinformatics Physical sciences/Engineering Health sciences/Health care Physical sciences/Mathematics and computing Deep Learning Graph Neural Networks Human Activity Recognition Multi-Sensor Data Fusion Self-Supervised Learning Transformer Networks Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7949557","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":566348828,"identity":"682c351f-c619-484e-9c3d-145e095075e4","order_by":0,"name":"Samatha R. Swamy","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABD0lEQVRIiWNgGAWjYJADNgYGAxs5EOvAA+J0MAO1FKQZg7UkEK/lw+HEBhAbnxbz/jOGn2621cnrtp8/9uCHAXP6/LDDD4G22MnpNmDXInMjx1g6t+2w4bYzyeyGPQZsuRtvpxkAtSQbmx3ArkVCgncDUMsBxm0HktkkeAx4cjfOTgBpOZC4DZcW/rObf+e21dlvO/+YTfKPgUS64ez0D/i1MORuA9rCnLjtRjKbNI+BQYK8dA4BWyTyv1nnnDucvO3GYzNpGYMEww3SOQUHEgzw+IX/WPLtnLI6223nE59JvvnzX15+dvrmDx8q7ORwaQEDRjYkjgFYpQEe5WDwB4kt30BI9SgYBaNgFIw0AADTIWLzj0gv4gAAAABJRU5ErkJggg==","orcid":"","institution":"Dayananda Sagar Academy of Technology and Management","correspondingAuthor":true,"prefix":"","firstName":"Samatha","middleName":"R.","lastName":"Swamy","suffix":""},{"id":566348832,"identity":"c5f0ebf6-c701-448b-be33-5ebff4b70d56","order_by":1,"name":"Nandini K Prasad","email":"","orcid":"","institution":"Dayananda Sagar Academy of Technology and Management","correspondingAuthor":false,"prefix":"","firstName":"Nandini","middleName":"K","lastName":"Prasad","suffix":""}],"badges":[],"createdAt":"2025-10-27 11:06:23","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7949557/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7949557/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99217258,"identity":"99fe39c5-30fe-411e-b4eb-7842b0989e01","added_by":"auto","created_at":"2025-12-30 09:10:21","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":380373,"visible":true,"origin":"","legend":"","description":"","filename":"HiFuseDEV2.docx","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/af550645339f6bfebf9f869d.docx"},{"id":99217416,"identity":"c728ae75-6ca4-4ddb-ab94-4bfe874756c9","added_by":"auto","created_at":"2025-12-30 09:10:34","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4403,"visible":true,"origin":"","legend":"","description":"","filename":"2d4696a419ec4751b4bbf4c1deb1080b.json","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/b75789910a4f9905379d75c2.json"},{"id":99217239,"identity":"f849faea-e391-4376-b641-d38a5232b1a7","added_by":"auto","created_at":"2025-12-30 09:10:18","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":129081,"visible":true,"origin":"","legend":"","description":"","filename":"2d4696a419ec4751b4bbf4c1deb1080b1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/5d763c57b1400ac67cacb587.xml"},{"id":99217181,"identity":"b058208e-43cd-4344-8fe0-cf4db2904bfc","added_by":"auto","created_at":"2025-12-30 09:10:15","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":206016,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/1332274f3352a5ec15ca5c50.png"},{"id":99217263,"identity":"8e8ca944-2756-4336-9a44-0b6a592a01ff","added_by":"auto","created_at":"2025-12-30 09:10:27","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":26091,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/477bd2faabd6666bf16effbc.png"},{"id":99217371,"identity":"4de794c9-5d41-4fb9-a39c-7b17ed0058a9","added_by":"auto","created_at":"2025-12-30 09:10:32","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":27030,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/29ba1926ca4abd28eeed6904.png"},{"id":99217233,"identity":"bcc4a81f-7dc2-49e7-8abd-681a9622907e","added_by":"auto","created_at":"2025-12-30 09:10:18","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":26776,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/e1cd0aafcce643f224ebf560.png"},{"id":99217413,"identity":"81b59a1a-dfd8-4325-be33-7bdce1eaaa79","added_by":"auto","created_at":"2025-12-30 09:10:34","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":28160,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/155179459f8fa00021ff5326.png"},{"id":99217419,"identity":"05071e17-5f57-4788-b922-df03115ead3c","added_by":"auto","created_at":"2025-12-30 09:10:35","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":18148,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/5fe17fb46af0b208e401eb50.png"},{"id":99217231,"identity":"e7b21ee1-1657-4109-9d46-87487d631da0","added_by":"auto","created_at":"2025-12-30 09:10:18","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9289,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/22f8cb1afa2ca70c3f85fb84.png"},{"id":99217256,"identity":"c904af58-fbf1-40cf-9b82-616c43c6270f","added_by":"auto","created_at":"2025-12-30 09:10:21","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9824,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/041c6917bcc3f3ddbea51726.png"},{"id":99217234,"identity":"f2180795-e990-44e5-a01f-b35ef4c872c6","added_by":"auto","created_at":"2025-12-30 09:10:18","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9235,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/45df3e7708e9afa362206014.png"},{"id":99217245,"identity":"811b911f-4fb7-453a-8cab-423ebf1bfe9c","added_by":"auto","created_at":"2025-12-30 09:10:19","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":12748,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/6c530bf3f7f127dac4664700.png"},{"id":99217255,"identity":"6e06aef6-bcc5-4444-98ad-68d72d761d3e","added_by":"auto","created_at":"2025-12-30 09:10:20","extension":"xml","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":127811,"visible":true,"origin":"","legend":"","description":"","filename":"2d4696a419ec4751b4bbf4c1deb1080b1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/e28b6014da4a833511588c3d.xml"},{"id":99217427,"identity":"c587bb7e-43be-4daf-9c00-058796223c8d","added_by":"auto","created_at":"2025-12-30 09:10:35","extension":"html","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":145246,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1/78bb9c8a63a9e4d113e0045b.html"},{"id":100726842,"identity":"203e7b2c-4edc-43b7-9674-8af5ba16b9c5","added_by":"auto","created_at":"2026-01-20 20:31:23","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":612425,"visible":true,"origin":"","legend":"","description":"","filename":"HiFuseDEV2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7949557/v1_covered_cd8ac8ca-9441-4007-8366-f2b05bbf5bcb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Hierarchical Fusion with Decision Enhancement for Human Activity Recognition Using Deep Learning Framework","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Deep Learning, Graph Neural Networks, Human Activity Recognition, Multi-Sensor Data Fusion, Self-Supervised Learning, Transformer Networks","lastPublishedDoi":"10.21203/rs.3.rs-7949557/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7949557/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eHuman Activity Recognition (HAR) is emerging as a critical enabler of context-aware applications in healthcare, fitness, and smart environments. In this research we present an approach that involves the Hierarchical Fusion with Decision Enhancement for Human Activity Recognition (Hi-FuseDE-HAR) framework. It contains four sequential hierarchical levels that transform raw sensor signals into a reliable HAR decision. At the input level, heterogeneous data streams are obtained from multiple wearable and ambient sensors. Applying Level 0 build the discriminative latent embedding for each sensor modality separately. This involves a number of CNN-based or Transformer-based encoders to transform each sensor dimension from raw to latent embedding. Level 1 fuses across sensors in groupings and determines the value of using each modality and determines its relative contribution to the overall feature importance. Level 2 applies a Graph Cross-Modal Transformer that learns relationship between sensors groups producing a globally consistent fused representation. Level 3 provides decision enhancement through uncertainty calibration and utility aware optimization to ensure the final estimates based. Experimental results indicate that the proposed framework achieves 97.6% accuracy and 96.7% F1-score on the PAMAP2 dataset, 95.5% accuracy and 93.2% F1-score on the OPPORTUNITY dataset, and 96.5% accuracy and 95.2% F1-score on the MHEALTH dataset respectively. Notably, Hi-FuseDE-HAR retains strong performance confirming its capability to generalize across varied sensor contexts and complex activity patterns.\u003c/p\u003e","manuscriptTitle":"Hierarchical Fusion with Decision Enhancement for Human Activity Recognition Using Deep Learning Framework","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-30 09:09:42","doi":"10.21203/rs.3.rs-7949557/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c426d396-0f01-412f-bc90-2eefe89be980","owner":[],"postedDate":"December 30th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":60286740,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":60286741,"name":"Physical sciences/Engineering"},{"id":60286742,"name":"Health sciences/Health care"},{"id":60286743,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2026-01-20T17:38:28+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-30 09:09:42","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7949557","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7949557","identity":"rs-7949557","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}