Wearable Signal Fusion through Deep Learning: Discriminating Affective States Levels in the Wild among Healthy and Depressed

Wearable Signal Fusion through Deep Learning: Discriminating Affective States Levels in the Wild among Healthy and Depressed | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Wearable Signal Fusion through Deep Learning: Discriminating Affective States Levels in the Wild among Healthy and Depressed Cristina Gallego Vázquez, Corinne Eicher, Reto Huber, Golo Kronenberg, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3961385/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The increasing prevalence of depression underscores the critical need for improved monitoring and personalized treatment options. While traditional assessment methods exhibit limitations, smartphone-based ecological momentary assessment has demonstrated validity and reliability in capturing real-time experiences. However, challenges such as time consumption and perceived invasiveness remain. To address these limitations, our research explored the potential of deep learning and wearable sensors to classify self-reported affective symptoms such as valence, arousal, and sleepiness based on objective and continuous passive data. Our study spanned 35 days, incorporating a diverse cohort of 26 participants (14 female, age 29 $\pm$9.0), including sixteen depressed patients and ten healthy controls. We used deep learning techniques to classify high-quality physiological data collected from a wearable patch, combining electrocardiogram signals, raw accelerometer data, and respiration rates into 3-levels of affective states. We identified optimal time windows for prediction — 24 hours for valence and 12 hours for arousal and sleepiness - and showed that combining longitudinal heart rate and heart rate variability metrics with physical activity enhanced the predictive performance for affective states classification compared to individual modalities. Our models achieved notable classification metrics for the 3-level affective states, with a balanced accuracy of 0.65 for valence, 0.56 for arousal and 0.53 for sleepiness, demonstrating competitive performance with previous work. This research contributes to advancing mental health monitoring practices, providing valuable insights into the relationships between affective and physiological states. Health sciences/Biomarkers/Predictive markers Health sciences/Diseases/Psychiatric disorders/Depression Full Text Additional Declarations No competing interests reported. Supplementary Files supplementaryMaterialgallegovazquezCristina.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3961385","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":273777644,"identity":"cada5bc9-812a-4358-8b4b-27b773fb0c4e","order_by":0,"name":"Cristina Gallego Vázquez","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABaElEQVRIie2RPUvDQBiA7whcl5SsB4X0L7whoIiIf+WODl1aKbhkiOEgcF0CXSsKDv6BOonbQSBZgnNBwQQhXRwCrqWYfljTVneHPHAfvPc+994HQjU1/xIsEIL1TKWVuAmbCSsb2VfUZlWxStyGzXaHyjL1e6wqfPKHYtyE/mcxeG0j4z5VzLkyIY6zF93xuo+N4SxFzvyiLbS8cmT6zCVVkFuC5qBYEtuQsM6pnpD+U5BYAiVwiQU5hh8FdCzLu4RYlKLiMuLXgkWtptT7k2kPCyyB+0gndEfxi1I5FzQuFF+UyiiTreaCduFtlgm8AC4PlOX+IRdGUFYRLh/RDmk1BTCYIktgATzYVWiCJU0g7EiqDxSLlG3QXDu5jZg1SXrWmEU2H2vkqKIYQeO9cObh2ciIH9LC9UxidLPph+u1y6dLi8I1+d3QzyvKFkJXnxKua2/Dq6D2S/6qnlr23p5SU1NTU4PQF1zGiLQKaBjYAAAAAElFTkSuQmCC","orcid":"","institution":"ETH Zurich","correspondingAuthor":true,"prefix":"","firstName":"Cristina","middleName":"Gallego","lastName":"Vázquez","suffix":""},{"id":273777645,"identity":"95be4f68-11aa-4ed9-ae8b-657ad1d5101f","order_by":1,"name":"Corinne Eicher","email":"","orcid":"","institution":"University of Zurich","correspondingAuthor":false,"prefix":"","firstName":"Corinne","middleName":"","lastName":"Eicher","suffix":""},{"id":273777646,"identity":"b8d09d95-c66d-42b5-ba90-200d63218d10","order_by":2,"name":"Reto Huber","email":"","orcid":"","institution":"University Children's Hospital Zurich","correspondingAuthor":false,"prefix":"","firstName":"Reto","middleName":"","lastName":"Huber","suffix":""},{"id":273777647,"identity":"829be337-bf45-451a-99af-e77758fee44e","order_by":3,"name":"Golo Kronenberg","email":"","orcid":"","institution":"University of Zurich","correspondingAuthor":false,"prefix":"","firstName":"Golo","middleName":"","lastName":"Kronenberg","suffix":""},{"id":273777648,"identity":"66ef9de6-7def-422d-8914-6124ddfc90c4","order_by":4,"name":"Hans-Peter Landolt","email":"","orcid":"","institution":"University of Zurich","correspondingAuthor":false,"prefix":"","firstName":"Hans-Peter","middleName":"","lastName":"Landolt","suffix":""},{"id":273777649,"identity":"42d2c476-9f3c-4ad7-9a2f-369fc39c082a","order_by":5,"name":"Erich Seifritz","email":"","orcid":"","institution":"University of Zurich","correspondingAuthor":false,"prefix":"","firstName":"Erich","middleName":"","lastName":"Seifritz","suffix":""},{"id":273777650,"identity":"b697c510-eab2-4ff3-b7ef-e17ec3fa9594","order_by":6,"name":"Giulia da Poian","email":"","orcid":"","institution":"ETH Zurich","correspondingAuthor":false,"prefix":"","firstName":"Giulia","middleName":"da","lastName":"Poian","suffix":""}],"badges":[],"createdAt":"2024-02-16 13:47:38","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3961385/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3961385/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":95619651,"identity":"25760fab-3be9-4453-a009-2ad28543d65a","added_by":"auto","created_at":"2025-11-11 09:25:02","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6207945,"visible":true,"origin":"","legend":"","description":"","filename":"npjMentalHealthResearchGallegoVazquezCristina.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3961385/v1_covered_b2bb370a-8ddb-4988-b226-40a9c3bd9b7e.pdf"},{"id":51412309,"identity":"0966272a-2ee3-4b19-9846-061547bdaaf7","added_by":"auto","created_at":"2024-02-21 05:34:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":313063,"visible":true,"origin":"","legend":"","description":"","filename":"supplementaryMaterialgallegovazquezCristina.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3961385/v1/f57e4f3bcf35e5f63678e604.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Wearable Signal Fusion through Deep Learning: Discriminating Affective States Levels in the Wild among Healthy and Depressed","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3961385/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3961385/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The increasing prevalence of depression underscores the critical need for improved monitoring and personalized treatment options. While traditional assessment methods exhibit limitations, smartphone-based ecological momentary assessment has demonstrated validity and reliability in capturing real-time experiences. However, challenges such as time consumption and perceived invasiveness remain. To address these limitations, our research explored the potential of deep learning and wearable sensors to classify self-reported affective symptoms such as valence, arousal, and sleepiness based on objective and continuous passive data. Our study spanned 35 days, incorporating a diverse cohort of 26 participants (14 female, age 29 $\\pm$9.0), including sixteen depressed patients and ten healthy controls. We used deep learning techniques to classify high-quality physiological data collected from a wearable patch, combining electrocardiogram signals, raw accelerometer data, and respiration rates into 3-levels of affective states. We identified optimal time windows for prediction — 24 hours for valence and 12 hours for arousal and sleepiness - and showed that combining longitudinal heart rate and heart rate variability metrics with physical activity enhanced the predictive performance for affective states classification compared to individual modalities. Our models achieved notable classification metrics for the 3-level affective states, with a balanced accuracy of 0.65 for valence, 0.56 for arousal and 0.53 for sleepiness, demonstrating competitive performance with previous work. This research contributes to advancing mental health monitoring practices, providing valuable insights into the relationships between affective and physiological states.","manuscriptTitle":"Wearable Signal Fusion through Deep Learning: Discriminating Affective States Levels in the Wild among Healthy and Depressed","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-21 05:34:50","doi":"10.21203/rs.3.rs-3961385/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6f185423-575a-4a98-b29b-1189ab46b419","owner":[],"postedDate":"February 21st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":28851917,"name":"Health sciences/Biomarkers/Predictive markers"},{"id":28851918,"name":"Health sciences/Diseases/Psychiatric disorders/Depression"}],"tags":[],"updatedAt":"2025-11-11T09:24:23+00:00","versionOfRecord":[],"versionCreatedAt":"2024-02-21 05:34:50","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3961385","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3961385","identity":"rs-3961385","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}