Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep

Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep Pavlos Ioannis Topalidis, Laurin Julius Kogler, Carina Mitterer, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8419739/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The rapid expansion of consumer sleep trackers (CST) in clinical and basic research has intensified the need for standardized performance evaluation frameworks capable of examining their limitations and capabilities against polysomnography (PSG). In this study, we systematically compared eight CST: sleep² withPolar Verity Sense (VS) and Polar H10, Oura Ring 3, Apple Watch Series 9, Fitbit Charge 6, Garmin Vivoactive 5 & Venu 3, WHOOP 4, and Circul+, againsthome-based ambulatory PSG. Eighteen participants completed five consecutive nights (Monday to Friday) of home PSG while wearing two paired devices of each CST. Controlled sleep disruptions, as well as restricted and extended sleep, were included in the protocol to challenge the devices’ algorithms undervaried conditions. Performance evaluation followed a standardised framework incorporating both epoch-by-epoch and discrepancy analyses to assess multi-class sleep-staging performance and agreement across key sleep parameters. Epoch-by-epoch accuracy and Cohen’s κ were as follows:sleep²-Polar VS 83.7% (κ =0.76), sleep²-Polar H10 84.04% (κ = 0.76), Oura Ring 3 72.5% (κ = 0.59), AppleWatch Series 9 72.3% (κ = 0.56), WHOOP 4 68.68% (κ = 0.54), Fitbit Charge 6 66.16% (κ = 0.47), Garmin Vivoactive 5 & Venu 3 63.4% (κ = 0.41), and Circul+ 55.59% (κ = 0.33). Discrepancy analysis revealed systematic and proportional biases across CST: most wrist-worn devices overestimated sleep time and underestimated wake after sleep onset, with error magnitude increasing drastically in atypical nights characterised by fragmented, restricted, or extended sleep. Arm- and chest-band sleep² derived sleep metrics, based on cardiac-signals, showed minor deviation from PSG, whereas Oura exhibited moderate but variable levels of bias. Overall, these findings reveal substantial variability across CST and highlight the importance of standardised benchmarking to support the scientificand clinical use of wearable-derived sleep macrostructure. JEL Classification: D8 , H51 MSC Classification: 35A01 , 65L10 Physical sciences/Engineering Health sciences/Health care Health sciences/Medical research wearables automatic sleep staging standardized framework consumer sleep trackers (CST) performance evaluation sleep2 with Polar Verity Sense (VS) and Polar H10 Oura Ring 3 Apple Watch Series 9 Fitbit Charge 6 Garmin Vivoactive 5 WHOOP 4 and Circul+ Full Text Additional Declarations Competing interest reported. M.S. is the co-founder of and CSO at the NUKKUAA GmbH. P.I.T is an employee of NUKKUAA GmbH. L.J.K., C.M., A.H., and S.B. declare no conflicts of interest. R.t.H. operates a YouTube channel that includes affiliate links to Oura and WHOOP, as well as a general Amazon affiliate link that applies to most other wearable brands; purchases made via these links may generate commission income. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8419739","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":587382043,"identity":"50abe374-bf66-4ed5-a0d8-fb77f06d4fda","order_by":0,"name":"Pavlos Ioannis Topalidis","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABI0lEQVRIie2RMUvDQBTHXyiky8mtL2ToV7hSUItivsodAV06OBVBqCmBc/EDpNgPoYvzhYAuxa4RO0QCzhk7ZDAXawU5qaPD/Ya7x8GP93/vACyW/4mjtgV3ESh0op2OVrAttOJFzpfS+YMCLgBTO5SD7oKpqoYJ9eNCFePDYPA6ld4aTnrQXSqTMrwZsXQmAb35I1P8GcXDKpU+gbAfkdDYhalTnu1FgCznTAmJfD8X0m9CcYBflOU7z0itlbNKK8EgETrYFQdampU8VBlxtTJquzh32JwEMg5o7jJMStXMgt5svjhvZ0lWaXxE2FNfYsmMG6NiWlT1MaX+9f3bejwJ6G2cvqwvLnuUisIY7PPCzc9s6vbdNcbaKvBTsVgsFss3HyMqYi50AfPuAAAAAElFTkSuQmCC","orcid":"","institution":"University of Salzburg","correspondingAuthor":true,"prefix":"","firstName":"Pavlos","middleName":"Ioannis","lastName":"Topalidis","suffix":""},{"id":587382044,"identity":"7f6c2d4c-5d15-4a8d-94ce-c69c65cf7c75","order_by":1,"name":"Laurin Julius Kogler","email":"","orcid":"","institution":"University of Salzburg","correspondingAuthor":false,"prefix":"","firstName":"Laurin","middleName":"Julius","lastName":"Kogler","suffix":""},{"id":587382045,"identity":"5902cf26-c1f1-4937-96ce-0443663e15e5","order_by":2,"name":"Carina Mitterer","email":"","orcid":"","institution":"University of Salzburg","correspondingAuthor":false,"prefix":"","firstName":"Carina","middleName":"","lastName":"Mitterer","suffix":""},{"id":587382046,"identity":"85f4e152-4d90-40a8-ac7e-276a8a05cf22","order_by":3,"name":"Alexandra Hinterberger","email":"","orcid":"","institution":"University of Salzburg","correspondingAuthor":false,"prefix":"","firstName":"Alexandra","middleName":"","lastName":"Hinterberger","suffix":""},{"id":587382047,"identity":"039b9984-8e49-47e9-af0f-c5a69937e463","order_by":4,"name":"Sebastian Baron","email":"","orcid":"","institution":"University of Salzburg","correspondingAuthor":false,"prefix":"","firstName":"Sebastian","middleName":"","lastName":"Baron","suffix":""},{"id":587382052,"identity":"9dff740e-bcdd-441d-9d3b-e5a15c24d95f","order_by":5,"name":"Rob ter Horst","email":"","orcid":"","institution":"Medical University of Vienna","correspondingAuthor":false,"prefix":"","firstName":"Rob","middleName":"ter","lastName":"Horst","suffix":""},{"id":587382054,"identity":"143c1e49-18a8-41bc-bf52-b039b35ed2dd","order_by":6,"name":"Manuel Schabus","email":"","orcid":"","institution":"University of Salzburg","correspondingAuthor":false,"prefix":"","firstName":"Manuel","middleName":"","lastName":"Schabus","suffix":""}],"badges":[],"createdAt":"2025-12-21 23:23:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8419739/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8419739/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":104808855,"identity":"ffce5b57-404d-4b83-b2c0-fa620988000f","added_by":"auto","created_at":"2026-03-17 12:40:18","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":83392951,"visible":true,"origin":"","legend":"","description":"","filename":"MultisensorStudy1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8419739/v1_covered_b360cadb-4302-44c8-9cf2-62da2c3e68f6.pdf"}],"financialInterests":"Competing interest reported. M.S. is the co-founder of and CSO at the NUKKUAA GmbH. P.I.T is an employee\nof NUKKUAA GmbH. L.J.K., C.M., A.H., and S.B. declare no conflicts of interest.\nR.t.H. operates a YouTube channel that includes affiliate links to Oura and WHOOP,\nas well as a general Amazon affiliate link that applies to most other wearable brands;\npurchases made via these links may generate commission income.","formattedTitle":"Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"wearables, automatic sleep staging, standardized framework, consumer sleep trackers (CST), performance evaluation, sleep2 with Polar Verity Sense (VS) and Polar H10, Oura Ring 3, Apple Watch Series 9, Fitbit Charge 6, Garmin Vivoactive 5, WHOOP 4, and Circul+","lastPublishedDoi":"10.21203/rs.3.rs-8419739/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8419739/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe rapid expansion of consumer sleep trackers (CST) in clinical and basic research has intensified the need for standardized performance evaluation frameworks capable of examining their limitations and capabilities against polysomnography (PSG). In this study, we systematically compared eight CST: sleep² withPolar Verity Sense (VS) and Polar H10, Oura Ring 3, Apple Watch Series 9, Fitbit Charge 6, Garmin Vivoactive 5 \u0026amp; Venu 3, WHOOP 4, and Circul+, againsthome-based ambulatory PSG. Eighteen participants completed five consecutive nights (Monday to Friday) of home PSG while wearing two paired devices of each CST. Controlled sleep disruptions, as well as restricted and extended sleep, were included in the protocol to challenge the devices’ algorithms undervaried conditions. Performance evaluation followed a standardised framework incorporating both epoch-by-epoch and discrepancy analyses to assess multi-class sleep-staging performance and agreement across key sleep parameters. Epoch-by-epoch accuracy and Cohen’s κ were as follows:sleep²-Polar VS 83.7% (κ =0.76), sleep²-Polar H10 84.04% (κ = 0.76), Oura Ring 3 72.5% (κ = 0.59), AppleWatch Series 9 72.3% (κ = 0.56), WHOOP 4 68.68% (κ = 0.54), Fitbit Charge 6 66.16% (κ = 0.47), Garmin Vivoactive 5 \u0026amp; Venu 3 63.4% (κ = 0.41), and Circul+ 55.59% (κ = 0.33). Discrepancy analysis revealed systematic and proportional biases across CST: most wrist-worn devices overestimated sleep time and underestimated wake after sleep onset, with error magnitude increasing drastically in atypical nights characterised by fragmented, restricted, or extended sleep. Arm- and chest-band sleep² derived sleep metrics, based on cardiac-signals, showed minor deviation from PSG, whereas Oura exhibited moderate but variable levels of bias. Overall, these findings reveal substantial variability across CST and highlight the importance of standardised benchmarking to support the scientificand clinical use of wearable-derived sleep macrostructure.\u003c/p\u003e\n\u003cp\u003eJEL Classification: D8 , H51\u003c/p\u003e\n\u003cp\u003eMSC Classification: 35A01 , 65L10\u003c/p\u003e","manuscriptTitle":"Beyond the Hype? A Standardised Real-World Evaluation of Consumer Sleep Trackers (CST) in Extracting Sleep","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-09 10:59:24","doi":"10.21203/rs.3.rs-8419739/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c3d10524-3570-4ec1-be88-40beb131058d","owner":[],"postedDate":"February 9th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62501711,"name":"Physical sciences/Engineering"},{"id":62501712,"name":"Health sciences/Health care"},{"id":62501713,"name":"Health sciences/Medical research"}],"tags":[],"updatedAt":"2026-04-09T15:24:26+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-09 10:59:24","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8419739","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8419739","identity":"rs-8419739","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}