Inter-rater disagreement in manual scoring of intensive care unit sleep data

Inter-rater disagreement in manual scoring of intensive care unit sleep data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Inter-rater disagreement in manual scoring of intensive care unit sleep data Laurens Reinke, Esther M. Heide, Pedro Fonseca, Anthony R. Absalom, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3975957/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Apr, 2025 Read the published version in BMC Research Notes → Version 1 posted 10 You are reading this latest preprint version Abstract Objective : Severe sleep disruption is common among intensive care unit (ICU) patients. However, the applicability of standard sleep scoring guidelines by the American Academy of Sleep Medicine (AASM) has been questioned, with most polysomnography (PSG) studies in critically ill patients reporting difficulties in setting up and processing and scoring the recordings. The present study explores human inter-rater agreement in sleep stage scoring following the AASM guidelines, within a heterogenous ICU patient cohort. Results : Two human experts independently scored a total of 51,454 epochs in 20 PSG recordings acquired at the ICU. Epoch-per-epoch comparison of scored stages revealed a Cohen’s κ coefficient of agreement of 0.36 for standard 5-stage scoring. Highest agreement occurred in Wake (κ = 0.46), while REM showed the lowest (κ = 0.12). Significant correlations were found between inter-rater agreement, and Simplified Acute Physiology Score (SAPS II, r=-0.506, p=0.038), and 12-month mortality (r=-0.524, p=0.031). Comparison with similar studies underscore challenges in applying AASM criteria to ICU patients. Despite accounting for artifacts, disparities persisted, emphasizing the need for a nuanced exploration of factors influencing scoring inconsistencies in critically ill patients. Figures Figure 1 Introduction Sleep is a dynamic, complex physiological process essential for homeostasis, recovery, and survival ( 1 , 2 ). Disrupted or delayed sleep is associated with impaired immune function ( 3 ), increased susceptibility to infections and impaired wound healing ( 4 , 5 ), impaired metabolic and endocrine function ( 6 ), increased pain perception ( 7 , 8 ) and impairment of neurophysiologic organization and memory consolidation ( 9 ). Sleep deprivation affects up to 60% of all critically ill patients admitted to an intensive care unit (ICU) ( 10 , 11 ). Sleep among these patients is often fragmented by frequent arousals and awakenings which hamper transitions to deeper stages of sleep, reduced duration of sleep, and disturbed distribution of sleep with up to half of the total sleep time occurring during the day ( 4 , 5 , 11 , 12 ). Poor sleep during critical illness is considered to be a major stressor for patients during and after ICU admission. It is associated with the development of ICU delirium and long-term cognitive decline, and has detrimental effects on recovery, morbidity, and mortality ( 13 , 14 , 15 ). The ICU is a unique environment where a multitude of intrinsic and environmental factors may hamper sleep (16, 17. 18, 19, 20, 21, 22). Although previous studies have provided new insights into the etiology and possible prevention of disturbed sleep in the ICU, their scope, statistical significance and reliability have thus far been constrained by the logistical challenges of measuring and assessing sleep objectively ( 2 , 4 , 20 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ). Electroencephalography (EEG) has historically been the primary tool for objective sleep monitoring ( 28 , 29 ). Polysomnography (PSG), combining EEG electromyography (EMG), and electrooculography (EOG) is the technique used to investigate sleep. The visual and manual annotation or scoring of these recordings commonly follows criteria originally set by Kales and Rechtschaffen ( 31 ), with additional changes later culminating in the American Academy of Sleep Medicine (AASM) Manual for the Scoring of Sleep ( 32 ). Hundreds or even thousands of 30 second epochs each comprising multiple channels of PSG data are typically processed by a single human expert. Although this method is considered to be the gold standard for routine clinical sleep analysis, most PSG studies in critically ill patients report difficulties in setting up, maintaining, and manually processing and scoring ICU sleep recordings ( 4 , 12 , 33 , 34 , 35 , 36 ). The practical expertise required to apply and maintain the array of electrodes required for human scoring further limits scalability and increases costs. Furthermore, the reliability and repeatability of manual analysis of ICU sleep recordings is lower than for other clinical recordings ( 37 ). While Elliott et al. reported observed ‘reasonable’ to ‘good’ agreement between two combinations of 3 human scorers in discerning wake from sleep activity, the agreement on detailed sleep staging was much lower depending on individual sleep stages and the combination of human scorers ( 23 ). The objective of this study is to investigate human inter-rater agreement in sleep staging following the AASM rules for sleep scoring, in a heterogeneous population of ICU patients. Methods Study population and patient recruitment We obtained 70 PSG recordings during an observational study (Trial NL5197, NTR5345) primarily investigating the influence of disrupted biorhythms on the quantity and quality of sleep among patients of the department of Critical Care of the University Medical Center Groningen (UMCG). After approval by the local ethics committee (UMCG METc, registration number 2015/00295), data collection started in September 2015 and finished in September 2018. All adult patients without a history of sleep pathology, an expected ICU stay of at least 48 hours, and a Richmond Agitation and Sedation Scale (RASS) above − 3 were eligible for inclusion in the study. Informed consent was gained from patients with capacity to do so. For patient lacking capacity, informed consent was first obtained from their legal representatives, followed by consent after they recovered consciousness. Neurosurgical patients, and patients taking melatonin supplements were excluded from participation. Data acquisition PSG was recorded for a period of 24–72 hours depending on patient’s tolerance, RASS scores, and ICU length of stay. The recording consisted of six EEG channels (F3, A1, A2, C3, C4, O1), two EOG channels and EMG of the left and right masseter or submental muscles. Ag/AgCl EEG-electrodes were placed according to the international 10–20 system after skin preparation according to standardized techniques. A BrainAmp DC32 amplifier with a BrainVision recorder (Brain Vision Solutions, Montreal, Canada) or an Alice 6 LDx system (Philips Respironics, Murrysville, USA) was used. EEG was recorded with a sample frequency of 256Hz. Anonymized data were stored for sleep scoring by two experienced human experts. Sleep analysis All recorded data were blindly assessed for data quality by a human expert and sets of sufficient quality were then analysed by a human expert scorer (M 1 ). We randomly selected 20 patients for further analysis by an additional human expert scorer (M 2 ). Human expert scorers were free to select either the C4-A1 or C3-A2 EEG channel for scoring depending on signal quality. The scoring of discrete wake and sleep stages (rapid eye-movement sleep, REM; non-REM sleep stages, N1, N2, N3) according to the latest AASM scoring guidelines by the scorers was done by visual interpretation of individual 30 second epochs in the Brain RT software (OSG, Rumst, Belgium). Statistical analysis Sleep-related parameters were calculated using Matlab (Matlab 2014b, Natick, MA, USA). Statistics were calculated using SPSS 24 (2016, IBM, Armonk, NY, USA). Cohen’s Kappa statistic was used to evaluate epoch-per-epoch agreement between human expert scorers for all sleep stages individually and for full 5-stage sleep scoring. Cohen’s Kappa is a dimensionless index that corrects for chance agreement due to imbalanced datasets, such as the imbalanced distribution of sleep and wake stages. Scoring agreement statistics for wake and individual sleep classes were calculated using a binary one-versus-rest strategy. For normative interpretation of inter-rater agreement we used the guidelines by Landis and Koch ( 38 ). Spearman’s correlation coefficient was used to quantify the correlation between inter-rater agreements, predicted mortality, and mortality. For estimation of statistical significance, an alpha of 0.05 was used. Unless indicated otherwise, results are presented as mean values (standard deviation). Results Seventy patients were included in the main study. PSG data from 4 (5.71%) were lost due to undetected technical failure of EEG equipment during measurement. A further 5 (7.14%) recordings were deemed entirely unscorable by the human expert scorers. Of the remaining 61 patient recordings a median of 0.22% of epochs (0.01–0.56% interquartile range, IQR) were entirely excluded due to artifacts, leaving 339,901 30-second epochs (2832.51 hours) for further analysis by scorer M 1 . In total 20 recordings were randomly selected for classification by a second scorer (M 2 ), 3 (15%) were rejected entirely due to low signal quality. Of the remaining 17 patient recordings 0.26% of epochs (0.09–0.86% IQR) were entirely excluded due to artifacts, leaving 51,454 epochs (428.78 hours) for analysis of inter-rater agreement between two human scorers. Patient characteristics for all valid recordings and for the subgroup scored by M 2 are summarized in Table 1 . Recordings randomly selected for additional classification by M 2 were from patients with a significantly shorter median ICU stay 7.01 (4.01–18.98 IQR) days, p = 0.014) days versus (14.01 (6.02–29.51 IQR), but there were no other significant differences from the rest of the sample. Table 1 Demographics of the group with valid recording, and the subgroup randomly selected for additional analysis by M 2 Valid recordings (n = 61) Analysed by M 2 (n = 17) p-value Characteristic N (%) N (%) Sex 23 (37.7) 8 (47.06) ICU admission diagnosis surgical 19 (31.15) 5 (29.41) medical 42 (68.85) 12 (70.59) 12 month mortality 19 (31.15) 4 (23.53) Median (IQR) Median (IQR) BMI 26 ( 23 – 29 ) 28 ( 24 – 32 ) Age, years 60 (52–67) 63 (52–67) Duration of hospital stay, days 31 (19-55.49) 21.99 (15.99-43) Duration of ICU stay, days 10.99 (5.51–25.50) 7.01 (4.01–18.98) Duration of recording, hours 47.74 (17.22) 46.66 (43-65.89) APACHE IV 69 (47–82) 74 (61.25–81.25) SAPS II 40 (31–49) 42 (36–44) Mechanical ventilation, days 7 ( 1 – 16 ) 5 ( 1 – 14 ) Mean (SD) Medication dose per day Benzodiazepines, mg Lorazepam equivalent 1.55 (4.82) 2.95 (8.77) Opioids, mg Morphine equivalent 1.80 (3.63) 0.95 (1.92) Propofol 2%, ml 1.81 (7.27) 2.96 (10.59) Table 2 indicates the prevalence of each sleep stage (according to M 1 scorings), and agreement between the two human scorers for the 5-class scoring task, as well as for each class versus the rest. Mean κ agreement for 5-classes was 0.36, with the best agreement obtained for Wake, with a κ of 0.46, and worst for REM, with a κ of 0.12. REM was also the least prevalent class, with an average number of 0.00 hours per 24-hour period. Table 2 Agreement between human scorers (M 1 vs. M 2 ) Class Wake REM N1 N2 N3 5 classes Prevalence based on M 1 , hours per day (SD) 13.98 (8.45) 0.00 (0.01) 1.22 (1.34) 6.15 (5.16) 2.66 (4.09) 24 Kappa, - (SD) 0.46 (0.27) 0.12 (0.24) 0.13 (0.13) 0.32 (0.21) 0.26 (0.24) 0.36 (0.21) Accuracy, % (SD) 80.62 (14.57) 97.89 (5.26) 90.33 (8.24) 80.65 (13.24) 91.54 (9.22) 70.51 (17.5) Sensitivity, % (SD) 81.3 (21.99) 11.72 (24.78) 15.92 (14.76) 53.02 (24.92) 36.04 (30.97) - Specificity, % (SD) 64.86 (26.93) 99.87 (0.33) 96.35 (3.92) 84.89 (13.73) 95.01 (8.96) - PPV, % (SD) 79.55 (19.38) 55.56 (39.59) 24.1 (17.91) 44.74 (22.67) 48.63 (36.49) - Performance statistics for individual classes were calculated using a binary one-vs-rest strategy. Per-subject κ agreement between M 1 and M 2 correlated significantly with the Simplified Acute Physiology Score (SAPS II) predictor of mortality (r=-0.506, p = 0.038), and with recorded 12-month mortality (r=-0.524, p = 0.031). Figure 1 illustrates the confusion matrix for the pooled classification of all epochs in the recordings scored by both human experts scorers. Even for the class with the best κagreement, i.e., Wake, inconsistent scoring was found between the two scorers: M 2 scored a large proportion of M 1 -Wake epochs as N2 and to a certain degree, even N3, whereas M 1 scored a larger proportion of M 2 -Wake as N1. Discussion Human inter-rater agreement in our sample was comparable to that between human scorers in other studies of ICU sleep. Elliott et al. ( 23 ) reported a Cohen’s kappa of κ = 0.58–0.68, which they deemed to be ‘reasonable’ to ‘good’ agreement ( 38 ), for sleep-wake scoring by two combinations of 3 manual/human scorers. Agreement for the results of detailed sleep staging was much lower, with only slight agreement for stage N1 (κ = 0.08–0.12), moderate agreement for N2 and REM (κ = 0.55–0.58 and κ = 0.41–0.44, respectively), and slight to good agreement for slow wave sleep (κ = 0.20–0.76), depending on the combinations of manual scorers. Similarly, disagreement in our sample was highest for REM and N1, likely due to a general deficit of this stage of sleep in ICU populations. Additional disagreement was found between individual sleep stages and the wake stage, which could be the result of the relatively high amount of EEG and EMG artifacts in this intensive care population being interpreted as proof of wakefulness. The remainder of substantial disagreement exists between the already notoriously difficult to separate N2 and N3 stages. Ambrogio et al. compared the agreement between two manual scorers for PSG recordings of 14 mechanically ventilated ICU patients and 17 ambulatory control patients ( 37 ). Inter-rater reliability was good (κ = 0.74) for recordings of ambulatory patients, but there was only slight agreement on the scoring of recordings of ICU patients (κ = 0.19). In conclusion, our data further confirms that the applicability of the AASM criteria for most ICU recorded PSG-data is debatable, particularly so among the most unwell patients. Although the source of confusion can be partially attributed to the high amount of EEG and EMG artifacts, this does not always explain the disparity in scoring between otherwise relatively unambiguous stages, such as Wake and N3, or REM and N2. For these patients, rather than deeming the scoring rules as inadequate, a better understanding of the factors driving this disparity could help shed light on the sleep of this critically ill population. Limitations PSG is notoriously labour-intensive during set-up, maintenance, and analysis, which limited the sample size of this study a priori. Despite our best efforts, the amount of usable data was further limited by artifacts from frequent and intensive care, electromagnetic pollution, motor restlessness, excessive sweating and other technical challenges. Study inclusion and exclusion criteria were chosen to minimize the likelihood of unproductive measurements but may have decreased the already limited generalizability of results from inherently heterogeneous ICU patients. Study inclusion did not always start immediately after ICU admission and varied in duration due to the unpredictable progression of critical illness. This caused an imbalance in the contribution of individual recordings to aggregated means, which is why all statistics were calculated from per-subject means. ICU patients could not be relied upon for subjective sleep evaluation, and the neurocognitive state of subjects was not assessed. The limited practical scalability of polysomnography and human expert sleep scoring has not only restricted the sample size of our comparison but has also limited our ability to do proper consensus scoring or full-sample multi-rater human expert scoring for this investigation. Future efforts to provide more comprehensive investigation of interrater agreements are still encouraged and could benefit from aggregating recordings from previous studies and the adherence to standardized scoring. Abbreviations AASM American Academy of Sleep Medicine EEG electroencephalography EMG electromyography EOG electrooculography ICU intensive care unit IQR interquartile range PSG polysomnography RASS Richmond Agitation and Sedation Scale REM rapid eye-movement (sleep) N1, N2, N3 non-REM sleep stages 1, 2, 3 SAPS II simplified acute physiology score UMCG University Medical Center Groningen Declarations Ethics approval and consent to participate Trial was registered as “Sleep and biorhythm in the ICU”, in trialregister.nl, with number NL5197 (NTR5345) (https://www.trialregister.nl/trial/5197). The study was approved by the local ethics committee (UMCG METc, registration number 2015/00295). Informed consent was obtained from patients with capacity to do so. Otherwise, informed consent was first obtained from their legal representatives, followed by patients after they recovered consciousness. Consent for publication Not applicable Availability of data and materials Data are available by contacting the corresponding author. Competing interests At the time of writing, EMH and PF are employed by Philips Research Eindhoven. The remaining authors did not have any conflicts of interest to declare. Funding No funding was obtained for the purpose of this study. The BrainAmp DC32 amplifier, BrainVision recorder, and disposables were property of the investigating hospital. The Alice 6 LDx system was supplied by Philips Research Eindhoven. LR received partial funding (paid to institution) from Philips Research Eindhoven for a PhD position at the University Medical Center Groningen. Authors' contributions L.R. drafted the first manuscript, all other authors provided feedback on drafts of the paper. All authors were equally responsible for the conception of the study. L.R. was responsible for implementation of the study and enrolled participants. A.R.A., J.E.T. and L.R. collated the data and analysed results. All authors contributed to, read, and approved the final manuscript. Acknowledgements(optional) We thank the entire UMCG ICV research team and student team for their support in setting up and executing this study. We thank dr. J.H. van der Hoeven for analysing PSG data, and Philips Sleep & Respiratory Care for automated sleep analysis. The initiation and success of this study is owed largely to the contribution of Technical Physicians in training. References Weinhouse GL, Schwab RJ. Sleep in the critically ill patient. Sleep [Internet]. 2006;29(5):707–16. Kamdar BB, Needham DM, Collop NA. Sleep Deprivation in Critical Illness: Its Role in Physical and Psychological Recovery. J Intensive Care Med [Internet]. 2012;27(2):97–111. Irwin M, McClintick J, Costlow C, Fortner M, White J, Gillin JC. Partial night sleep deprivation reduces natural killer and celhdar immune responses in humans. FASEB J [Internet]. 1996;10(5):643–53. Cooper AB, Thornley KS, Young GB, Slutsky AS, Stewart TE, Hanly PJ. Sleep in critically ill patients requiring mechanical ventilation. Chest [Internet]. 2000 Mar;1(3):809–18. Friese RS, Diaz-Arrastia R, McBride D, Frankel H, Gentilello LM. Quantity and quality of sleep in the surgical intensive care unit: are our patients sleeping? J Trauma [Internet]. 2007;63(6):1210–4. Spiegel K, Leproult R, Van Cauter E. Impact of sleep debt on metabolic and endocrine function. Lancet [Internet] 1999 Oct 23 ;354(9188):1435–9. Lautenbacher S, Kundermann B, Krieg JC. Sleep deprivation and pain perception. Sleep Med Rev [Internet]. 2006;10(5):357–69. Onen SH, Alloui A, Gross A, Eschallier A, Dubray C. The effects of total sleep deprivation, selective sleep interruption and sleep recovery on pain tolerance thresholds in healthy subjects. J Sleep Res [Internet]. 2001 Mar;4(1):35–42. Boyko Y, Ørding H, Jennum P. Sleep disturbances in critically ill patients in ICU: How much do we know? Acta Anaesthesiol Scand [Internet]. 2012;56(8):950–8. Mistraletti G, Carloni E, Cigada M, Zambrelli E, Taverna M, Sabbatini G, et al. Sleep and delirium in the intensive care unit. Minerva Anestesiol [Internet]. 2008;74(6):329–33. Freedman NS, Kotzer N, Schwab RJ. Patient perception of sleep quality and etiology of sleep disruption in the intensive care unit. Am J Respir Crit Care Med [Internet]. 1999;159(4 I):1155–62. Bourne RS, Minelli C, Mills GH, Kandler R. Clinical review: Sleep measurement in critical care patients: research and clinical implications. Crit Care [Internet]. 2007;11(4):226. Roche Campo F, Drouot X, Thille AW, Galia F, Cabello B, d’Ortho M-P, et al. Poor sleep quality is associated with late noninvasive ventilation failure in patients with acute hypercapnic respiratory failure. Crit Care Med [Internet]. 2010;38(2):477–85. McNicoll L, Pisani MA, Zhang Y, Ely EW, Siegel MD, Inouye SK. Delirium in the intensive care unit: occurrence and clinical course in older patients. J Am Geriatr Soc [Internet]. 2003;51(5):591–8. Ely EW, Speroff T, Gordon SM, Harrell FE, Inouye SK, Bernard GR. Delirium as a Predictor of Mortality in Mechanically Ventilated Patients in the Intensive Care Unit. JAMA [Internet]. 2004 Apr 14 ;291(14):1753–62. Weinhouse GL, Watson PL. Sedation and sleep disturbances in the ICU. Anesthesiol Clin [Internet]. 2011;29(4):675–85. Van Rompaey B, Elseviers MM, Van Drom W, Fromont V, Jorens PG. The effect of earplugs during the night on the onset of delirium and sleep perception: a randomized controlled trial in intensive care patients. Crit Care [Internet]. 2012;16(3):R73. Gabor JY, Cooper AB, Crombach SA, Lee B, Kadikar N, Bettger HE, et al. Contribution of the intensive care unit environment to sleep disruption in mechanically ventilated patients and healthy subjects. Am J Respir Crit Care Med [Internet]. 2003 Mar;1(5):708–15. Walder B, Francioli D, Meyer J-J, Lançon M, Romand J-A. Effects of guidelines implementation in a surgical intensive care unit to control nighttime light and noise levels. Crit Care Med [Internet]. 2000;28(7):2242. Tembo AC, Parker V. Factors that impact on sleep in intensive care patients. Intensive Crit Care Nurs [Internet]. 2009;25(6):314–22. Bosma KJ, Ranieri VM. Filtering out the noise: Evaluating the impact of noise and sound reduction strategies on sleep quality for ICU patients. Crit Care [Internet]. 2009;13(3):151. Friese RS. Sleep and recovery from critical illness and injury: a review of theory, current practice, and future directions. Crit Care Med [Internet]. 2008;36(3):697–705. Elliott R, McKinley S, Cistulli P, Fien M. Characterisation of sleep in intensive care using 24-hour polysomnography: An observational study. Crit Care. 2013;17(2). Figueroa-Ramos MI, Arroyo-Novoa CM, Lee KA, Padilla G, Puntillo KA. Sleep and delirium in ICU patients: A review of mechanisms and manifestations. Intensive Care Med [Internet]. 2009;35(5):781–95. Weinhouse GL, Schwab RJ, Watson PL, Patil N, Vaccaro B, Pandharipande P, et al. Bench-to-bedside review: Delirium in ICU patients - importance of sleep deprivation. Crit Care [Internet]. 2009;13(6):234. Litton E, Carnegie V, Elliott R, Webb SAR. The Efficacy of Earplugs as a Sleep Hygiene Strategy for Reducing Delirium in the ICU. Crit Care Med [Internet]. 2016; Publish Ah:1. Beltrami FG, Nguyen XL, Pichereau C, Maury E, Fleury B, Fagondes S. Sono na unidade de terapia intensiva. J Bras Pneumol [Internet]. 2015;41(6):539–46. Andersen JH, Boesen HC, Skovgaard Olsen K. Sleep in the Intensive Care Unit measured by polysomnography. Minerva Anestesiol [Internet]. 2013;79(7):804–15. Loomis AL, Harvey EN, Hobart GA. Cerebral states during sleep, as studied by human brain potentials. J Exp Psychol [Internet]. 1937;21(2):127–44. Aserinsky E, Kleitman N. Regularly occurring periods of eye motility, and concomitant phenomena, during sleep. Sci [Internet]. 1953;118(3062):273–4. Kales A, Rechtschaffen A, Washington DC, Bethesda. Md.,: U.S. National Institute of Neurological Diseases and Blindness, Neurological Information Network; 1968. Iber CC, Ancoli-israel S, Chesson AL, Quan SF, Medicine AA. of S. The AASM manual for the scoring of sleep and associated events; rules, terminology and technical specifications [Internet]. 1st ed. Westchester, IL: American Academy of Sleep Medicine; 2007. Freedman NS, Gazendam J, Levan L, Pack AI, Schwab RJ. Abnormal sleep/wake cycles and the effect of environmental noise on sleep disruption in the intensive care unit. Am J Respir Crit Care Med [Internet]. 2001;163(2):451–7. Drouot X, Roche-Campo F, Thille AW, Cabello B, Galia F, Margarit L, et al. A new classification for sleep analysis in critically ill patients. Sleep Med [Internet]. 2012;13(1):7–14. Foreman B, Westwood AJ, Claassen J, Bazil CW. Sleep in the Neurological Intensive Care Unit. J Clin Neurophysiol [Internet]. 2015;32(1):66–74. Watson PL. Measuring sleep in critically ill patients: beware the pitfalls. Crit Care [Internet]. 2007;11(4):159. Ambrogio C, Koebnick J, Quan SF, Ranieri M, Parthasarathy S. Assessment of sleep in ventilator-supported critically III patients. Sleep [Internet]. 2008;31(11):1559–68. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. Additional Declarations Competing interest reported. The BrainAmp DC32 amplifier, BrainVision recorder, and disposables were property of the investigating hospital. The Alice 6 LDx system was supplied by Philips Research Eindhoven. LR received partial funding (paid to institution) from Philips Research Eindhoven for a PhD position at the University Medical Center Groningen. EMH and PF were employed by Philips Research Eindhoven. The remaining authors did not have any conflicts of interest to declare. Cite Share Download PDF Status: Published Journal Publication published 01 Apr, 2025 Read the published version in BMC Research Notes → Version 1 posted Editorial decision: Revision requested 30 May, 2024 Reviews received at journal 17 May, 2024 Reviews received at journal 13 May, 2024 Reviewers agreed at journal 22 Apr, 2024 Reviewers agreed at journal 22 Apr, 2024 Reviewers invited by journal 20 Apr, 2024 Editor invited by journal 03 Apr, 2024 Submission checks completed at journal 29 Mar, 2024 Editor assigned by journal 29 Mar, 2024 First submitted to journal 21 Feb, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3975957","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":285405889,"identity":"ad900c7a-68ea-4d4a-8b9a-62c51d453fe2","order_by":0,"name":"Laurens Reinke","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAt0lEQVRIiWNgGAWjYBACPhCRAIT8UAEZglrYYFokGyACPMRpAWkyOEC8Ft6HDx7uSJMzvpGd+PELwx1itLAbGySeyTE2u5G7WVqG4RkxWtjYJBLbKhK33cjdxizBcJgoLew/gFrqN88gQQsbQ2JbToKBRO42xg9EaWFmYwY6LM1wxpm3m6UZDIjQws/exvjxZ1uyPH977saPPyoOyxHUwsCMzOYxIKwBFTD+IFXHKBgFo2AUjAgAADPCMit4HzQWAAAAAElFTkSuQmCC","orcid":"","institution":"University of Groningen, University Medical Center Groningen","correspondingAuthor":true,"prefix":"","firstName":"Laurens","middleName":"","lastName":"Reinke","suffix":""},{"id":285405890,"identity":"7ce4ef3f-6331-46a5-8e9a-8821b1c009b8","order_by":1,"name":"Esther M. Heide","email":"","orcid":"","institution":"Philips Research","correspondingAuthor":false,"prefix":"","firstName":"Esther","middleName":"M.","lastName":"Heide","suffix":""},{"id":285405891,"identity":"9268c7cc-fcc6-4a12-ab03-3cdc27c3e64c","order_by":2,"name":"Pedro Fonseca","email":"","orcid":"","institution":"Philips Research","correspondingAuthor":false,"prefix":"","firstName":"Pedro","middleName":"","lastName":"Fonseca","suffix":""},{"id":285405894,"identity":"d140d555-38d7-4488-a7c0-38c99520e8ad","order_by":3,"name":"Anthony R. Absalom","email":"","orcid":"","institution":"University of Groningen, University Medical Center Groningen","correspondingAuthor":false,"prefix":"","firstName":"Anthony","middleName":"R.","lastName":"Absalom","suffix":""},{"id":285405896,"identity":"bd406b42-f8e1-483c-a1df-7e6f4e3934ae","order_by":4,"name":"Jaap E. Tulleken","email":"","orcid":"","institution":"University of Groningen, University Medical Center Groningen","correspondingAuthor":false,"prefix":"","firstName":"Jaap","middleName":"E.","lastName":"Tulleken","suffix":""}],"badges":[],"createdAt":"2024-02-21 14:32:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3975957/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3975957/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13104-025-07198-z","type":"published","date":"2025-04-01T15:57:41+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":54036592,"identity":"2b6f73a1-0fbc-4325-971c-6bda9bec42ad","added_by":"auto","created_at":"2024-04-03 17:03:14","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":112726,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix for scoring by human scorer M1 versus human scorer M2. Percentages are calculated from class-totals as scored by M1.\u003c/p\u003e","description":"","filename":"Figure1ConfusionmatrixM1vM2NoCaption.png","url":"https://assets-eu.researchsquare.com/files/rs-3975957/v1/6c1e59f8f2e20949b963144c.png"},{"id":80082396,"identity":"9f433708-0e85-4435-b42f-768324924ee0","added_by":"auto","created_at":"2025-04-07 16:08:47","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":687204,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3975957/v1/b93b47a6-b257-48b9-946b-20d2a9e848a4.pdf"}],"financialInterests":"Competing interest reported. The BrainAmp DC32 amplifier, BrainVision recorder, and disposables were property of the investigating hospital. The Alice 6 LDx system was supplied by Philips Research Eindhoven. LR received partial funding (paid to institution) from Philips Research Eindhoven for a PhD position at the University Medical Center Groningen. EMH and PF were employed by Philips Research Eindhoven. The remaining authors did not have any conflicts of interest to declare.","formattedTitle":"Inter-rater disagreement in manual scoring of intensive care unit sleep data","fulltext":[{"header":"Introduction","content":"\u003cp\u003eSleep is a dynamic, complex physiological process essential for homeostasis, recovery, and survival (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Disrupted or delayed sleep is associated with impaired immune function (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e), increased susceptibility to infections and impaired wound healing (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e), impaired metabolic and endocrine function (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e), increased pain perception (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e) and impairment of neurophysiologic organization and memory consolidation (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eSleep deprivation affects up to 60% of all critically ill patients admitted to an intensive care unit (ICU) (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). Sleep among these patients is often fragmented by frequent arousals and awakenings which hamper transitions to deeper stages of sleep, reduced duration of sleep, and disturbed distribution of sleep with up to half of the total sleep time occurring during the day (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). Poor sleep during critical illness is considered to be a major stressor for patients during and after ICU admission. It is associated with the development of ICU delirium and long-term cognitive decline, and has detrimental effects on recovery, morbidity, and mortality (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe ICU is a unique environment where a multitude of intrinsic and environmental factors may hamper sleep (16, 17. 18, 19, 20, 21, 22). Although previous studies have provided new insights into the etiology and possible prevention of disturbed sleep in the ICU, their scope, statistical significance and reliability have thus far been constrained by the logistical challenges of measuring and assessing sleep objectively (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eElectroencephalography (EEG) has historically been the primary tool for objective sleep monitoring (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e). Polysomnography (PSG), combining EEG electromyography (EMG), and electrooculography (EOG) is the technique used to investigate sleep. The visual and manual annotation or scoring of these recordings commonly follows criteria originally set by Kales and Rechtschaffen (\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e), with additional changes later culminating in the American Academy of Sleep Medicine (AASM) Manual for the Scoring of Sleep (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e). Hundreds or even thousands of 30 second epochs each comprising multiple channels of PSG data are typically processed by a single human expert. Although this method is considered to be the gold standard for routine clinical sleep analysis, most PSG studies in critically ill patients report difficulties in setting up, maintaining, and manually processing and scoring ICU sleep recordings (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e). The practical expertise required to apply and maintain the array of electrodes required for human scoring further limits scalability and increases costs. Furthermore, the reliability and repeatability of manual analysis of ICU sleep recordings is lower than for other clinical recordings (\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e). While Elliott et al. reported observed \u0026lsquo;reasonable\u0026rsquo; to \u0026lsquo;good\u0026rsquo; agreement between two combinations of 3 human scorers in discerning wake from sleep activity, the agreement on detailed sleep staging was much lower depending on individual sleep stages and the combination of human scorers (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe objective of this study is to investigate human inter-rater agreement in sleep staging following the AASM rules for sleep scoring, in a heterogeneous population of ICU patients.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy population and patient recruitment\u003c/h2\u003e \u003cp\u003eWe obtained 70 PSG recordings during an observational study (Trial NL5197, NTR5345) primarily investigating the influence of disrupted biorhythms on the quantity and quality of sleep among patients of the department of Critical Care of the University Medical Center Groningen (UMCG). After approval by the local ethics committee (UMCG METc, registration number 2015/00295), data collection started in September 2015 and finished in September 2018. All adult patients without a history of sleep pathology, an expected ICU stay of at least 48 hours, and a Richmond Agitation and Sedation Scale (RASS) above \u0026minus;\u0026thinsp;3 were eligible for inclusion in the study. Informed consent was gained from patients with capacity to do so. For patient lacking capacity, informed consent was first obtained from their legal representatives, followed by consent after they recovered consciousness. Neurosurgical patients, and patients taking melatonin supplements were excluded from participation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eData acquisition\u003c/h2\u003e \u003cp\u003ePSG was recorded for a period of 24\u0026ndash;72 hours depending on patient\u0026rsquo;s tolerance, RASS scores, and ICU length of stay. The recording consisted of six EEG channels (F3, A1, A2, C3, C4, O1), two EOG channels and EMG of the left and right masseter or submental muscles. Ag/AgCl EEG-electrodes were placed according to the international 10\u0026ndash;20 system after skin preparation according to standardized techniques. A BrainAmp DC32 amplifier with a BrainVision recorder (Brain Vision Solutions, Montreal, Canada) or an Alice 6 LDx system (Philips Respironics, Murrysville, USA) was used. EEG was recorded with a sample frequency of 256Hz. Anonymized data were stored for sleep scoring by two experienced human experts.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eSleep analysis\u003c/h2\u003e \u003cp\u003eAll recorded data were blindly assessed for data quality by a human expert and sets of sufficient quality were then analysed by a human expert scorer (M\u003csub\u003e1\u003c/sub\u003e). We randomly selected 20 patients for further analysis by an additional human expert scorer (M\u003csub\u003e2\u003c/sub\u003e). Human expert scorers were free to select either the C4-A1 or C3-A2 EEG channel for scoring depending on signal quality.\u003c/p\u003e \u003cp\u003e The scoring of discrete wake and sleep stages (rapid eye-movement sleep, REM; non-REM sleep stages, N1, N2, N3) according to the latest AASM scoring guidelines by the scorers was done by visual interpretation of individual 30 second epochs in the Brain RT software (OSG, Rumst, Belgium).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eStatistical analysis\u003c/h2\u003e \u003cp\u003eSleep-related parameters were calculated using Matlab (Matlab 2014b, Natick, MA, USA). Statistics were calculated using SPSS 24 (2016, IBM, Armonk, NY, USA). Cohen\u0026rsquo;s Kappa statistic was used to evaluate epoch-per-epoch agreement between human expert scorers for all sleep stages individually and for full 5-stage sleep scoring. Cohen\u0026rsquo;s Kappa is a dimensionless index that corrects for chance agreement due to imbalanced datasets, such as the imbalanced distribution of sleep and wake stages. Scoring agreement statistics for wake and individual sleep classes were calculated using a binary one-versus-rest strategy. For normative interpretation of inter-rater agreement we used the guidelines by Landis and Koch (\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e). Spearman\u0026rsquo;s correlation coefficient was used to quantify the correlation between inter-rater agreements, predicted mortality, and mortality. For estimation of statistical significance, an alpha of 0.05 was used. Unless indicated otherwise, results are presented as mean values (standard deviation).\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eSeventy patients were included in the main study. PSG data from 4 (5.71%) were lost due to undetected technical failure of EEG equipment during measurement. A further 5 (7.14%) recordings were deemed entirely unscorable by the human expert scorers. Of the remaining 61 patient recordings a median of 0.22% of epochs (0.01\u0026ndash;0.56% interquartile range, IQR) were entirely excluded due to artifacts, leaving 339,901 30-second epochs (2832.51 hours) for further analysis by scorer M\u003csub\u003e1\u003c/sub\u003e. In total 20 recordings were randomly selected for classification by a second scorer (M\u003csub\u003e2\u003c/sub\u003e), 3 (15%) were rejected entirely due to low signal quality. Of the remaining 17 patient recordings 0.26% of epochs (0.09\u0026ndash;0.86% IQR) were entirely excluded due to artifacts, leaving 51,454 epochs (428.78 hours) for analysis of inter-rater agreement between two human scorers. Patient characteristics for all valid recordings and for the subgroup scored by M\u003csub\u003e2\u003c/sub\u003e are summarized in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. Recordings randomly selected for additional classification by M\u003csub\u003e2\u003c/sub\u003e were from patients with a significantly shorter median ICU stay 7.01 (4.01\u0026ndash;18.98 IQR) days, p\u0026thinsp;=\u0026thinsp;0.014) days versus (14.01 (6.02\u0026ndash;29.51 IQR), but there were no other significant differences from the rest of the sample.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDemographics of the group with valid recording, and the subgroup randomly selected for additional analysis by M\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eValid recordings \u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;61)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAnalysed by M\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;17)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003ep-value\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCharacteristic\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSex\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e23 (37.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8 (47.06)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eICU admission diagnosis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003esurgical\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e19 (31.15)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5 (29.41)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003emedical\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e42 (68.85)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12 (70.59)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e12 month mortality\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e19 (31.15)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4 (23.53)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003eMedian (IQR)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003eMedian (IQR)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBMI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e26 (\u003cspan additionalcitationids=\"CR24 CR25 CR26 CR27 CR28\" citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e28 (\u003cspan additionalcitationids=\"CR25 CR26 CR27 CR28 CR29 CR30 CR31\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAge, years\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e60 (52\u0026ndash;67)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e63 (52\u0026ndash;67)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDuration of hospital stay, days\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e31 (19-55.49)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e21.99 (15.99-43)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDuration of ICU stay, days\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e10.99 (5.51\u0026ndash;25.50)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7.01 (4.01\u0026ndash;18.98)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDuration of recording, hours\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e47.74 (17.22)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e46.66 (43-65.89)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAPACHE IV\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e69 (47\u0026ndash;82)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e74 (61.25\u0026ndash;81.25)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSAPS II\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e40 (31\u0026ndash;49)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e42 (36\u0026ndash;44)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMechanical ventilation, days\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e7 (\u003cspan additionalcitationids=\"CR2 CR3 CR4 CR5 CR6 CR7 CR8 CR9 CR10 CR11 CR12 CR13 CR14 CR15\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5 (\u003cspan additionalcitationids=\"CR2 CR3 CR4 CR5 CR6 CR7 CR8 CR9 CR10 CR11 CR12 CR13\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003eMean (SD)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedication dose per day\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBenzodiazepines, mg Lorazepam equivalent\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.55 (4.82)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.95 (8.77)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOpioids, mg Morphine equivalent\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.80 (3.63)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.95 (1.92)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePropofol 2%, ml\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.81 (7.27)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.96 (10.59)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e indicates the prevalence of each sleep stage (according to M\u003csub\u003e1\u003c/sub\u003e scorings), and agreement between the two human scorers for the 5-class scoring task, as well as for each class versus the rest. Mean κ agreement for 5-classes was 0.36, with the best agreement obtained for Wake, with a κ of 0.46, and worst for REM, with a κ of 0.12. REM was also the least prevalent class, with an average number of 0.00 hours per 24-hour period.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAgreement between human scorers (M\u003csub\u003e1\u003c/sub\u003e vs. M\u003csub\u003e2\u003c/sub\u003e)\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eClass\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWake\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eREM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN2\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN3\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003e5 classes\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrevalence based on M\u003csub\u003e1\u003c/sub\u003e, hours per day (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e13.98 (8.45)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.00 (0.01)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.22 (1.34)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e6.15 (5.16)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e2.66 (4.09)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e24\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eKappa, - (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.46 (0.27)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.12 (0.24)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.13 (0.13)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.32 (0.21)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.26 (0.24)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.36 (0.21)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAccuracy, % (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e80.62 (14.57)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e97.89 (5.26)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e90.33 (8.24)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e80.65 (13.24)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e91.54 (9.22)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e70.51 (17.5)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSensitivity, % (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e81.3 (21.99)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11.72 (24.78)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e15.92 (14.76)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e53.02 (24.92)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e36.04 (30.97)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSpecificity, % (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e64.86 (26.93)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e99.87 (0.33)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e96.35 (3.92)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e84.89 (13.73)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e95.01 (8.96)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePPV, % (SD)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e79.55 (19.38)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e55.56 (39.59)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e24.1 (17.91)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e44.74 (22.67)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e48.63 (36.49)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"7\"\u003ePerformance statistics for individual classes were calculated using a binary one-vs-rest strategy.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003ePer-subject κ agreement between M\u003csub\u003e1\u003c/sub\u003e and M\u003csub\u003e2\u003c/sub\u003e correlated significantly with the Simplified Acute Physiology Score (SAPS II) predictor of mortality (r=-0.506, p\u0026thinsp;=\u0026thinsp;0.038), and with recorded 12-month mortality (r=-0.524, p\u0026thinsp;=\u0026thinsp;0.031).\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates the confusion matrix for the pooled classification of all epochs in the recordings scored by both human experts scorers. Even for the class with the best κagreement, i.e., Wake, inconsistent scoring was found between the two scorers: M\u003csub\u003e2\u003c/sub\u003e scored a large proportion of M\u003csub\u003e1\u003c/sub\u003e-Wake epochs as N2 and to a certain degree, even N3, whereas M\u003csub\u003e1\u003c/sub\u003e scored a larger proportion of M\u003csub\u003e2\u003c/sub\u003e-Wake as N1.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eHuman inter-rater agreement in our sample was comparable to that between human scorers in other studies of ICU sleep. Elliott et al. (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) reported a Cohen\u0026rsquo;s kappa of κ\u0026thinsp;=\u0026thinsp;0.58\u0026ndash;0.68, which they deemed to be \u0026lsquo;reasonable\u0026rsquo; to \u0026lsquo;good\u0026rsquo; agreement (\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e), for sleep-wake scoring by two combinations of 3 manual/human scorers. Agreement for the results of detailed sleep staging was much lower, with only slight agreement for stage N1 (κ\u0026thinsp;=\u0026thinsp;0.08\u0026ndash;0.12), moderate agreement for N2 and REM (κ\u0026thinsp;=\u0026thinsp;0.55\u0026ndash;0.58 and κ\u0026thinsp;=\u0026thinsp;0.41\u0026ndash;0.44, respectively), and slight to good agreement for slow wave sleep (κ\u0026thinsp;=\u0026thinsp;0.20\u0026ndash;0.76), depending on the combinations of manual scorers. Similarly, disagreement in our sample was highest for REM and N1, likely due to a general deficit of this stage of sleep in ICU populations. Additional disagreement was found between individual sleep stages and the wake stage, which could be the result of the relatively high amount of EEG and EMG artifacts in this intensive care population being interpreted as proof of wakefulness. The remainder of substantial disagreement exists between the already notoriously difficult to separate N2 and N3 stages.\u003c/p\u003e \u003cp\u003eAmbrogio et al. compared the agreement between two manual scorers for PSG recordings of 14 mechanically ventilated ICU patients and 17 ambulatory control patients (\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e). Inter-rater reliability was good (κ\u0026thinsp;=\u0026thinsp;0.74) for recordings of ambulatory patients, but there was only slight agreement on the scoring of recordings of ICU patients (κ\u0026thinsp;=\u0026thinsp;0.19).\u003c/p\u003e \u003cp\u003eIn conclusion, our data further confirms that the applicability of the AASM criteria for most ICU recorded PSG-data is debatable, particularly so among the most unwell patients. Although the source of confusion can be partially attributed to the high amount of EEG and EMG artifacts, this does not always explain the disparity in scoring between otherwise relatively unambiguous stages, such as Wake and N3, or REM and N2. For these patients, rather than deeming the scoring rules as inadequate, a better understanding of the factors driving this disparity could help shed light on the sleep of this critically ill population.\u003c/p\u003e"},{"header":"Limitations","content":"\u003cp\u003ePSG is notoriously labour-intensive during set-up, maintenance, and analysis, which limited the sample size of this study a priori. Despite our best efforts, the amount of usable data was further limited by artifacts from frequent and intensive care, electromagnetic pollution, motor restlessness, excessive sweating and other technical challenges. Study inclusion and exclusion criteria were chosen to minimize the likelihood of unproductive measurements but may have decreased the already limited generalizability of results from inherently heterogeneous ICU patients.\u003c/p\u003e \u003cp\u003eStudy inclusion did not always start immediately after ICU admission and varied in duration due to the unpredictable progression of critical illness. This caused an imbalance in the contribution of individual recordings to aggregated means, which is why all statistics were calculated from per-subject means.\u003c/p\u003e \u003cp\u003eICU patients could not be relied upon for subjective sleep evaluation, and the neurocognitive state of subjects was not assessed.\u003c/p\u003e \u003cp\u003eThe limited practical scalability of polysomnography and human expert sleep scoring has not only restricted the sample size of our comparison but has also limited our ability to do proper consensus scoring or full-sample multi-rater human expert scoring for this investigation. Future efforts to provide more comprehensive investigation of interrater agreements are still encouraged and could benefit from aggregating recordings from previous studies and the adherence to standardized scoring.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eAASM\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAmerican Academy of Sleep Medicine\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEEG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eelectroencephalography\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEMG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eelectromyography\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEOG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eelectrooculography\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eICU\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eintensive care unit\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eIQR\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003einterquartile range\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePSG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003epolysomnography\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eRASS\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eRichmond Agitation and Sedation Scale\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eREM\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003erapid eye-movement (sleep)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eN1, N2, N3\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003enon-REM sleep stages 1, 2, 3\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSAPS II\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003esimplified acute physiology score\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eUMCG\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eUniversity Medical Center Groningen\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTrial was registered as \u0026ldquo;Sleep and biorhythm in the ICU\u0026rdquo;, in trialregister.nl, with number NL5197 (NTR5345) (https://www.trialregister.nl/trial/5197).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe study was approved by the local ethics committee (UMCG METc, registration number 2015/00295). Informed consent was obtained from patients with capacity to do so. Otherwise, informed consent was first obtained from their legal representatives, followed by patients after they recovered consciousness.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData are available by contacting the corresponding author.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAt the time of writing, EMH and PF are employed by Philips Research Eindhoven. The remaining authors did not have any conflicts of interest to declare.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo funding was obtained for the purpose of this study. The BrainAmp DC32 amplifier, BrainVision recorder, and disposables were property of the investigating hospital. The Alice 6 LDx system was supplied by Philips Research Eindhoven. LR received partial funding (paid to institution) from Philips Research Eindhoven for a PhD position at the University Medical Center Groningen.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eL.R. drafted the first manuscript, all other authors provided feedback on drafts of the paper. All authors were equally responsible for the conception of the study. L.R. was responsible for implementation of the study and enrolled participants. A.R.A., J.E.T. and L.R. collated the data and analysed results. All authors contributed to, read, and approved the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements(optional)\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe thank the entire UMCG ICV research team and student team for their support in setting up and executing this study. We thank dr. J.H. van der Hoeven for analysing PSG data, and Philips Sleep \u0026amp; Respiratory Care for automated sleep analysis. The initiation and success of this study is owed largely to the contribution of Technical Physicians in training.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eWeinhouse GL, Schwab RJ. Sleep in the critically ill patient. Sleep [Internet]. 2006;29(5):707\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKamdar BB, Needham DM, Collop NA. Sleep Deprivation in Critical Illness: Its Role in Physical and Psychological Recovery. J Intensive Care Med [Internet]. 2012;27(2):97\u0026ndash;111.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIrwin M, McClintick J, Costlow C, Fortner M, White J, Gillin JC. Partial night sleep deprivation reduces natural killer and celhdar immune responses in humans. FASEB J [Internet]. 1996;10(5):643\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCooper AB, Thornley KS, Young GB, Slutsky AS, Stewart TE, Hanly PJ. Sleep in critically ill patients requiring mechanical ventilation. Chest [Internet]. 2000 Mar;1(3):809\u0026ndash;18.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFriese RS, Diaz-Arrastia R, McBride D, Frankel H, Gentilello LM. Quantity and quality of sleep in the surgical intensive care unit: are our patients sleeping? J Trauma [Internet]. 2007;63(6):1210\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSpiegel K, Leproult R, Van Cauter E. Impact of sleep debt on metabolic and endocrine function. Lancet [Internet] 1999 Oct 23 ;354(9188):1435\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLautenbacher S, Kundermann B, Krieg JC. Sleep deprivation and pain perception. Sleep Med Rev [Internet]. 2006;10(5):357\u0026ndash;69.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOnen SH, Alloui A, Gross A, Eschallier A, Dubray C. The effects of total sleep deprivation, selective sleep interruption and sleep recovery on pain tolerance thresholds in healthy subjects. J Sleep Res [Internet]. 2001 Mar;4(1):35\u0026ndash;42.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBoyko Y, \u0026Oslash;rding H, Jennum P. Sleep disturbances in critically ill patients in ICU: How much do we know? Acta Anaesthesiol Scand [Internet]. 2012;56(8):950\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMistraletti G, Carloni E, Cigada M, Zambrelli E, Taverna M, Sabbatini G, et al. Sleep and delirium in the intensive care unit. Minerva Anestesiol [Internet]. 2008;74(6):329\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFreedman NS, Kotzer N, Schwab RJ. Patient perception of sleep quality and etiology of sleep disruption in the intensive care unit. Am J Respir Crit Care Med [Internet]. 1999;159(4 I):1155\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBourne RS, Minelli C, Mills GH, Kandler R. Clinical review: Sleep measurement in critical care patients: research and clinical implications. Crit Care [Internet]. 2007;11(4):226.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRoche Campo F, Drouot X, Thille AW, Galia F, Cabello B, d\u0026rsquo;Ortho M-P, et al. Poor sleep quality is associated with late noninvasive ventilation failure in patients with acute hypercapnic respiratory failure. Crit Care Med [Internet]. 2010;38(2):477\u0026ndash;85.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcNicoll L, Pisani MA, Zhang Y, Ely EW, Siegel MD, Inouye SK. Delirium in the intensive care unit: occurrence and clinical course in older patients. J Am Geriatr Soc [Internet]. 2003;51(5):591\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEly EW, Speroff T, Gordon SM, Harrell FE, Inouye SK, Bernard GR. Delirium as a Predictor of Mortality in Mechanically Ventilated Patients in the Intensive Care Unit. JAMA [Internet]. 2004 Apr 14 ;291(14):1753\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWeinhouse GL, Watson PL. Sedation and sleep disturbances in the ICU. Anesthesiol Clin [Internet]. 2011;29(4):675\u0026ndash;85.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVan Rompaey B, Elseviers MM, Van Drom W, Fromont V, Jorens PG. The effect of earplugs during the night on the onset of delirium and sleep perception: a randomized controlled trial in intensive care patients. Crit Care [Internet]. 2012;16(3):R73.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGabor JY, Cooper AB, Crombach SA, Lee B, Kadikar N, Bettger HE, et al. Contribution of the intensive care unit environment to sleep disruption in mechanically ventilated patients and healthy subjects. Am J Respir Crit Care Med [Internet]. 2003 Mar;1(5):708\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWalder B, Francioli D, Meyer J-J, Lan\u0026ccedil;on M, Romand J-A. Effects of guidelines implementation in a surgical intensive care unit to control nighttime light and noise levels. Crit Care Med [Internet]. 2000;28(7):2242.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTembo AC, Parker V. Factors that impact on sleep in intensive care patients. Intensive Crit Care Nurs [Internet]. 2009;25(6):314\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBosma KJ, Ranieri VM. Filtering out the noise: Evaluating the impact of noise and sound reduction strategies on sleep quality for ICU patients. Crit Care [Internet]. 2009;13(3):151.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFriese RS. Sleep and recovery from critical illness and injury: a review of theory, current practice, and future directions. Crit Care Med [Internet]. 2008;36(3):697\u0026ndash;705.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eElliott R, McKinley S, Cistulli P, Fien M. Characterisation of sleep in intensive care using 24-hour polysomnography: An observational study. Crit Care. 2013;17(2).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFigueroa-Ramos MI, Arroyo-Novoa CM, Lee KA, Padilla G, Puntillo KA. Sleep and delirium in ICU patients: A review of mechanisms and manifestations. Intensive Care Med [Internet]. 2009;35(5):781\u0026ndash;95.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWeinhouse GL, Schwab RJ, Watson PL, Patil N, Vaccaro B, Pandharipande P, et al. Bench-to-bedside review: Delirium in ICU patients - importance of sleep deprivation. Crit Care [Internet]. 2009;13(6):234.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLitton E, Carnegie V, Elliott R, Webb SAR. The Efficacy of Earplugs as a Sleep Hygiene Strategy for Reducing Delirium in the ICU. Crit Care Med [Internet]. 2016; Publish Ah:1.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBeltrami FG, Nguyen XL, Pichereau C, Maury E, Fleury B, Fagondes S. Sono na unidade de terapia intensiva. J Bras Pneumol [Internet]. 2015;41(6):539\u0026ndash;46.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndersen JH, Boesen HC, Skovgaard Olsen K. Sleep in the Intensive Care Unit measured by polysomnography. Minerva Anestesiol [Internet]. 2013;79(7):804\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoomis AL, Harvey EN, Hobart GA. Cerebral states during sleep, as studied by human brain potentials. J Exp Psychol [Internet]. 1937;21(2):127\u0026ndash;44.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAserinsky E, Kleitman N. Regularly occurring periods of eye motility, and concomitant phenomena, during sleep. Sci [Internet]. 1953;118(3062):273\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKales A, Rechtschaffen A, Washington DC, Bethesda. Md.,: U.S. National Institute of Neurological Diseases and Blindness, Neurological Information Network; 1968.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIber CC, Ancoli-israel S, Chesson AL, Quan SF, Medicine AA. of S. The AASM manual for the scoring of sleep and associated events; rules, terminology and technical specifications [Internet]. 1st ed. Westchester, IL: American Academy of Sleep Medicine; 2007.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFreedman NS, Gazendam J, Levan L, Pack AI, Schwab RJ. Abnormal sleep/wake cycles and the effect of environmental noise on sleep disruption in the intensive care unit. Am J Respir Crit Care Med [Internet]. 2001;163(2):451\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDrouot X, Roche-Campo F, Thille AW, Cabello B, Galia F, Margarit L, et al. A new classification for sleep analysis in critically ill patients. Sleep Med [Internet]. 2012;13(1):7\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eForeman B, Westwood AJ, Claassen J, Bazil CW. Sleep in the Neurological Intensive Care Unit. J Clin Neurophysiol [Internet]. 2015;32(1):66\u0026ndash;74.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWatson PL. Measuring sleep in critically ill patients: beware the pitfalls. Crit Care [Internet]. 2007;11(4):159.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAmbrogio C, Koebnick J, Quan SF, Ranieri M, Parthasarathy S. Assessment of sleep in ventilator-supported critically III patients. Sleep [Internet]. 2008;31(11):1559\u0026ndash;68.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLandis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159\u0026ndash;74.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-research-notes","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"resn","sideBox":"Learn more about [BMC Research Notes](http://bmcresnotes.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/resn/default.aspx","title":"BMC Research Notes","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3975957/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3975957/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eObjective\u003c/strong\u003e: Severe sleep disruption is common among intensive care unit (ICU) patients. However, the applicability of standard sleep scoring guidelines by the American Academy of Sleep Medicine (AASM) has been questioned, with most polysomnography (PSG) studies in critically ill patients reporting difficulties in setting up and processing and scoring the recordings. The present study explores human inter-rater agreement in sleep stage scoring following the AASM guidelines, within a heterogenous ICU patient cohort.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e: Two human experts independently scored a total of 51,454 epochs in 20 PSG recordings acquired at the ICU. Epoch-per-epoch comparison of scored stages revealed a Cohen’s κ coefficient of agreement of 0.36 for standard 5-stage scoring. Highest agreement occurred in Wake (κ = 0.46), while REM showed the lowest (κ = 0.12). Significant correlations were found between inter-rater agreement, and Simplified Acute Physiology Score (SAPS II, r=-0.506, p=0.038), and 12-month mortality (r=-0.524, p=0.031).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eComparison with similar studies underscore challenges in applying AASM criteria to ICU patients. Despite accounting for artifacts, disparities persisted, emphasizing the need for a nuanced exploration of factors influencing scoring inconsistencies in critically ill patients.\u003c/p\u003e","manuscriptTitle":"Inter-rater disagreement in manual scoring of intensive care unit sleep data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-03 17:03:09","doi":"10.21203/rs.3.rs-3975957/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-05-30T11:57:09+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-17T14:46:44+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-13T10:57:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"9531fb08-60b1-4fbc-ac9c-d4bc68312be9","date":"2024-04-22T10:31:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"0aaf4d4a-4735-4e99-b1c4-3450bf99eb57","date":"2024-04-22T08:42:41+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-04-20T08:10:47+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-04-03T15:47:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-03-29T14:05:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-03-29T14:05:47+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Research Notes","date":"2024-02-21T14:30:01+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-research-notes","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"resn","sideBox":"Learn more about [BMC Research Notes](http://bmcresnotes.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/resn/default.aspx","title":"BMC Research Notes","twitterHandle":"@BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"87553311-056b-4bae-82eb-af10e4d58fa1","owner":[],"postedDate":"April 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-04-07T16:06:30+00:00","versionOfRecord":{"articleIdentity":"rs-3975957","link":"https://doi.org/10.1186/s13104-025-07198-z","journal":{"identity":"bmc-research-notes","isVorOnly":false,"title":"BMC Research Notes"},"publishedOn":"2025-04-01 15:57:41","publishedOnDateReadable":"April 1st, 2025"},"versionCreatedAt":"2024-04-03 17:03:09","video":"","vorDoi":"10.1186/s13104-025-07198-z","vorDoiUrl":"https://doi.org/10.1186/s13104-025-07198-z","workflowStages":[]},"version":"v1","identity":"rs-3975957","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3975957","identity":"rs-3975957","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}