Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use

Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use Frances Griffith, Garrett Ash, Madilyn Augustine, Leah Latimer, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3977182/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Evaluating user experiences with digital interventions is critical to increase uptake and adherence, but traditional methods have limitations. We incorporated natural language processing (NLP) with convergent mixed methods to evaluate a personalized feedback and coaching digital sleep intervention for alcohol risk reduction: ‘Call it a Night’ ( CIAN; N = 120). In this randomized clinical trial with young adults with heavy drinking, control conditions were A + SM : web-based advice + active and passive monitoring; and A : advice + passive monitoring. Findings converged to show that the CIAN treatment condition group found feedback and coaching most helpful, whereas participants across conditions generally found advice helpful. Further, most participants across groups were interested in varied whole-health sleep-related factors besides alcohol use (e.g., physical activity), and many appreciated increased awareness through monitoring with digital tools. All groups had high adherence, satisfaction, and reported feasibility, but participants in CIAN and A + SM reported significantly higher effectiveness than those in A. NLP corroborated positive sentiments across groups and added critical insight that sleep, not alcohol use, was a main participant motivator. Digital sleep interventions are an acceptable, novel alcohol treatment strategy, and improving sleep and overall wellness may be important motivations for young adults. Further, NLP provides an efficient convergent method for evaluating experiences with digital interventions. Biological sciences/Computational biology and bioinformatics/Machine learning Scientific community and society/Social sciences/Psychology/Human behaviour Health sciences/Risk factors Health sciences/Medical research/Translational research natural language processing digital health sleep alcohol use user experience Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Most mental health disorders have a peak age of onset during or before young adulthood (18–25 years) 1 . However, less than half of young adults with any mental health disorder receive treatment, with the proportion dropping to around one-third among young adults of color 2 . Clinical services are often unavailable or unaffordable for young adults, and the gap between treatment demand and availability has increased since the COVID-19-pandemic 3 . Further, young adults may not seek treatment for some highly prevalent but normalized mental health concerns, like alcohol use disorders (AUDs) and heavy (“binge”) drinking 2 , 4 – 5 . In contrast, young adults show concern for other common wellness issues, like sleep quality, that are related to alcohol use 6 – 7 . In a sample of young adults with similar levels of sleep complaints and heavy drinking, most (80%) were concerned about sleep whereas very few (5%) were concerned about drinking 8 . Therefore, sleep interventions could offer a more appealing alternative for young adults facing sensitive issues like alcohol risk 9 . Digital interventions can also increase appeal, access, and convenience of mental health treatment for young adults 10 . This population makes extensive use of digital health tools compared to other adults 11 , with generally positive experiences 12 . Despite a saturated market of mobile mental health apps and other digital tools, user ratings may be discrepant and uninformative to app designers and potential users alike 13 . Over 95% of users stop using most mobile mental health apps after 30 days 14 and adherence and engagement are generally low 15 . In the case of sleep and alcohol apps, user engagement may increase with features like personalized feedback, interaction and support from providers, self-monitoring, and user-friendliness 16 – 19 . Further, gamification may significantly reduce attrition in mobile mental health app use in general 20 . With the incorporation of machine learning, apps are also becoming increasingly more personalized. Assessing both users’ perspectives and intervention quality is critical to promote higher treatment uptake and adherence in a rapidly growing, global digital health market 21 . Therefore, implementing precise, consistent evaluations of user experiences represents a significant gap in digital intervention research. Incorporating natural language processing (NLP) with convergent mixed methods can support user-centered design for digital health interventions 22 . Natural language processing is an umbrella term for a growing suite of machine learning methods that uses artificial intelligence (AI) to understand (e.g., summarize, retrieve information) and/or generate language content. Traditional evaluation methods, such as qualitative thematic analysis of exit interviews, can reveal nuance and detail, but can be impractical, especially with large sample sizes 23 . Further, exit surveys using Likert-type satisfaction scales may be subject to response bias. A hybrid approach combining NLP, quantitative survey analysis, and targeted qualitative interview analysis, may reveal broad and rich user experiences 23 while maximizing researcher time and effort. The current study used convergent mixed methods to evaluate user experiences in a randomized clinical trial of “Call it a Night” ( CIAN) and two control conditions (see Fig. 1 ). CIAN is a personalized feedback and coaching digital sleep intervention for young adults that addresses heavy alcohol and other substance use (NCT #03658954). In the parent trial 6 , CIAN was tested against control conditions comprising either web-based advice only ( A) or advice + active diary self-monitoring ( A + SM) . All participants wore sleep and alcohol biosensors. NLP methods, particularly topic modeling analysis (Latent Dirichlet Allocation) and sentiment analysis, were selected to assess convergence with qualitative thematic analysis of exit interviews and descriptive and predictive analysis with exit surveys. The aims of the study were to leverage these NLP evaluation methods to: characterize young adults’ user experiences during the digital sleep interventions for heavy drinking and determine whether user experiences varied with their demography (e.g., age, race, gender, student status, psychiatric diagnoses) or trial condition. Results Sample We recruited 120 trial participants, and 118 completed the exit survey. Half (51%) were female and 80% were white with a mean age of 21.14 years (see Table 1 ). Three-fourths (74%) were college students. Most (81%) met lifetime criteria for any MHD or substance use disorder (SUD) based on a diagnostic interview, and most also met lifetime criteria for an AUD (72%). Table 1 CIAN Exit Survey Sample Characteristics (n = 118) Intervention group A A + SM CIAN Total a Sample characteristic n (% of 29) n (% of 29) n (% of 60) n (% of 118) Gender Female 14 (48.3) 16 (55.2) 30 (50.0) 60 (50.8) Male 15 (51.7) 13 (44.8) 30 (50.0) 58 (49.2) Race Asian 3 (10.3) 2 (6.9) 5 (8.3) 10 (8.5) Black 1 (3.4) 1 (3.4) 7 (11.7) 9 (7.6) White 25 (86.2) 24 (82.8) 45 (75.0) 94 (79.7) Multiracial 0 (0.0) 0 (0.0) 1 (1.7) 1 (0.8) Other 0 (0.0) 2 (6.9) 2 (3.3) 4 (3.4) Ethnicity Not Hispanic, Latine 24 (82.8) 24 (82.8) 51 (85.0) 99 (83.9) Hispanic, Latine 5 (17.2) 5 (17.2) 9 (15.0) 19 (16.1) Student status Student 21 (72.4) 23 (79.3) 43 (71.7) 87 (73.7) Non-student 8 (27.6) 6 (20.7) 17 (28.3) 31 (26.3) History of mental health or substance use diagnosis Yes 23 (79.3) 25 (86.2) 48 (80.0) 96 (81.4) No 6 (20.7) 4 (13.8) 12 (20.0) 22 (18.6) History of alcohol use diagnosis Yes 17 (58.6) 24 (82.8) 44 (73.3) 85 (72.0) No 12 (41.4) 5 (17.2) 16 (26.7) 33 (28.0) AUDIT Total Score ( M , SD , range) = 13.02, 4.70, 5–30 score Age ( M , SD , range) = 21.14, 1.76, 18–25 years Note . “CIAN” stands for Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice. “A + SM” stands for advice plus self-monitoring control condition, which also includes passive monitoring. “A” stands for advice control condition, which also includes passive monitoring. “AUDIT” stands for Alcohol Use Disorders Identification Test 33 . a There were no significant differences between trial groups on any demographic variable. Thematic Analysis of Exit Interviews Qualitative thematic analysis of exit interviews resulted in overarching thematic categories related to helpfulness of different intervention components and suggestions from participants (see Table 2 for themes and salience; see Supplemental Materials for all thematic analysis definitions and exemplar quotes). Table 2 CIAN Thematic Analysis Salience (N = 107) Thematic Category Theme n (%) a Helpful aspects of website General helpfulness of website 69 ( 64.5% of 107) Helpful sleep-related information 69 ( 64.5% of 107) New information from website 58 (54.2% of 107) Reminders of known information 42 (39.3% of 107) Usefulness of website strategies 38 (35.5% of 107) Website understandability 28 (26.2% of 107) Helpful alcohol-related information 25 (23.4% of 107) User-friendliness of website 25 (23.4% of 107) Memorability of information 9 (8.4% of 107) Unhelpful aspects of website Lack of new information 14 ( 13.1% of 107) Irrelevance of information 5 (4.7% of 107) Difficulty implementing strategies 5 (4.7% of 107) Helpful aspects of biosensors Ankle biosensor increased awareness 36 ( 33.6% of 107) General helpfulness of biosensor(s) 35 (32.7% of 107) Watch biosensor increased awareness 33 (30.8% of 107) Neutral aspect of biosensors Neutral aspect of biosensors 65 ( 60.7% of 107) Unhelpful aspects of biosensors Burdensomeness of ankle biosensor 22 ( 20.6% of 107) Unhelpful without data feedback 16 (15.0% of 107) Lack of behavior change 12 (11.2% of 107) Burdensomeness of watch biosensor 8 (7.5% of 107) Stigma of ankle biosensor 6 (5.6% of 107) Ankle biosensor increased drinking 5 (4.7% of 107) Helpful aspects of diaries Increased mindfulness of sleep 42 ( 53.8% of 78) General helpfulness of diaries 38 (48.7% of 78) Increased mindfulness of alcohol use 31 (39.7% of 78) New experience of keeping a diary 22 (28.2% of 78) Ease of answering questions 17 (21.8% of 78) Motivation to change behaviors 10 (12.8% of 78) Neutral aspect of diaries Neutral aspect of diaries 13 ( 16.7% of 78) Unhelpful aspects of diaries Challenges answering questions 13 ( 16.7% of 78) Lack of behavior change 6 (7.7% of 78) Lack of new information 6 (7.7% of 78) Helpful aspects of feedback Helpfulness of personalization 45 ( 90.0% of 50) General helpfulness of feedback 29 (58.0% of 50) Helpful data presentations 27 (54.0% of 50) Unhelpful aspects of feedback Lack of new information 4 ( 8.0% of 50) Most influential interventions Feedback was most influential 36 ( 72.0% of 50) Website was most influential 39 (36.4% of 107) Diary was most influential 25 (32.1% of 78) Biosensors were most influential 17 (15.9% of 107) Multiple things were helpful 15 (14.0% of 107) Suggestions for interventions Willing to wear more devices (watch) 65 ( 60.7% of 107) Want diet and exercise-related feedback 63 (58.9% of 107) Willing to complete new diary questions 60 (56.1% of 107) Want caffeine-related feedback 53 (49.5% of 107) Want other potential feedback 33 (30.8% of 107) Want environment-related feedback 26 (24.3% of 107) Unwilling to add question and/or device 18 (16.8% of 107) Improvements for feedback 5 (10.0% of 50) Improvements for diaries 6 (7.7% of 78) Improvements for biosensors 8 (7.5% of 107) Improvements for website 4 (3.7% of 107) Note. Theme salience (% of those who stated the theme) is based on the number of participants who answered the interview question that resulted in the theme. General questions and questions about the website and biosensors were asked of all 107 participants across trial groups. Questions about diaries were asked of the 78 CIAN and A + SM group participants who completed the diaries and the exit interviews. Questions about feedback were asked of the 50 CIAN participants who received feedback and completed the exit interview. Web-based Sleep Advice (Website): Helpful and Unhelpful Aspects (n = 107) Participants highlighted various aspects of the web-based advice intervention, both helpful and unhelpful. The most mentioned helpful aspects were General helpfulness of the website and Helpful sleep-related information : “I think the particularly helpful ones [tips] were…things to do when you wake up, because I think …when I wake up, I just don't want to get out of bed, so I’ll…stay in bed on my phone for a long time.” Participants appreciated learning New information from the website, Reminders of known information , and Usefulness of website strategies : “It [the website] talked about…your body temperature…keeping a window open or having a fan to keep your body [cool].” Others noted helpful aspects of Website understandability, User-friendliness of website, Helpful alcohol-related information , and Memorability of information . By contrast, relatively few participants described unhelpful aspects. Some mentioned Lack of new information as a drawback, while others described Irrelevance of information , such as content related to cannabis and sleep. A few participants reported Difficulty implementing strategies recommended on the website. Passive Monitoring (Biosensors): Helpful, Neutral, and Unhelpful Aspects (n = 107) Participants across trial groups provided comments on whether wearing the biosensors (transdermal alcohol ankle monitor and sleep watch) was helpful on its own without consideration of the data collected. Most described Neutral aspects of biosensors and did not find them helpful or unhelpful without data. Among those who found the biosensors helpful on their own, the most mentioned aspect was that the Ankle biosensor increased awareness of their alcohol use: “The physical monitors all over you…they definitely make you…think like, ‘Wow…should I really be drinking this much?’” Likewise, others noted the General helpfulness of the biosensor(s) or that the Watch biosensor increased awareness of their sleep patterns irrespective of data obtained: “If I saw my sleep watch before I was going to bed, I was like, ‘You know what, I should probably try and go to bed early tonight.’” Among those who found one or both biosensors unhelpful, the most common complaint was Burdensomeness of ankle biosensor , including itchiness, discomfort, and difficulty completing some activities: “I just really didn't like the [ankle] device…couldn't run as much.” Other unhelpful aspects mentioned were Unhelpful without data feedback , Lack of behavior change, Burdensomeness of watch biosensor (e.g., forgetting to push buttons), Stigma of ankle biosensor (e.g., connection to court-ordered monitors), and Ankle biosensor increased drinking. Active Self-Monitoring (Diaries): Helpful, Unhelpful, and Neutral Aspects (n = 78) CIAN and A + SM participants commented on the helpfulness of completing diaries, a component of their study conditions. Most described one or more helpful aspects, most commonly that diaries Increased mindfulness of sleep : “I found it…helpful to…realize it [my sleep schedule] and…just reflect on…my lifestyle and…how much sleep I'm actually getting.” Others noted the General helpfulness of diaries and Increased mindfulness of alcohol use : “I…never really…used to like keeping track [of drinking]. So, I'd…drink a lot, and then I'd feel too drunk.” Some noted the helpfulness of the New experience of keeping a diary , the Ease of answering questions in the diaries, or the Motivation to change behaviors prompted by diaries. Fewer participants noted unhelpful aspects of diaries, such as Challenges answering questions in the diaries, Lack of behavior change , and Lack of new information (i.e., self-monitoring did not increase awareness of patterns nor motivate them to change behaviors. Some also noted a Neutral aspect of diaries , they were neither helpful nor unhelpful. Personalized, Tailored Coaching (Feedback): Helpful and Unhelpful Aspects (n = 50) Participants in the CIAN condition with personalized feedback and coaching largely described helpful aspects of the feedback and coaching intervention component. Almost all CIAN group participants described the Helpfulness of personalization , the individualized nature of reports and suggestions based on monitoring data: “Having that sort of feedback, both the data and just the explanation behind it, I think that's a really…unique insight into…a part of yourself that…most people wouldn't ever really get to actually see mapped out.” Most also noted aspects, such as the General helpfulness of feedback and Helpful data presentations : “She [the coach] made it very easy for me to understand the charts and what everything meant.” Only a few described one unhelpful aspect of feedback: Lack of new information , i.e., reports did not contribute new insights into their sleep or drinking. Most Influential Interventions Each participant was asked to select which received intervention component(s) were most influential for behavior change. In order from most to least helpful, participants selected personalized feedback/tailored coaching (i.e., three-fourths of CIAN participants), the website (i.e., over one-third of all participants), diaries (i.e., almost one-third of CIAN and A + SM participants), and biosensors (i.e., less than one-fifth of all participants). Suggestions for Future Interventions During exit interviews, participants also offered suggestions for current and future intervention improvement. Many participants were interested in measuring additional sleep-related factors, most commonly that they Want diet and exercise-related feedback : “[I’d like to know] what time when you exercise, how it affects your sleep…what that does to…your whole body chemistry, everything.” Others mentioned they Want caffeine-related feedback , Want environment-related feedback (e.g., impacts of light, noise, and temperature on sleep), or Want other potential feedback (e.g., cognitive alertness, tobacco, and stress): “Maybe you can see how stressed you are during… the week. Just get those numbers, and then I think that would be pretty interesting to figure [out].” To receive additional feedback, most responded that they would be Willing to wear more devices and Willing to complete new diary questions . Fewer stated they were Unwilling to add a question and/or device , usually disinterest in another ankle biosensor. Several volunteered suggestions for received interventions in the current study. These included Improvements for feedback (e.g., combined, summary report for both intervention weeks, Improvements for diaries (e.g., higher pay per diary), Improvements for biosensors (e.g., eliminating ankle biosensors, measuring GPS location), and Improvements for website (e.g., wider distribution, alternative summaries). Nonmetric Multidimensional Scaling of Interview Themes We used multivariate analysis, nonmetric multidimensional scaling (NMDS) 24 , to visualize the interrelationships among qualitative themes and participants. NMDS is an ordination method that condenses variation in matrices, such as participant-by-theme frequency, to a small number of orthogonal dimensions 25 . We used five orthogonal dimensions based on stress-level testing, and the first two dimensions are plotted in vector space in Fig. 2 . Also, multivariate correlational analysis allowed us to fit participant characteristics as factors to the NMDS ordination of themes. Trial group ( R 2 = .29, p = .001) and lifetime history of any MHD/SUD ( R 2 = .03, p = .04) were significantly correlated with NMDS scores. That is, participants’ statements about study interventions in their exit interviews were associated with trial group and diagnostic history. Other participant characteristics (e.g., age, gender, race, ethnicity, student status) were not associated with NMDS scores. In the NMDS shown in Fig. 3 , CIAN and A + SM participants were closer to themes of feedback and diary helpfulness than website or biosensor helpfulness in vector space, which suggests their preference for feedback and diaries. Conversely, control A and A + SM participants were closer to themes of helpful website aspects in vector space than CIAN participants, and control participants were also significantly more likely to state these themes during interviews ( X 2 = 27.34, p < .001). Natural Language Processing of Exit Interviews ( n = 112) We also used quantitative Latent Dirichlet Allocation (LDA), a topic modeling analysis that identifies the likelihood of terms occurring in topics and topics occurring in documents 26 , with an expanded dataset of 112 participants’ exit interviews and coaching transcripts and identified nine topics across participants’ statements (see Fig. 3 ; see Supplemental Material for all topic definitions and example quotes). The goals of NLP analyses were to help qualitative thematic analysis be more targeted and assess convergence of findings with thematic analysis and exit survey analyses. These topics did not vary significantly by trial condition ( X 2 = 16.72, p = .40). We found that Awareness with monitoring was the most likely topic in the largest number of interviews ( n = 18, 16%). This included participants’ perceptions that both active diary self-monitoring and passive biosensor monitoring increased their awareness and mindfulness of their behaviors. As one participant stated, “Knowing that I had to do the sleep diary the next day and…people were watching what I was doing…I was just more aware of my habits.” Other common topics focused on web-based advice, especially sleep strategies. The topic, Website improves sleep ( n = 16, 14%), included experiences of beginning to improve sleep using sleep strategies, such as consistent sleep schedules. A participant described, “My sleeping habits were definitely awful before now, and I think they've improved a lot, since starting this [study]…I'm starting to go to bed earlier and…reduce my activities before bed.” Almost as many interviews were most likely to include the topic, Strategy barriers ( n = 15, 13%), or descriptions of challenges implementing sleep strategies, such as situational factors (e.g., work and school schedules, dormitory environment) or personal factors (e.g., memory, motivation). The topic Changing poor sleep ( n = 15, 13%), included descriptions of poor sleep quality or nonrestorative sleep, how this motivated study participation, and a desire to learn about different sleep factors. Other topics focused on participants’ interest in gaining more wellness-related knowledge to change sleep or alcohol use. Thirteen interviews (12%) were most likely to include the topic Learning and reduced drinking , which focused on gaining new information from digital interventions and reducing drinking given its impacts on sleep. The topic Using feedback and website ( n = 11, 10%) included participants’ accounts of how they integrated personalized feedback and coaching with web-based advice, including information and strategies, to alter their sleep and alcohol use. The topic Multiple strategies and factors ( n = 10, 9%) focused on participants’ curiosity and interest in the impacts of varied sleep-related factors, such as situations, environments, substances other than alcohol, and diet and exercise. The two least frequent topics included the burden of wearable biosensor devices. The topic Strategies, not devices ( n = 8, 7%) focused on helpful website aspects, attempts to implement sleep strategies, and challenges with wearable devices, especially the transdermal ankle monitor. Similarly, the topic Feedback, not devices ( n = 6, 5%) focused on the benefits of personalized feedback and coaching while also discussing difficulties with wearable devices. The sentiment scores of exit interviews and transcripts ( n = 112) were generally positive ( M = 15.07, SD = 10.54) and ranged from − 16 to 47 (see Fig. 4 ). Positive vs. negative sentiment scores reflect the aggregate valence of participants’ word usage in exit interviews, so “0” reflects a neutral net score in an interview. Sentiment scores did not vary as a function of trial condition ( CIAN , A + SM , and A ; F (2, 109) = 0.78, p = .46). Positive sentiment scores also did not vary by demographic characteristics, including gender ( F (1, 110) = 0.62, p = .43), age ( b = 0.03, t = 0.06, p = .95), race ( F (4, 107) = 1.23, p = .30), ethnicity ( F (1, 110) = 0.20, p = .66), student status ( F (3, 108) = 0.75, p = .53), and history of any MHD/SUD ( F (1, 110) = 0.23, p = .63). However, the word length of each document was positively correlated with sentiment scores ( r = 0.21, p = .03), indicating a tendency for participants who spoke at longer length to speak more positively about their experiences. To boost rigor and account for this correlation, we derived a valence-per-word score for each document and reran sentiment comparisons across condition and demographic groups with the valence-per-word score as the outcome. There were still no between-group differences ( p > 0.05). Exit Survey ( n = 118) On the exit survey, participants across conditions ( CIAN , A + SM , and A ) reported generally positive user experiences ( n = 118; see Table 3 ). On a 5-point scale, participants reported high overall program satisfaction, website helpfulness, diary helpfulness, and feedback helpfulness. Mean feasibility ratings were generally high for the overall program, website, watch biosensor, and diaries. However, mean feasibility ratings for the ankle biosensor were lower based on visual inspection, especially due to interference with clothing, feeling uncomfortable, and being noticeable. Perceived effectiveness was higher among CIAN (Δ = 0.48, p = .008) and A + SM participants (Δ = 0.55, p < .001) than A participants ( F (2, 115) = 8.45, p < .001). Further, young adults with a lifetime history of any MHD/SUD rated their intervention as more effective on average than those without diagnoses ( F (1, 116) = 4.64, p < .001; Δ = 0.32, p = .03). Table 3 CIAN Exit Survey Outcomes (N = 118) Outcome (Agreement 1–5) A A + SM CIAN Total M ( SD ) M ( SD ) M ( SD ) M ( SD ) General program outcomes Overall satisfaction 4.45 (0.74) 4.48 (0.57) 4.60 (0.56) 4.53 (0.61) Understandability 4.52 (0.87) 4.59 (0.83) 4.65 (0.66) 4.60 (0.75) Schedule workability 4.66 (0.94) 4.62 (0.86) 4.67 (0.82) 4.65 (0.85) Appropriate visit length 4.31 (1.11) 4.10 (1.14) 4.27 (0.92) 4.24 (1.02) Lifestyle change promotion 4.03 (0.87) 4.10 (0.77) 4.23 (0.96) 4.15 (0.89) Comfortability 4.76 (0.79) 4.66 (0.81) 4.57 (0.81) 4.64 (0.80) Hope for change promotion 4.00 (0.89) 3.97 (0.91) 4.10 (1.02) 4.04 (0.96) Importance of target habits 4.48 (0.87) 4.52 (0.69) 4.57 (0.81) 4.53 (0.79) Overall effectiveness 4.00 (0.76) 4.48 (0.51) 4.55 (0.57) 4.40 (0.64) Advice (A) outcomes Module 1 helpfulness 4.24 (0.74) 4.24 (0.64) 4.43 (0.65) 4.34 (0.67) Module 2 helpfulness 4.07 (0.96) 4.31 (0.66) 4.45 (0.65) 4.32 (0.75) User-friendliness 4.52 (0.51) 4.41 (0.50) 4.40 (0.94) 4.43 (0.76) Comparability to other websites 4.76 (0.44) 4.52 (0.57) 4.73 (0.58) 4.68 (0.55) Enjoyment 3.97 (0.91) 3.97 (0.73) 4.00 (0.77) 3.98 (0.79) Passive-monitoring (biosensor) outcomes Actiwatch comfortable 3.93 (1.13) 4.14 (0.69) 3.92 (0.94) 3.97 (0.94) Actiwatch not embarrassing 4.00 (1.16) 4.24 (0.64) 4.23 (0.79) 4.18 (0.86) Actiwatch did not interfere with work 4.34 (0.97) 4.55 (0.51) 4.52 (0.62) 4.48 (0.70) Actiwatch did not interfere with exercise 4.41 (0.78) 4.31 (0.66) 4.32 (0.91) 4.34 (0.82) Actiwatch did not interfere with sleep 4.41 (0.68) 4.34 (0.67) 4.30 (0.89) 4.34 (0.79) Actiwatch did not interfere with concentration 4.59 (0.57) 4.55 (0.57) 4.52 (0.60) 4.54 (0.58) Actiwatch did not interfere with clothing 4.41 (0.87) 4.21 (0.98) 4.18 (1.02) 4.25 (0.97) Actiwatch not noticeable 4.07 (1.00) 4.17 (0.93) 4.17 (0.96) 4.14 (0.95) Actiwatch did not change routine 4.59 (0.57) 4.38 (0.82) 4.28 (1.01) 4.38 (0.88) Actiwatch wear more 4.24 (0.87) 4.34 (0.81) 4.27 (0.92) 4.28 (0.88) Ankle monitor comfortable 2.15 (0.95) 2.66 (1.11) 2.60 (1.08) 2.51 (1.07) Ankle monitor not embarrassing 2.59 (1.19) 2.66 (1.42) 2.98 (1.38) 2.81 (1.35) Ankle monitor did not interfere with work 3.37 (1.08) 3.79 (1.05) 4.05 (1.00) 3.83 (1.06) Ankle monitor did not interfere with exercise 2.48 (1.16) 3.03 (1.30) 3.02 (1.32) 2.90 (1.29) Ankle monitor did not interfere with sleep 3.04 (1.19) 3.34 (1.14) 3.25 (1.16) 3.22 (1.16) Ankle mon. did not Interfere with concentration 3.56 (1.25) 3.97 (0.91) 4.05 (1.02) 3.91 (1.06) Ankle monitor did not interfere with clothing 2.33 (1.47) 2.17 (1.28) 2.55 (1.43) 2.41 (1.40) Ankle monitor not noticeable 2.85 (1.22) 2.66 (1.37) 2.60 (1.24) 2.67 (1.26) Ankle monitor did not change routine 3.11 (1.25) 3.21 (1.18) 3.10 (1.36) 3.12 (1.28) Ankle monitor wear more 3.26 (1.53) 2.93 (1.25) 3.42 (1.37) 3.26 (1.38) Self-monitoring (SM) outcomes Diary helpfulness - 4.07 (0.72) 4.19 (0.79) 4.15 (0.76) Diary easiness - 4.34 (0.61) 4.58 (0.56) 4.51 (0.59) Diary not burdensome - 3.97 (1.09) 4.17 (0.91) 4.10 (0.97) Diary did not change schedule - 4.24 (0.83) 4.27 (1.06) 4.26 (0.98) Diary easy to remember - 3.72 (1.16) 3.92 (1.00) 3.85 (1.05) Diary enjoyable - 3.52 (0.91) 3.83 (0.91) 3.73 (0.91) Diary would continue - 3.52 (1.15) 3.83 (0.92) 3.73 (1.01) Feedback (F) outcomes Sleep feedback helpfulness - - 4.70 (0.53) 4.70 (0.53) Alcohol feedback helpfulness - - 4.65 (0.55) 4.65 (0.55) Note . “CIAN” stands for Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice. “A + SM” stands for advice plus self-monitoring control condition, which also includes passive monitoring. “A” stands for advice control condition, which also includes passive monitoring. Total means and standard deviations are based on 118 participants who completed the exit interview, including 60 CIAN, 29 A + SM, and 29 A participants. Some items had minor levels of missingness (< 5 participants). Adherence to interventions was also high across study phases. Almost all participants (98%) completed the two-week intervention phase, and 96% completed the 12-week follow-up appointment. Regarding monitoring activities, 98% of participants wore the sleep watch biosensor for 14 days, and 95% wore the alcohol ankle biosensor for 14 days. Further, 95% of participants in the CIAN and A + SM groups completed their assigned 14 days of active diary self-monitoring of sleep and alcohol use. Discussion The current study used an innovative hybrid approach in which NLP findings converged with and corroborated more traditional mixed methods to efficiently evaluate user experiences with a digital sleep intervention to reduce drinking. Major findings showed that participants across conditions generally found all intervention components helpful, satisfying, and feasible, but the CIAN group markedly preferred the personalized feedback and tailored coaching with a health professional they received to other components. Convergent results also underscored young adults’ strong interest in gaining a more holistic picture of their wellness in the future and increasing their awareness using monitoring through diaries and biosensors. Our hybrid approach using NLP enabled qualitative thematic analysis to target specific interview questions more efficiently, optimizing researcher’s time. Further, assessing the convergence of findings between NLP and other methods helped address potential researcher bias in thematic analysis and potential user response bias in exit surveys. The Importance of Personalized and Interactive Feedback and Coaching Mixed methods findings on user experiences, including NLP, emphasized greater effectiveness and satisfaction with personalized feedback and coaching than other intervention components despite generally favorable experiences with web-based advice, active diary self-monitoring, and passive biosensor monitoring. CIAN and A + SM group participants who received feedback and/or actively self-monitored (diaries) rated their intervention as more effective than A control participants in the exit survey. Topic modeling (LDA) and thematic analysis showed that some participants found web-based advice strategies difficult to implement, and negative themes about web-based advice were associated more with CIAN participants. Our study adds insight to extant literature on young adults’ preferences for interactive digital interventions, including alcohol and sleep treatment. An umbrella overview of meta-analyses and systematic reviews 21 found that digital interventions offering interaction with a health professional or other social support had higher adherence and effectiveness and lower attrition. Mobile alcohol treatment apps providing personalized feedback may be especially engaging 17 . Further, recent research on mobile sleep apps 16 , 18 – 19 highlights increased engagement and satisfaction with personalized feedback and communication with health professionals. Our study adds findings that individually tailored coaching from a health professional significantly enhanced participants’ experience interpreting digital health data, which likely supported their understanding. Current research also shows that user-friendliness 17 and evidence-based information 18 increase engagement, which was consistent with our finding that web-based advice was considered helpful and user-friendly, especially by control groups. Therefore, whereas web-based advice and other automated interventions are likely to be helpful to many young adults, tailored feedback and coaching with individualized data are likely to be preferable in the current digital health market. Empowering Young Adults with Digital Health Tools for General Wellness Consistent with prior findings 11 , our results showed that young adults particularly want to use digital tools to pursue holistic health and wellness goals. Thematic analysis and LDA with interviews converged to show that participants are interested in understanding varied other lifestyle factors that impact sleep beyond alcohol (e.g., caffeine, diet, exercise, stress, environment). Most participants expressed a willingness to use additional digital tools (e.g., more biosensors, additional diary self-monitoring) to attain whole-health, personalized feedback. LDA results highlighted greater perceived awareness and mindfulness of sleep and drinking through diary self-monitoring and/or wearing biosensors. Thus, while young adults are interested in and capable of using varied digital health tools, incorporating appealing treatment foci could be important to increase young adults’ generally low engagement and adherence 15 . Our study suggests that digital interventions improving overall health may represent a more appealing treatment approach to young adults for potentially stigmatizing diagnoses, such as AUDs. Most available digital SUD interventions for young adults involve web-based advice explicitly targeting alcohol use to the exclusion of other substance use or wellness factors 27 . Whereas young adults may not be concerned about alcohol use or seek traditional alcohol treatment 5 , they are concerned about improving sleep quality 6 , 8 and other aspects of their general wellness. In this study, LDA with exit interviews showed that participants commonly joined the study to improve their sleep quality rather than address their alcohol use. Further, survey and interview thematic analysis results, respectively, showed that participants with a lifetime history of an MHD/SUD found the current intervention to be especially effective and described intervention helpfulness differently than those with no diagnoses. Therefore, digital interventions, like the current study which focused more explicitly on improving wellness (e.g., enhancing sleep quality), may be especially engaging to young adults with AUDs or other diagnoses who have not presented to treatment. Implications and Recommendations Our study has implications for clinical researchers and mobile app designers making user-centered app designs. User experience evaluations of digital health tools should incorporate a hybrid, convergent approach to optimize depth, breadth, and efficiency. Each user experience evaluation method has distinct strengths and potential weaknesses 23 . Thematic analysis of exit interviews can provide detailed information across complex subjective experiences. In the current study, along with post hoc analyses, thematic analysis revealed more distinctions between perceived helpfulness of different intervention components than any other method. However, as sample sizes increase, qualitative thematic analysis can become impractical and unfeasible. Our use of NLP topic modeling (LDA) to derive overarching topics across interview questions enabled us to target and abbreviate our thematic analysis to interview questions focusing on intervention component helpfulness. Exit surveys are also time-efficient like NLP but may be subject to user response bias. NLP, specifically sentiment analysis, helped confirm the generally positive perceptions derived from exit survey analyses. Our convergent evaluation findings also have specific implications for digital intervention design. Young adults are enthusiastic about receiving a broad, holistic picture of factors impacting their wellness, including different aspects of their lifestyle, in addition to appreciating individualized and specific coaching. Therefore, mobile mental health apps and linked biosensors should have a variety of choices for active and passive monitoring that can be tailored to user feasibility and interest. Digital tools should integrate seamlessly into young adults’ schedules that often include exercise, which may make ankle transdermal monitors challenging and potentially stigmatizing 28 compared to watch biosensors. Also, digital interventions, including for sleep and alcohol use, should empower young adults to actively explore their wellness with support from a health professional. While participants in the current study valued the mindfulness derived from digital health monitoring, they preferred to also receive personalized data reports with explanations and suggested health tips from a coach. Some individuals may lack the expertise to interpret their health data or to devise action plans for improvement. Thus, digital interventions should include mechanisms for interactive feedback, such as an on-call health provider, pre-scheduled video or text check-ins with a health provider, or a chatbot that provides a spectrum of suggestions and individual data descriptions. Digital interventions for specific, sensitive clinical issues, like heavy drinking, could engage more young adults via a focus on related general health and wellness behaviors, like sleep. Interventions that increase accessibility and engagement for young adults are important given that only half of young adults with psychiatric disorders currently receive any mental health treatment 2 . Digital sleep interventions for reduced drinking can target two risky, prevalent, and modifiable issues: drinking and sleep problems. Young adults commonly experience AUDs, binge drinking 2 , and sleep problems 7 . Further, sleep problems in adolescence lead to heavier drinking and AUDs in adulthood 29 – 31 . Notably, young adults who drink heavily may be less concerned about their drinking than their sleep 8 . In general, digital tools that explicitly focus on less sensitive behavioral goals, like improved sleep, with implications for improving potentially stigmatized goals, like reduced drinking, are likely to be more appealing and engaging to young adults. Strengths and Limitations Distinct strengths of the current study included a relatively large sample size, especially for a qualitative analysis of exit interviews; generally high adherence across intervention phases and groups; and minor amounts of missing data. Each of these attributes increased the validity of findings. Further, our hybrid, convergent approach incorporating NLP represents an important strength as these methods are promising and relatively new in digital medicine 22 – 23 . Our research also had notable limitations. Although missing data were minor, it is possible that additional themes would have emerged in a complete set of exit interviews. NLP helped address response bias in exit survey items, but survey responses were not normally distributed and generally skewed towards higher responses. Our sample was primarily college students, so it may not be representative of young adults in general. Further, study participants were paid, so implementation and uptake may differ in clinical or other naturalistic settings. Conclusion The current evaluation demonstrates the value of NLP in convergent mixed methods to efficiently capture broad and nuanced user experiences with digital health interventions. Specifically, our results show that digital sleep interventions for heavy drinking may increase appeal and access to alcohol treatment for young adults, especially when they include tailored coaching. Broadly, the current findings emphasize the importance of digital tools for young adults that provide a holistic, dynamic view of health coupled with interactive, individualized feedback. Consistent and precise evaluations that leverage user feedback are critical to support higher uptake and adherence to effective digital health interventions, which can begin to address the mental health treatment demand gap for young adults. Methods Study design In the current study, we evaluated user experiences during a randomized clinical trial of CIAN , a novel, two-session personalized feedback and coaching digital sleep intervention developed for young adults to reduce drinking ( N = 120). Participants were randomly assigned to one of three conditions using a 2:1:1 ratio for two weeks: 1) CIAN : two-session “Call it a Night” intervention ( n = 60) or 2) one of two control conditions, A : brief advice only ( n = 30) or A + SM : brief advice + active sleep diary self-monitoring ( n = 30). For this outpatient study in participants’ natural environments, all wore sleep and alcohol biosensors daily for two weeks and viewed a two-module web-based sleep advice program during brief, midweek research appointments. There were no significant differences between the trial groups on any demographic variable (i.e., age, gender, race, ethnicity, student status, lifetime history of psychiatric diagnosis, or baseline alcohol use). Of the participants, 118 completed the exit survey and 107 completed the exit interview with study staff. To evaluate user experiences, we used convergent mixed methods with NLP following the two-week intervention phase. Exit surveys and interviews were conducted in parallel with participants across trial conditions to attain both broad and deep perspectives on satisfaction and intervention acceptability and feasibility. Both qualitative thematic analysis and quantitative NLP examined exit interviews. Further, in line with convergent mixed methodology, exit survey results were analyzed using descriptive and predictive analyses to compare these findings with exit interview results. This study was approved by the Institutional Review Board of Yale University (IRB# 2000021048). Additional study details can be found in the published study protocol paper 6 . Recruitment To be eligible for the CIAN clinical trial, participants needed to a) be 18–25 years of age, b) be fluent in English, c) self-report three or more heavy drinking occasions in the last two weeks (i.e., 5 or more drinks on one occasion for men or 4 or more for women), d) score at risk of harm from drinking on the AUDIT-C 32 – 33 , and d) self-report sleep concerns. Exclusion criteria are described further in the study protocol paper 6 . Participants were recruited using online (e.g., Facebook, Instagram, Snapchat) and in-person advertisements placed around the local community. Online advertisements and flyers directed individuals to a web-based pre-screening survey, and those who met pre-screening eligibility were invited to an intake visit for final eligibility determination. Exit Survey and Interview Upon completion of the two-week intervention phase, we asked participants to complete a self-report exit survey (see Supplemental Material for exit survey items). The exit survey included Likert-type and yes/no questions to assess user experiences, including overall satisfaction, intervention helpfulness and effectiveness, and intervention feasibility. Participants were also asked to detail areas for improvement in open-ended response items. Participants across trial groups also completed semi-structured exit interviews to provide information about their subjective user experiences (see Supplemental Material for exit interview protocol). Exit interviews were administered by three research staff members (two men, one woman), who were trained in the protocol and familiar with study interventions. The exit interview protocol included questions about the following topics: participants’ sleep and alcohol use before and after the study, perceptions of these behaviors relative to their peers, use of sleep hygiene strategies from the intervention, helpfulness of intervention components, the impact of payment on study participation, what drew them to the study, and suggestions for intervention improvement or future studies targeting sleep or alcohol use. Data Analysis We conducted both qualitative thematic analysis and quantitative NLP with exit interviews. For thematic analysis, we used Braun and Clarke’s six-step process 34 , during which we used a recursive and iterative team coding process to ensure rigor and trustworthiness in the process of identifying, naming, and categorizing recurrent themes. Two members of the research team completed an initial open coding of the entire text of a large subset of exit interviews ( n = 80) to derive initial themes. Then, guided by findings from NLP topic modeling analysis, the current thematic analysis targeted key portions of the exit interviews, including questions related to helpfulness of intervention components and participants’ suggestions. Three research team members engaged in a rigorous team coding process of the entire dataset of completed two-week exit interviews ( n = 107). Using an initial codebook based on previous first-round, open thematic coding, these three researchers independently coded a random selection of 20% of the entire interview dataset ( n = 21) to evaluate interrater reliability. All themes had percent agreement of 95% or greater, but an extensive reconciliation meeting was still undertaken to audit the initial codebook and resolve any disagreements until an overall kappa exceeding 0.7 was reached between each pair of coders. The remaining exit interviews were divided among the three researchers to code using the revised codebook. Auditing was used throughout the qualitative process to ensure transparency, including field notes maintained by each coder and reviewed by other coders and ongoing check-ins with the entire research team. To visualize the interrelationship between participants and themes, we conducted NMDS with theme frequency counts using the vegan package 24 in the programming language R 35 . NMDS is a multivariate ordination method that condenses variation in matrices to a small number of orthogonal dimensions 25 . We selected NMDS for post-hoc analysis of themes because it does not place assumptions of normality on frequency count data compared to other multidimensional scaling methods 25 . To assess the goodness of fit when selecting the number of orthogonal dimensions, we used stress level cutoffs and a stress plot (Shepard diagram) 24 . We selected five orthogonal dimensions, resulting in a good to fair stress score of .13, which is likely to result in minimal distortion 36 . As shown in Fig. 2 , we used a multivariate correlation analysis with vegan in R to fit factors (trial condition and participant gender, race, ethnicity, student status, and diagnosis history) and a vector (participant age) to the NMDS ordination of participants and themes. This enabled us to assess whether participants’ user experiences (as interview themes) varied significantly with their assigned trial condition or personal characteristics. For NLP of exit interviews, we conducted topic modeling 26 and sentiment analyses 37 with an expanded dataset of all of participants’ statements during completed exit interviews and feedback session transcripts ( n = 112). LDA with the topicmodels package 26 in R was used for topic modeling analysis. To boost rigor and ensure the most parsimonious number of topics ( k ) were used for LDA, we undertook a preliminary analysis with three model selection methods to test different k values 38 – 40 in the ldatuning package 41 in R and ultimately selected nine topics. Text in participants’ statements was preprocessed before NLP, including removing punctuation, numbers, capitalization, and English ‘stopwords’ as well as stemming all remaining words. The LDA determined which of the nine topics was most likely to occur in each document and which terms were most likely to occur in each topic. Topics were systematically named by closely reading five or more interview transcripts determined most likely to contain the topic by LDA. Chi-squared tests determined whether topics varied with participants’ assigned trial condition. For sentiment analysis, we used the AFINN lexicon 42 and the syuzhet package 37 in R. The AFINN lexicon is a well-established and commonly used sentiment lexicon 37 that assigns a valence score from − 5 to 5 to selected terms, and documents are scored based on an aggregate of their terms’ scores. An aggregate AFINN sentiment score of “0” for a document indicates neutral term valence. Bivariate regression models, including analysis of variance (ANOVA), were used to assess whether participants’ trial condition or personal characteristics predicted the sentiment of their statements during exit interviews. Descriptive and predictive analyses were conducted with exit survey results. These included summary statistics of each item and linear modeling to determine if participants’ trial condition or personal characteristics predicted responses to exit survey items. Declarations Acknowledgments: This research was supported by the National Institute of Health and the National Institute on Alcohol Abuse and Alcoholism under Grants # R34AA026021, R34AA026021-S1, and R01AA03013 and the National Institute on Drug Abuse under Grant # T32DA019426. Further, we would like to thank Daniel Griffith for his consultation on statistical analyses conducted in R. Competing interests statement: All authors declare no financial or non-financial competing interests. Data availability statement: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request. Code availability statement: The underlying code for this study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author. Author contributions statement: FJG: Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review & editing GIA: Conceptualization, Methodology, Investigation, Project administration, Writing—review & editing MA: Data curation, Formal analysis, Validation, Writing—review & editing LL: Formal analysis, Validation, Writing—review & editing NV: Formal analysis NSR: Conceptualization, Methodology, Writing—review & editing SSO: Conceptualization, Methodology, Writing—review & editing KSD: Conceptualization, Methodology, Supervision, Writing—review & editing LMF: Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Validation, Writing—review & editing References Solmi M et al. Age at onset of mental disorders worldwide: Large-scale meta-analysis of 192 epidemiological studies. Molecular Psychiatry. 2022;27(1):281–295. https://doi.org/10.1038/s41380-021-01161-7 SAMHSA, Center for Behavioral Health Statistics and Quality. Results from the 2022 National Survey on Drug Use and Health. 2023. Retrieved from https://www.samhsa.gov/data/report/2022-nsduh-detailed-tables Moreno C et al. How mental health care should change as a consequence of the COVID-19 pandemic. The Lancet Psychiatry. 2020;7(9):813–824. https://doi.org/10.1016/S2215-0366(20)30307-2 Lee MR, Sher KJ. “Maturing out” of binge and problem drinking. Alcohol Research. 2018;39:31–42. Singleton Jr. RA, Wolfson AR. Alcohol consumption, sleep, and academic performance among college students. Journal of Studies on Alcohol and Drugs. 2009;70(3):355–63. http://doi.org/10.15288/jsad.2009.70.355 Fucito LM et al. A multimodal mobile sleep intervention for young adults engaged in risky drinking: protocol for a randomized controlled trial. JMIR Research Protocols. 2021;10(2):e26557. http://doi.org/10.2196/26557 Sivertsen et al. Sleep patterns and insomnia in young adults: A national survey of Norwegian university students. Journal of Sleep Research. 2019;28(2):e12790. http://doi.org/10.1111/jsr.12790 Ash GI et al. Using web-based social media to recruit heavy-drinking young adults for sleep intervention: Prospective observational study. JMIR. 2020;22(8):e17449. http://doi.org/10.2196/17449 Brooks AT, Wallen GR. Sleep disturbances in individuals with alcohol-related disorders: A review of cognitive-behavioral therapy for insomnia (CBT-I) and associated non-pharmacological therapies. Substance Abuse. 2014;8:55–62. http://doi.org/10.4137/SART.S18446 Domhardt M. et al. Mechanisms of change in digital health interventions for mental disorders in youth: Systematic review. JMIR. 2021;23(11):e29742. http://doi.org/10.2196/29742 Paradis S, Roussel J, Bosson JL, Kern JB. Use of smartphone health apps among patients aged 18 to 69 years in primary care: Population-based cross-sectional survey. JMIR Formative Research. 2022;6(6):e34882. http://doi.org/10.2196/34882 Clement A, Ravet M, Stanger C, Gabrielli J. Feasibility, usability, and acceptability of MobileCoach-Teen: A smartphone app-based preventative intervention for risky adolescent drinking behavior. Journal of Substance Use and Addiction Treatment. 2024;159:209275. https://doi.org/10.1016/j.josat.2023.209275 Carlo AD, Hosseini Ghomi R, Renn BN, Areán PA. By the numbers: Ratings and utilization of behavioral health mobile applications. Npj Digital Medicine. 2019;2(1):1–8. https://doi.org/10.1038/s41746-019-0129-6 Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: Systematic search and panel-based usage analysis. JMIR. 2019;21(9):e14567. http://doi.org/10.2196/14567 Garrido S et al. What works and what doesn’t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Frontiers in Psychiatry. 2019;10. http://doi.org/10.3389/fpsyt.2019.00759 Aji M et al. Exploring user needs and preferences for mobile apps for sleep disturbance: Mixed methods study. JMIR Mental Health. 2019;6(5):e13895. http://doi.org/10.2196/13895 Crane D, Garnett C, Brown J, West R, Michie S. Factors influencing usability of a smartphone app to reduce excessive alcohol consumption: Think aloud and interview studies. Frontiers in Public Health . 2017;5:240357. https://doi.org/10.3389/fpubh.2017.00039 Lee-Tobin PA, Ogeil RP, Savic M, Lubman DI. Rate my sleep: Examining the information, function, and basis in empirical evidence within sleep applications for mobile devices. Journal of Clinical Sleep Medicine. 2017;13(11):1349–1354. http://dx.doi.org/10.5664/jcsm.6814 Oyebode O, Alhasani M, Mulchandani D, Olagunju T, Orji R. SleepFit: A persuasive mobile app for improving sleep habits in young adults. 2021 IEEE 9th International Conference on Serious Games and Applications for Health (SeGAH). IEEE . 2021;1–8. Litvin S, Saunders R, Maier MA, Lüttke S. Gamification as an approach to improve resilience and reduce attrition in mobile mental health interventions: A randomized controlled trial. PLOS ONE . 2020;15(9):e0237220. https://doi.org/10.1371/journal.pone.0237220 Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on Digital Mental Health Interventions for Adolescents and Young People: Systematic Overview. JMIR Mental Health . 2021;8(4). https://doi.org/10.2196/25847 Petersen CL et al. (2020). Using natural language processing and sentiment analysis to augment traditional user-centered design: Development and usability study. JMIR Mhealth Uhealth. 2020;8(8):e16862. http://doi.org/10.2196/16862 Skeen SJ, Jones SS, Cruse CM, Horvath KJ. Integrating natural language processing and interpretive thematic analyses to gain human-centered design insights on HIV mobile health: Proof-of-concept analysis. JMIR Human Factors . 2022.;9(3):e37350. http://doi.org/10.2196/37350 Oksanen J, et al. vegan : Community ecology package. R package version 2.6-4. 2022. https://CRAN.R-project.org/package=vegan Ding CS. Fundamentals of applied multidimensional scaling for educational and psychological research. Springer International Publishing . 2018. https://doi.org/10.1007/978-3-319-78172-3 Grün B, Hornik K. topicmodels: Topic models . R package version 0.2–14. 2023. https://CRAN.R-project.org/package=topicmodels Monarque M, Sabetti J, Ferrari M. Digital interventions for substance use disorders in young people: rapid review. Subst Abuse Treat Prev Policy. 2023;18(1):13. http://doi.org/10.1186/s13011-023-00518-1 Ash GI et al. Sensitivity, specificity, and tolerability of the BACTrack Skyn compared to other alcohol monitoring approaches among young adults in a field-based setting. Alcoholism: Clinical and Experimental Research. 2022;46(5):783–796. http://doi.org/10.1111/acer.14804 Hasler BP, Martin CS, Wood DS, Rosario B, Clark DB. A longitudinal study of insomnia and other sleep complaints in adolescents with and without alcohol use disorders. Alcoholism: Clinical and Experimental Research. 2014;38(8):2225–2233. http://doi.org/10.1111/acer.12474 Hasler BP, Kirisci L, Clark DB. Restless sleep and variable sleep timing during late childhood accelerate the onset of alcohol and other drug involvement. Journal of Studies on Alcohol and Drugs 2016;77(4):649–655. http://doi.org/10.15288/jsad.2016.77.649 Mike TB, Shaw DS, Forbes EE, Sitnick SL, Hasler BP. The hazards of bad sleep—Sleep duration and quality as predictors of adolescent alcohol and cannabis use. Drug and Alcohol Dependence. 2016;168:335–339. https://doi.org/10.1016/j.drugalcdep.2016.08.009 Demartini KS, Carey KB. Correlates of AUDIT risk status for male and female college students. Journal of American College Health . 2009;58(3):233–9. http://doi.org/10.1080/07448480903295342 Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption II. Addiction . 1993;88:791–804. http://doi.org/10.1111/j.1360-0443.1993.tb02093.x Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology . 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. 2023. Vienna, Austria. https://www.R-project.org/ Clarke KR. Non-parametric multivariate analyses of changes in community structure. Aust. J. Ecol . 1993;18:117–143. http://doi.org/10.1111/j.1442-9993.1993.tb00438.x Jockers ML. syuzhet: Extract sentiment and plot arcs from text. 2015. https://github.com/mjockers/syuzhet Arun R, Suresh V, Veni Madhavan CE, Narasimha Murthy MN. On finding the natural number of topics with Latent Dirichlet Allocation: Some observations. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (Eds.). Advances in Knowledge Discovery and Data Mining. PAKDD 2010. Lecture Notes in Computer Science . 2010;6118. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13657-3_43 Griffiths TL, Steyvers M, Blei DM, Tenenbaum JB. (2004). Integrating topics and syntax. In Advances in Neural Information Processing Systems . 2004;17. Cao J, Xia T, Li J, Zhang Y, Tang S. A density-based method for adaptive LDA model selection. Neurocomputing. 2009;72(7–9):1775–1781. https://doi.org/10.1016/j.neucom.2008.06.011 Nikita M. ldatuning: Tuning of the Latent Dirichlet Allocation models parameters. R package version 1.0.2. 2020. https://CRAN.R-project.org/package=ldatuning Nielsen, F. A. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. In Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big things come in small packages 718 in CEUR Workshop Proceedings. 2011:93–98. http://arxiv.org/abs/1103.2903 Additional Declarations (Not answered) Supplementary Files SupplementalMaterials.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: revise 01 Aug, 2024 Review # 3 received at journal 17 Jul, 2024 Reviewer # 3 agreed at journal 07 Jul, 2024 Review # 2 received at journal 05 Jul, 2024 Review # 1 received at journal 29 May, 2024 Reviewer # 2 agreed at journal 20 May, 2024 Reviewer # 1 agreed at journal 17 May, 2024 Reviewers invited by journal 25 Mar, 2024 Editor assigned by journal 22 Feb, 2024 Submission checks completed at journal 22 Feb, 2024 First submitted to journal 21 Feb, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3977182","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":283583770,"identity":"f0dbe9d8-8473-47a9-aa40-606e6bd70be5","order_by":0,"name":"Frances Griffith","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABE0lEQVRIiWNgGAWjYBAC+wNwJmPDhwoDhgR+IOsAmItDiwGCydg444yBQYIkUCmxWhgYZ5xhMEgwOEBIi/ThZw9+VNQxyEcfbmw4UPAnz/hG8oMDHxhsZDccwK7Fni/N3LDnzGEGw3OJQC0GBsVmN9IMDs5gSDPGpcWAh8FMmrHtAINhD2P74w8GBonbbiQYHOZhOJyIWwv7N2nGf3UgLWBbEjfPSP9w+A/DfzxaeIC2NDAzyPNAtWyQyDE4DAwBfFrKJHuOHeYxgGgxTpxx5k3BwR6DZOOZuB22TeJHTZ2cfA/7w4YDf+QS+9vTNwLD0E62D4cWGOAxgCsQSGBAjS9cQL4BxuInYPooGAWjYBSMOAAAClVm2qQplFEAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0001-9238-0212","institution":"Yale School of Medicine","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Frances","middleName":"","lastName":"Griffith","suffix":""},{"id":283583771,"identity":"52b4e9e4-9961-458b-95f5-2605af5c3c9c","order_by":1,"name":"Garrett Ash","email":"","orcid":"","institution":"Yale","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Garrett","middleName":"","lastName":"Ash","suffix":""},{"id":283583772,"identity":"1859d52d-69fc-43e5-857f-646da4ac98c5","order_by":2,"name":"Madilyn Augustine","email":"","orcid":"","institution":"Yale School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Madilyn","middleName":"","lastName":"Augustine","suffix":""},{"id":283583773,"identity":"0bbb343f-c899-4684-a252-7e3f7c0987eb","order_by":3,"name":"Leah Latimer","email":"","orcid":"","institution":"Yale School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Leah","middleName":"","lastName":"Latimer","suffix":""},{"id":283583774,"identity":"49b1bd9a-b38c-4f3b-a2d8-5697050d45be","order_by":4,"name":"Naomi Verne","email":"","orcid":"","institution":"Yale School of Public Health","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Naomi","middleName":"","lastName":"Verne","suffix":""},{"id":283583775,"identity":"8bca02c2-b15f-4008-817f-ec4d122bff33","order_by":5,"name":"Nancy Redeker","email":"","orcid":"","institution":"University of Connecticut School of Nursing","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Nancy","middleName":"","lastName":"Redeker","suffix":""},{"id":283583776,"identity":"82a5b96d-1447-45cb-8f8a-a8037593e3be","order_by":6,"name":"Stephanie O'Malley","email":"","orcid":"","institution":"Yale School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Stephanie","middleName":"","lastName":"O'Malley","suffix":""},{"id":283583777,"identity":"343b7735-5894-4234-ad95-a55c85750f55","order_by":7,"name":"Kelly DeMartini","email":"","orcid":"","institution":"Yale School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Kelly","middleName":"","lastName":"DeMartini","suffix":""},{"id":283583778,"identity":"d8ef6a8a-432b-440d-a839-d83f6f473c07","order_by":8,"name":"Lisa Fucito","email":"","orcid":"","institution":"Yale School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Lisa","middleName":"","lastName":"Fucito","suffix":""}],"badges":[],"createdAt":"2024-02-22 01:02:46","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3977182/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3977182/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53682192,"identity":"939b94d0-6fab-4d13-bf45-3e31f7c52b60","added_by":"auto","created_at":"2024-03-28 20:36:49","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":49981,"visible":true,"origin":"","legend":"\u003cp\u003eHybrid convergent mixed methods with NLP for user experience evaluation. Descriptive and predictive analyses were used with exit surveys. Targeted qualitative thematic analysis, post hoc NMDS and multivariate correlation, and NLP were used with exit interviews.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/83a91ad0a61a878612c17caf.png"},{"id":53682193,"identity":"dd736871-fe76-4607-ab0a-66627c96ae41","added_by":"auto","created_at":"2024-03-28 20:36:49","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":385189,"visible":true,"origin":"","legend":"\u003cp\u003eNMDS ordination plot of user experience themes from exit interviews. This ordination plot shows study participants (points) and themes (text) from thematic analysis mapped on the first two dimensions of the NMDS ordination. Polygons drawn around participant points indicate trial groups. Themes about helpful aspects of interventions are shown in blue, themes about unhelpful aspects of interventions are shown in orange, and themes about self-reported most influential interventions are shown in black. “W” stands for website, “B” stands for biosensors, “D” stands for diary, and “FB” stands for feedback.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/3d469d448d0622d46a105820.png"},{"id":53682194,"identity":"3b4c9365-7bf0-4d99-afce-25206d503b91","added_by":"auto","created_at":"2024-03-28 20:36:49","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":136957,"visible":true,"origin":"","legend":"\u003cp\u003eLDA topic and term frequency in exit interviews. This bar plot shows the frequency of the topics from the LDA among 112 exit interviews and feedback transcripts. The ten most common terms within each topic are shown in the order of frequency.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/fb5ab50534d0e049a1dc8050.png"},{"id":53682680,"identity":"ac86c235-09f4-4a92-ba09-2519932b79d4","added_by":"auto","created_at":"2024-03-28 20:44:49","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":18414,"visible":true,"origin":"","legend":"\u003cp\u003eExit interview sentiment frequency. This histogram shows the frequency of interviews’ net sentiment scores based on the AFINN lexicon\u003csup\u003e42\u003c/sup\u003e among 112 exit interviews and feedback transcripts.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/fe9c3007c7f5e57a69abbbb5.png"},{"id":53682824,"identity":"a0b3ced0-e183-4620-8461-2b02b5cd3d82","added_by":"auto","created_at":"2024-03-28 20:52:50","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1047155,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/d81f7554-898b-4acb-82c6-e273df3ffe4d.pdf"},{"id":53682195,"identity":"aaa63631-73dd-457e-8d1c-1dce92d542ab","added_by":"auto","created_at":"2024-03-28 20:36:49","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":263264,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementalMaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3977182/v1/62e8652a817273fcc9716608.pdf"}],"financialInterests":"(Not answered)","formattedTitle":"Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use","fulltext":[{"header":"Introduction","content":"\u003cp\u003eMost mental health disorders have a peak age of onset during or before young adulthood (18\u0026ndash;25 years)\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. However, less than half of young adults with any mental health disorder receive treatment, with the proportion dropping to around one-third among young adults of color\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Clinical services are often unavailable or unaffordable for young adults, and the gap between treatment demand and availability has increased since the COVID-19-pandemic\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e. Further, young adults may not seek treatment for some highly prevalent but normalized mental health concerns, like alcohol use disorders (AUDs) and heavy (\u0026ldquo;binge\u0026rdquo;) drinking\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e,\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. In contrast, young adults show concern for other common wellness issues, like sleep quality, that are related to alcohol use\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e. In a sample of young adults with similar levels of sleep complaints and heavy drinking, most (80%) were concerned about sleep whereas very few (5%) were concerned about drinking\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e. Therefore, sleep interventions could offer a more appealing alternative for young adults facing sensitive issues like alcohol risk\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eDigital interventions can also increase appeal, access, and convenience of mental health treatment for young adults\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e. This population makes extensive use of digital health tools compared to other adults\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e, with generally positive experiences\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e. Despite a saturated market of mobile mental health apps and other digital tools, user ratings may be discrepant and uninformative to app designers and potential users alike\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e. Over 95% of users stop using most mobile mental health apps after 30 days\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e and adherence and engagement are generally low\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e. In the case of sleep and alcohol apps, user engagement may increase with features like personalized feedback, interaction and support from providers, self-monitoring, and user-friendliness\u003csup\u003e\u003cspan additionalcitationids=\"CR17 CR18\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e. Further, gamification may significantly reduce attrition in mobile mental health app use in general\u003csup\u003e\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e. With the incorporation of machine learning, apps are also becoming increasingly more personalized. Assessing both users\u0026rsquo; perspectives and intervention quality is critical to promote higher treatment uptake and adherence in a rapidly growing, global digital health market\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e. Therefore, implementing precise, consistent evaluations of user experiences represents a significant gap in digital intervention research.\u003c/p\u003e \u003cp\u003eIncorporating natural language processing (NLP) with convergent mixed methods can support user-centered design for digital health interventions\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e. Natural language processing is an umbrella term for a growing suite of machine learning methods that uses artificial intelligence (AI) to understand (e.g., summarize, retrieve information) and/or generate language content. Traditional evaluation methods, such as qualitative thematic analysis of exit interviews, can reveal nuance and detail, but can be impractical, especially with large sample sizes\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. Further, exit surveys using Likert-type satisfaction scales may be subject to response bias. A hybrid approach combining NLP, quantitative survey analysis, and targeted qualitative interview analysis, may reveal broad and rich user experiences \u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e while maximizing researcher time and effort.\u003c/p\u003e \u003cp\u003eThe current study used convergent mixed methods to evaluate user experiences in a randomized clinical trial of \u0026ldquo;Call it a Night\u0026rdquo; (\u003cem\u003eCIAN)\u003c/em\u003e and two control conditions (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). \u003cem\u003eCIAN\u003c/em\u003e is a personalized feedback and coaching digital sleep intervention for young adults that addresses heavy alcohol and other substance use (NCT #03658954). In the parent trial \u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e, \u003cem\u003eCIAN\u003c/em\u003e was tested against control conditions comprising either web-based advice only (\u003cem\u003eA)\u003c/em\u003e or advice\u0026thinsp;+\u0026thinsp;active diary self-monitoring (\u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM)\u003c/em\u003e. All participants wore sleep and alcohol biosensors. NLP methods, particularly topic modeling analysis (Latent Dirichlet Allocation) and sentiment analysis, were selected to assess convergence with qualitative thematic analysis of exit interviews and descriptive and predictive analysis with exit surveys. The aims of the study were to leverage these NLP evaluation methods to:\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003echaracterize young adults\u0026rsquo; user experiences during the digital sleep interventions for heavy drinking and\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003edetermine whether user experiences varied with their demography (e.g., age, race, gender, student status, psychiatric diagnoses) or trial condition.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n \u003ch2\u003eSample\u003c/h2\u003e\n \u003cp\u003eWe recruited 120 trial participants, and 118 completed the exit survey. Half (51%) were female and 80% were white with a mean age of 21.14 years (see Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e). Three-fourths (74%) were college students. Most (81%) met lifetime criteria for any MHD or substance use disorder (SUD) based on a diagnostic interview, and most also met lifetime criteria for an AUD (72%).\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003e\u003cem\u003eCIAN Exit Survey Sample Characteristics (n\u0026thinsp;=\u0026thinsp;118)\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eIntervention group\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eA\u003c/em\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eCIAN\u003c/em\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTotal \u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSample characteristic\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003en\u003c/em\u003e (% of 29)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003en\u003c/em\u003e (% of 29)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003en\u003c/em\u003e (% of 60)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003en\u003c/em\u003e (% of 118)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGender\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14 (48.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16 (55.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30 (50.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e60 (50.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15 (51.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13 (44.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30 (50.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e58 (49.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRace\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAsian\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3 (10.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2 (6.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (8.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10 (8.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlack\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (3.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (3.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7 (11.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9 (7.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWhite\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (86.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e24 (82.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e45 (75.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e94 (79.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMultiracial\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0 (0.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0 (0.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (1.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (0.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOther\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0 (0.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2 (6.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2 (3.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (3.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEthnicity\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNot Hispanic, Latine\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e24 (82.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e24 (82.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e51 (85.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99 (83.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHispanic, Latine\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (17.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (17.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9 (15.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19 (16.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStudent status\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStudent\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21 (72.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e23 (79.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e43 (71.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e87 (73.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNon-student\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 (27.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (20.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17 (28.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e31 (26.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHistory of mental health or\u003c/p\u003e\n \u003cp\u003esubstance use diagnosis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e23 (79.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (86.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e48 (80.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e96 (81.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (20.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (13.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12 (20.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e22 (18.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHistory of alcohol use diagnosis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17 (58.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e24 (82.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e44 (73.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e85 (72.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12 (41.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (17.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16 (26.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e33 (28.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"5\" align=\"left\"\u003e\n \u003cp\u003eAUDIT Total Score (\u003cem\u003eM\u003c/em\u003e, \u003cem\u003eSD\u003c/em\u003e, range)\u0026thinsp;=\u0026thinsp;13.02, 4.70, 5\u0026ndash;30 score\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"5\" align=\"left\"\u003e\n \u003cp\u003eAge (\u003cem\u003eM\u003c/em\u003e, \u003cem\u003eSD\u003c/em\u003e, range)\u0026thinsp;=\u0026thinsp;21.14, 1.76, 18\u0026ndash;25 years\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003cem\u003eNote\u003c/em\u003e. \u0026ldquo;CIAN\u0026rdquo; stands for Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice. \u0026ldquo;A\u0026thinsp;+\u0026thinsp;SM\u0026rdquo; stands for advice plus self-monitoring control condition, which also includes passive monitoring. \u0026ldquo;A\u0026rdquo; stands for advice control condition, which also includes passive monitoring. \u0026ldquo;AUDIT\u0026rdquo; stands for Alcohol Use Disorders Identification Test\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n \u003cp\u003e\u003csup\u003ea\u003c/sup\u003e There were no significant differences between trial groups on any demographic variable.\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n \u003ch2\u003eThematic Analysis of Exit Interviews\u003c/h2\u003e\n \u003cp\u003eQualitative thematic analysis of exit interviews resulted in overarching thematic categories related to helpfulness of different intervention components and suggestions from participants (see Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e for themes and salience; see Supplemental Materials for all thematic analysis definitions and exemplar quotes).\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\u0026nbsp;\u003cem\u003eCIAN Thematic Analysis Salience (N = 107)\u003c/em\u003e\u003c/div\u003e\n \u003c/caption\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThematic Category\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003en\u003c/em\u003e (%) \u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful aspects of website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGeneral helpfulness of website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e69 (\u003cstrong\u003e64.5%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful sleep-related information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e69 (\u003cstrong\u003e64.5%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNew information from website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e58 (54.2% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eReminders of known information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42 (39.3% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUsefulness of website strategies\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e38 (35.5% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWebsite understandability\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e28 (26.2% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful alcohol-related information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (23.4% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUser-friendliness of website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (23.4% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMemorability of information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9 (8.4% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnhelpful aspects of website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLack of new information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14 (\u003cstrong\u003e13.1%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIrrelevance of information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (4.7% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDifficulty implementing strategies\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (4.7% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful aspects of biosensors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle biosensor increased awareness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e36 (\u003cstrong\u003e33.6%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGeneral helpfulness of biosensor(s)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e35 (32.7% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWatch biosensor increased awareness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e33 (30.8% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNeutral aspect of biosensors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNeutral aspect of biosensors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e65 (\u003cstrong\u003e60.7%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnhelpful aspects of\u003c/p\u003e\n \u003cp\u003ebiosensors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBurdensomeness of ankle biosensor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e22 (\u003cstrong\u003e20.6%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnhelpful without data feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16 (15.0% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLack of behavior change\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12 (11.2% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBurdensomeness of watch biosensor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 (7.5% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStigma of ankle biosensor\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (5.6% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle biosensor increased drinking\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (4.7% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful aspects of diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIncreased mindfulness of sleep\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42 (\u003cstrong\u003e53.8%\u003c/strong\u003e of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGeneral helpfulness of diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e38 (48.7% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIncreased mindfulness of alcohol use\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e31 (39.7% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNew experience of keeping a diary\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e22 (28.2% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEase of answering questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17 (21.8% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMotivation to change behaviors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10 (12.8% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNeutral aspect of diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNeutral aspect of diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13 (\u003cstrong\u003e16.7%\u003c/strong\u003e of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnhelpful aspects of diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eChallenges answering questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13 (\u003cstrong\u003e16.7%\u003c/strong\u003e of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLack of behavior change\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (7.7% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLack of new information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (7.7% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful aspects of feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpfulness of personalization\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e45 (\u003cstrong\u003e90.0%\u003c/strong\u003e of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGeneral helpfulness of feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e29 (58.0% of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHelpful data presentations\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e27 (54.0% of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnhelpful aspects of feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLack of new information\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (\u003cstrong\u003e8.0%\u003c/strong\u003e of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMost influential interventions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFeedback was most influential\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e36 (\u003cstrong\u003e72.0%\u003c/strong\u003e of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWebsite was most influential\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e39 (36.4% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary was most influential\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (32.1% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBiosensors were most influential\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17 (15.9% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMultiple things were helpful\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15 (14.0% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSuggestions for interventions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWilling to wear more devices (watch)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e65 (\u003cstrong\u003e60.7%\u003c/strong\u003e of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWant diet and exercise-related feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e63 (58.9% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWilling to complete new diary questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e60 (56.1% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWant caffeine-related feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e53 (49.5% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWant other potential feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e33 (30.8% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWant environment-related feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e26 (24.3% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnwilling to add question and/or device\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e18 (16.8% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImprovements for feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (10.0% of 50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImprovements for diaries\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6 (7.7% of 78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImprovements for biosensors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 (7.5% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImprovements for website\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (3.7% of 107)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003ctfoot\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\"\u003e\u003cem\u003eNote.\u003c/em\u003e Theme salience (% of those who stated the theme) is based on the number of participants who answered the interview question that resulted in the theme. General questions and questions about the website and biosensors were asked of all 107 participants across trial groups. Questions about diaries were asked of the 78 \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e group participants who completed the diaries and the exit interviews. Questions about feedback were asked of the 50 \u003cem\u003eCIAN\u003c/em\u003e participants who received feedback and completed the exit interview.\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tfoot\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003eWeb-based Sleep Advice (Website): Helpful and Unhelpful Aspects (n\u0026thinsp;=\u0026thinsp;107)\u003c/h2\u003e\n \u003cp\u003eParticipants highlighted various aspects of the web-based advice intervention, both helpful and unhelpful. The most mentioned helpful aspects were \u003cem\u003eGeneral helpfulness of the website\u003c/em\u003e and \u003cem\u003eHelpful sleep-related information\u003c/em\u003e: \u0026ldquo;I think the particularly helpful ones [tips] were\u0026hellip;things to do when you wake up, because I think \u0026hellip;when I wake up, I just don\u0026apos;t want to get out of bed, so I\u0026rsquo;ll\u0026hellip;stay in bed on my phone for a long time.\u0026rdquo; Participants appreciated learning \u003cem\u003eNew information from the website, Reminders of known information\u003c/em\u003e, and \u003cem\u003eUsefulness of website strategies\u003c/em\u003e: \u0026ldquo;It [the website] talked about\u0026hellip;your body temperature\u0026hellip;keeping a window open or having a fan to keep your body [cool].\u0026rdquo; Others noted helpful aspects of \u003cem\u003eWebsite understandability, User-friendliness of website, Helpful alcohol-related information\u003c/em\u003e, and \u003cem\u003eMemorability of information\u003c/em\u003e. By contrast, relatively few participants described unhelpful aspects. Some mentioned \u003cem\u003eLack of new information\u003c/em\u003e as a drawback, while others described \u003cem\u003eIrrelevance of information\u003c/em\u003e, such as content related to cannabis and sleep. A few participants reported \u003cem\u003eDifficulty implementing strategies\u003c/em\u003e recommended on the website.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n \u003ch2\u003ePassive Monitoring (Biosensors): Helpful, Neutral, and Unhelpful Aspects (n\u0026thinsp;=\u0026thinsp;107)\u003c/h2\u003e\n \u003cp\u003eParticipants across trial groups provided comments on whether wearing the biosensors (transdermal alcohol ankle monitor and sleep watch) was helpful on its own without consideration of the data collected. Most described \u003cem\u003eNeutral aspects of biosensors\u003c/em\u003e and did not find them helpful or unhelpful without data. Among those who found the biosensors helpful on their own, the most mentioned aspect was that the \u003cem\u003eAnkle biosensor increased awareness\u003c/em\u003e of their alcohol use: \u0026ldquo;The physical monitors all over you\u0026hellip;they definitely make you\u0026hellip;think like, \u0026lsquo;Wow\u0026hellip;should I really be drinking this much?\u0026rsquo;\u0026rdquo; Likewise, others noted the \u003cem\u003eGeneral helpfulness of the biosensor(s)\u003c/em\u003e or that the \u003cem\u003eWatch biosensor increased awareness\u003c/em\u003e of their sleep patterns irrespective of data obtained: \u0026ldquo;If I saw my sleep watch before I was going to bed, I was like, \u0026lsquo;You know what, I should probably try and go to bed early tonight.\u0026rsquo;\u0026rdquo; Among those who found one or both biosensors unhelpful, the most common complaint was \u003cem\u003eBurdensomeness of ankle biosensor\u003c/em\u003e, including itchiness, discomfort, and difficulty completing some activities: \u0026ldquo;I just really didn\u0026apos;t like the [ankle] device\u0026hellip;couldn\u0026apos;t run as much.\u0026rdquo; Other unhelpful aspects mentioned were \u003cem\u003eUnhelpful without data feedback\u003c/em\u003e, \u003cem\u003eLack of behavior change, Burdensomeness of watch biosensor\u003c/em\u003e (e.g., forgetting to push buttons), \u003cem\u003eStigma of ankle biosensor\u003c/em\u003e (e.g., connection to court-ordered monitors), and \u003cem\u003eAnkle biosensor increased drinking.\u003c/em\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003eActive Self-Monitoring (Diaries): Helpful, Unhelpful, and Neutral Aspects (n\u0026thinsp;=\u0026thinsp;78)\u003c/h2\u003e\n \u003cp\u003e\u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e participants commented on the helpfulness of completing diaries, a component of their study conditions. Most described one or more helpful aspects, most commonly that diaries \u003cem\u003eIncreased mindfulness of sleep\u003c/em\u003e: \u0026ldquo;I found it\u0026hellip;helpful to\u0026hellip;realize it [my sleep schedule] and\u0026hellip;just reflect on\u0026hellip;my lifestyle and\u0026hellip;how much sleep I\u0026apos;m actually getting.\u0026rdquo; Others noted the \u003cem\u003eGeneral helpfulness of diaries\u003c/em\u003e and \u003cem\u003eIncreased mindfulness of alcohol use\u003c/em\u003e: \u0026ldquo;I\u0026hellip;never really\u0026hellip;used to like keeping track [of drinking]. So, I\u0026apos;d\u0026hellip;drink a lot, and then I\u0026apos;d feel too drunk.\u0026rdquo; Some noted the helpfulness of the \u003cem\u003eNew experience of keeping a diary\u003c/em\u003e, the \u003cem\u003eEase of answering questions\u003c/em\u003e in the diaries, or the \u003cem\u003eMotivation to change behaviors\u003c/em\u003e prompted by diaries. Fewer participants noted unhelpful aspects of diaries, such as \u003cem\u003eChallenges answering questions\u003c/em\u003e in the diaries, \u003cem\u003eLack of behavior change\u003c/em\u003e, and \u003cem\u003eLack of new information\u003c/em\u003e (i.e., self-monitoring did not increase awareness of patterns nor motivate them to change behaviors. Some also noted a \u003cem\u003eNeutral aspect of diaries\u003c/em\u003e, they were neither helpful nor unhelpful.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003ePersonalized, Tailored Coaching (Feedback): Helpful and Unhelpful Aspects (n\u0026thinsp;=\u0026thinsp;50)\u003c/h2\u003e\n \u003cp\u003eParticipants in the \u003cem\u003eCIAN\u003c/em\u003e condition with personalized feedback and coaching largely described helpful aspects of the feedback and coaching intervention component. Almost all \u003cem\u003eCIAN\u003c/em\u003e group participants described the \u003cem\u003eHelpfulness of personalization\u003c/em\u003e, the individualized nature of reports and suggestions based on monitoring data: \u0026ldquo;Having that sort of feedback, both the data and just the explanation behind it, I think that\u0026apos;s a really\u0026hellip;unique insight into\u0026hellip;a part of yourself that\u0026hellip;most people wouldn\u0026apos;t ever really get to actually see mapped out.\u0026rdquo; Most also noted aspects, such as the \u003cem\u003eGeneral helpfulness of feedback\u003c/em\u003e and \u003cem\u003eHelpful data presentations\u003c/em\u003e: \u0026ldquo;She [the coach] made it very easy for me to understand the charts and what everything meant.\u0026rdquo; Only a few described one unhelpful aspect of feedback: \u003cem\u003eLack of new information\u003c/em\u003e, i.e., reports did not contribute new insights into their sleep or drinking.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n \u003ch2\u003eMost Influential Interventions\u003c/h2\u003e\n \u003cp\u003eEach participant was asked to select which received intervention component(s) were most influential for behavior change. In order from most to least helpful, participants selected personalized feedback/tailored coaching (i.e., three-fourths of \u003cem\u003eCIAN\u003c/em\u003e participants), the website (i.e., over one-third of all participants), diaries (i.e., almost one-third of \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e participants), and biosensors (i.e., less than one-fifth of all participants).\u003c/p\u003e\n \u003cdiv id=\"Sec10\" class=\"Section3\"\u003e\n \u003ch2\u003eSuggestions for Future Interventions\u003c/h2\u003e\n \u003cp\u003eDuring exit interviews, participants also offered suggestions for current and future intervention improvement. Many participants were interested in measuring additional sleep-related factors, most commonly that they \u003cem\u003eWant diet and exercise-related feedback\u003c/em\u003e: \u0026ldquo;[I\u0026rsquo;d like to know] what time when you exercise, how it affects your sleep\u0026hellip;what that does to\u0026hellip;your whole body chemistry, everything.\u0026rdquo; Others mentioned they \u003cem\u003eWant caffeine-related feedback\u003c/em\u003e, \u003cem\u003eWant environment-related feedback\u003c/em\u003e (e.g., impacts of light, noise, and temperature on sleep), or \u003cem\u003eWant other potential feedback\u003c/em\u003e (e.g., cognitive alertness, tobacco, and stress): \u0026ldquo;Maybe you can see how stressed you are during\u0026hellip; the week. Just get those numbers, and then I think that would be pretty interesting to figure [out].\u0026rdquo; To receive additional feedback, most responded that they would be \u003cem\u003eWilling to wear more devices\u003c/em\u003e and \u003cem\u003eWilling to complete new diary questions\u003c/em\u003e. Fewer stated they were \u003cem\u003eUnwilling to add a question and/or device\u003c/em\u003e, usually disinterest in another ankle biosensor. Several volunteered suggestions for received interventions in the current study. These included \u003cem\u003eImprovements for feedback\u003c/em\u003e (e.g., combined, summary report for both intervention weeks, \u003cem\u003eImprovements for diaries\u003c/em\u003e (e.g., higher pay per diary), \u003cem\u003eImprovements for biosensors\u003c/em\u003e (e.g., eliminating ankle biosensors, measuring GPS location), and \u003cem\u003eImprovements for website\u003c/em\u003e (e.g., wider distribution, alternative summaries).\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\n \u003ch2\u003eNonmetric Multidimensional Scaling of Interview Themes\u003c/h2\u003e\n \u003cp\u003eWe used multivariate analysis, nonmetric multidimensional scaling (NMDS)\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e, to visualize the interrelationships among qualitative themes and participants. NMDS is an ordination method that condenses variation in matrices, such as participant-by-theme frequency, to a small number of orthogonal dimensions\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e. We used five orthogonal dimensions based on stress-level testing, and the first two dimensions are plotted in vector space in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e. Also, multivariate correlational analysis allowed us to fit participant characteristics as factors to the NMDS ordination of themes. Trial group (\u003cem\u003eR\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;.29, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.001) and lifetime history of any MHD/SUD (\u003cem\u003eR\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;.03, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.04) were significantly correlated with NMDS scores. That is, participants\u0026rsquo; statements about study interventions in their exit interviews were associated with trial group and diagnostic history. Other participant characteristics (e.g., age, gender, race, ethnicity, student status) were not associated with NMDS scores. In the NMDS shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e participants were closer to themes of feedback and diary helpfulness than website or biosensor helpfulness in vector space, which suggests their preference for feedback and diaries. Conversely, control \u003cem\u003eA\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e participants were closer to themes of helpful website aspects in vector space than \u003cem\u003eCIAN\u003c/em\u003e participants, and control participants were also significantly more likely to state these themes during interviews (\u003cem\u003eX\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;27.34, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;.001).\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eNatural Language Processing of Exit Interviews (\u003c/strong\u003e \u003cstrong\u003en\u003c/strong\u003e\u0026thinsp;\u003cstrong\u003e=\u0026thinsp;112)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eWe also used quantitative Latent Dirichlet Allocation (LDA), a topic modeling analysis that identifies the likelihood of terms occurring in topics and topics occurring in documents\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e, with an expanded dataset of 112 participants\u0026rsquo; exit interviews and coaching transcripts and identified nine topics across participants\u0026rsquo; statements (see Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e; see Supplemental Material for all topic definitions and example quotes). The goals of NLP analyses were to help qualitative thematic analysis be more targeted and assess convergence of findings with thematic analysis and exit survey analyses. These topics did not vary significantly by trial condition (\u003cem\u003eX\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;16.72, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.40). We found that \u003cem\u003eAwareness with monitoring\u003c/em\u003e was the most likely topic in the largest number of interviews (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;18, 16%). This included participants\u0026rsquo; perceptions that both active diary self-monitoring and passive biosensor monitoring increased their awareness and mindfulness of their behaviors. As one participant stated, \u0026ldquo;Knowing that I had to do the sleep diary the next day and\u0026hellip;people were watching what I was doing\u0026hellip;I was just more aware of my habits.\u0026rdquo;\u003c/p\u003e\n \u003cp\u003eOther common topics focused on web-based advice, especially sleep strategies. The topic, \u003cem\u003eWebsite improves sleep\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;16, 14%), included experiences of beginning to improve sleep using sleep strategies, such as consistent sleep schedules. A participant described, \u0026ldquo;My sleeping habits were definitely awful before now, and I think they\u0026apos;ve improved a lot, since starting this [study]\u0026hellip;I\u0026apos;m starting to go to bed earlier and\u0026hellip;reduce my activities before bed.\u0026rdquo; Almost as many interviews were most likely to include the topic, \u003cem\u003eStrategy barriers\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;15, 13%), or descriptions of challenges implementing sleep strategies, such as situational factors (e.g., work and school schedules, dormitory environment) or personal factors (e.g., memory, motivation). The topic \u003cem\u003eChanging poor sleep\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;15, 13%), included descriptions of poor sleep quality or nonrestorative sleep, how this motivated study participation, and a desire to learn about different sleep factors.\u003c/p\u003e\n \u003cp\u003eOther topics focused on participants\u0026rsquo; interest in gaining more wellness-related knowledge to change sleep or alcohol use. Thirteen interviews (12%) were most likely to include the topic \u003cem\u003eLearning and reduced drinking\u003c/em\u003e, which focused on gaining new information from digital interventions and reducing drinking given its impacts on sleep. The topic \u003cem\u003eUsing feedback and website\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;11, 10%) included participants\u0026rsquo; accounts of how they integrated personalized feedback and coaching with web-based advice, including information and strategies, to alter their sleep and alcohol use. The topic \u003cem\u003eMultiple strategies and factors\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;10, 9%) focused on participants\u0026rsquo; curiosity and interest in the impacts of varied sleep-related factors, such as situations, environments, substances other than alcohol, and diet and exercise.\u003c/p\u003e\n \u003cp\u003eThe two least frequent topics included the burden of wearable biosensor devices. The topic \u003cem\u003eStrategies, not devices\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;8, 7%) focused on helpful website aspects, attempts to implement sleep strategies, and challenges with wearable devices, especially the transdermal ankle monitor. Similarly, the topic \u003cem\u003eFeedback, not devices\u003c/em\u003e (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;6, 5%) focused on the benefits of personalized feedback and coaching while also discussing difficulties with wearable devices.\u003c/p\u003e\n \u003cp\u003eThe sentiment scores of exit interviews and transcripts (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;112) were generally positive (\u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;15.07, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;10.54) and ranged from \u0026minus;\u0026thinsp;16 to 47 (see Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e). Positive vs. negative sentiment scores reflect the aggregate valence of participants\u0026rsquo; word usage in exit interviews, so \u0026ldquo;0\u0026rdquo; reflects a neutral net score in an interview. Sentiment scores did not vary as a function of trial condition (\u003cem\u003eCIAN\u003c/em\u003e, \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e, and \u003cem\u003eA\u003c/em\u003e; \u003cem\u003eF\u003c/em\u003e(2, 109)\u0026thinsp;=\u0026thinsp;0.78, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.46). Positive sentiment scores also did not vary by demographic characteristics, including gender (\u003cem\u003eF\u003c/em\u003e(1, 110)\u0026thinsp;=\u0026thinsp;0.62, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.43), age (\u003cem\u003eb\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.03, \u003cem\u003et\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.06, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.95), race (\u003cem\u003eF\u003c/em\u003e(4, 107)\u0026thinsp;=\u0026thinsp;1.23, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.30), ethnicity (\u003cem\u003eF\u003c/em\u003e(1, 110)\u0026thinsp;=\u0026thinsp;0.20, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.66), student status (\u003cem\u003eF\u003c/em\u003e(3, 108)\u0026thinsp;=\u0026thinsp;0.75, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.53), and history of any MHD/SUD (\u003cem\u003eF\u003c/em\u003e(1, 110)\u0026thinsp;=\u0026thinsp;0.23, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.63). However, the word length of each document was positively correlated with sentiment scores (\u003cem\u003er\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.21, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.03), indicating a tendency for participants who spoke at longer length to speak more positively about their experiences. To boost rigor and account for this correlation, we derived a valence-per-word score for each document and reran sentiment comparisons across condition and demographic groups with the valence-per-word score as the outcome. There were still no between-group differences (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026gt;\u0026thinsp;0.05).\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eExit Survey (\u003c/strong\u003e \u003cstrong\u003en\u003c/strong\u003e\u0026thinsp;\u003cstrong\u003e=\u0026thinsp;118)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eOn the exit survey, participants across conditions (\u003cem\u003eCIAN\u003c/em\u003e, \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e, and \u003cem\u003eA\u003c/em\u003e) reported generally positive user experiences (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;118; see Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e). On a 5-point scale, participants reported high overall program satisfaction, website helpfulness, diary helpfulness, and feedback helpfulness. Mean feasibility ratings were generally high for the overall program, website, watch biosensor, and diaries. However, mean feasibility ratings for the ankle biosensor were lower based on visual inspection, especially due to interference with clothing, feeling uncomfortable, and being noticeable. Perceived effectiveness was higher among \u003cem\u003eCIAN\u003c/em\u003e (\u0026Delta;\u0026thinsp;=\u0026thinsp;0.48, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.008) and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e participants (\u0026Delta;\u0026thinsp;=\u0026thinsp;0.55, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;.001) than \u003cem\u003eA\u003c/em\u003e participants (\u003cem\u003eF\u003c/em\u003e(2, 115)\u0026thinsp;=\u0026thinsp;8.45, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;.001). Further, young adults with a lifetime history of any MHD/SUD rated their intervention as more effective on average than those without diagnoses (\u003cem\u003eF\u003c/em\u003e(1, 116)\u0026thinsp;=\u0026thinsp;4.64, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;.001; \u0026Delta;\u0026thinsp;=\u0026thinsp;0.32, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.03).\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003e\u003cem\u003eCIAN Exit Survey Outcomes (N\u0026thinsp;=\u0026thinsp;118)\u003c/em\u003e\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eOutcome (Agreement 1\u0026ndash;5)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eA\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCIAN\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTotal\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGeneral program outcomes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOverall satisfaction\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.45 (0.74)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.48 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.60 (0.56)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.53 (0.61)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnderstandability\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.87)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.59 (0.83)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.65 (0.66)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.60 (0.75)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSchedule workability\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.66 (0.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.62 (0.86)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.67 (0.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.65 (0.85)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAppropriate visit length\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.31 (1.11)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.10 (1.14)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.27 (0.92)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (1.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLifestyle change promotion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.03 (0.87)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.10 (0.77)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.23 (0.96)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.15 (0.89)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eComfortability\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.76 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.66 (0.81)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.57 (0.81)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.64 (0.80)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHope for change promotion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.00 (0.89)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.10 (1.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.04 (0.96)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImportance of target habits\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.48 (0.87)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.69)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.57 (0.81)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.53 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOverall effectiveness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.00 (0.76)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.48 (0.51)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.55 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.40 (0.64)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAdvice (A) outcomes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModule 1 helpfulness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (0.74)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (0.64)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.43 (0.65)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.67)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModule 2 helpfulness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.07 (0.96)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.31 (0.66)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.45 (0.65)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.32 (0.75)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUser-friendliness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.51)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.41 (0.50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.40 (0.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.43 (0.76)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eComparability to other\u003c/p\u003e\n \u003cp\u003ewebsites\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.76 (0.44)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.73 (0.58)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.68 (0.55)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEnjoyment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (0.73)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.00 (0.77)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.98 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePassive-monitoring\u003c/p\u003e\n \u003cp\u003e(biosensor) outcomes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch comfortable\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.93 (1.13)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.14 (0.69)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.92 (0.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (0.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch not embarrassing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.00 (1.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (0.64)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.23 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.18 (0.86)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not interfere\u003c/p\u003e\n \u003cp\u003ewith work\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.97)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.55 (0.51)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.62)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.48 (0.70)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not interfere\u003c/p\u003e\n \u003cp\u003ewith exercise\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.41 (0.78)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.31 (0.66)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.32 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not interfere\u003c/p\u003e\n \u003cp\u003ewith sleep\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.41 (0.68)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.30 (0.89)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not interfere\u003c/p\u003e\n \u003cp\u003ewith concentration\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.59 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.55 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.52 (0.60)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.54 (0.58)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not interfere\u003c/p\u003e\n \u003cp\u003ewith clothing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.41 (0.87)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.21 (0.98)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.18 (1.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.25 (0.97)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch not noticeable\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.07 (1.00)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.17 (0.93)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.17 (0.96)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.14 (0.95)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch did not change\u003c/p\u003e\n \u003cp\u003eroutine\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.59 (0.57)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.38 (0.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.28 (1.01)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.38 (0.88)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eActiwatch wear more\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (0.87)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.81)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.27 (0.92)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.28 (0.88)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor comfortable\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.15 (0.95)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.66 (1.11)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.60 (1.08)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.51 (1.07)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor not\u003c/p\u003e\n \u003cp\u003eembarrassing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.59 (1.19)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.66 (1.42)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.98 (1.38)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.81 (1.35)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor did not\u003c/p\u003e\n \u003cp\u003einterfere with work\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.37 (1.08)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.79 (1.05)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.05 (1.00)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.83 (1.06)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor did not\u003c/p\u003e\n \u003cp\u003einterfere with exercise\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.48 (1.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.03 (1.30)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.02 (1.32)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.90 (1.29)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor did not\u003c/p\u003e\n \u003cp\u003einterfere with sleep\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.04 (1.19)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.34 (1.14)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.25 (1.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.22 (1.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle mon. did not\u003c/p\u003e\n \u003cp\u003eInterfere with\u003c/p\u003e\n \u003cp\u003econcentration\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.56 (1.25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.05 (1.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.91 (1.06)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor did not\u003c/p\u003e\n \u003cp\u003einterfere with clothing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.33 (1.47)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.17 (1.28)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.55 (1.43)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.41 (1.40)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor not\u003c/p\u003e\n \u003cp\u003enoticeable\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.85 (1.22)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.66 (1.37)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.60 (1.24)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.67 (1.26)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor did not\u003c/p\u003e\n \u003cp\u003echange routine\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.11 (1.25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.21 (1.18)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.10 (1.36)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.12 (1.28)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAnkle monitor wear more\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.26 (1.53)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.93 (1.25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.42 (1.37)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.26 (1.38)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSelf-monitoring (SM)\u003c/p\u003e\n \u003cp\u003eoutcomes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary helpfulness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.07 (0.72)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.19 (0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.15 (0.76)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary easiness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34 (0.61)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.58 (0.56)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.51 (0.59)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary not burdensome\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.97 (1.09)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.17 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.10 (0.97)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary did not change\u003c/p\u003e\n \u003cp\u003eschedule\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.24 (0.83)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.27 (1.06)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.26 (0.98)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary easy to remember\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.72 (1.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.92 (1.00)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.85 (1.05)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary enjoyable\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.52 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.83 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.73 (0.91)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDiary would continue\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.52 (1.15)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.83 (0.92)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.73 (1.01)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFeedback (F) outcomes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSleep feedback helpfulness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.70 (0.53)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.70 (0.53)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAlcohol feedback\u003c/p\u003e\n \u003cp\u003ehelpfulness\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.65 (0.55)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.65 (0.55)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003ctfoot\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"5\"\u003e\u003cem\u003eNote\u003c/em\u003e. \u0026ldquo;CIAN\u0026rdquo; stands for Call it a Night, the full digital sleep intervention, including personalized feedback and coaching, active diary self-monitoring, passive biosensor monitoring, and web-based advice. \u0026ldquo;A\u0026thinsp;+\u0026thinsp;SM\u0026rdquo; stands for advice plus self-monitoring control condition, which also includes passive monitoring. \u0026ldquo;A\u0026rdquo; stands for advice control condition, which also includes passive monitoring.\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"5\"\u003eTotal means and standard deviations are based on 118 participants who completed the exit interview, including 60 CIAN, 29 A\u0026thinsp;+\u0026thinsp;SM, and 29 A participants. Some items had minor levels of missingness (\u0026lt;\u0026thinsp;5 participants).\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tfoot\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eAdherence to interventions was also high across study phases. Almost all participants (98%) completed the two-week intervention phase, and 96% completed the 12-week follow-up appointment. Regarding monitoring activities, 98% of participants wore the sleep watch biosensor for 14 days, and 95% wore the alcohol ankle biosensor for 14 days. Further, 95% of participants in the \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e groups completed their assigned 14 days of active diary self-monitoring of sleep and alcohol use.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe current study used an innovative hybrid approach in which NLP findings converged with and corroborated more traditional mixed methods to efficiently evaluate user experiences with a digital sleep intervention to reduce drinking. Major findings showed that participants across conditions generally found all intervention components helpful, satisfying, and feasible, but the \u003cem\u003eCIAN\u003c/em\u003e group markedly preferred the personalized feedback and tailored coaching with a health professional they received to other components. Convergent results also underscored young adults\u0026rsquo; strong interest in gaining a more holistic picture of their wellness in the future and increasing their awareness using monitoring through diaries and biosensors. Our hybrid approach using NLP enabled qualitative thematic analysis to target specific interview questions more efficiently, optimizing researcher\u0026rsquo;s time. Further, assessing the convergence of findings between NLP and other methods helped address potential researcher bias in thematic analysis and potential user response bias in exit surveys.\u003c/p\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eThe Importance of Personalized and Interactive Feedback and Coaching\u003c/h2\u003e \u003cp\u003eMixed methods findings on user experiences, including NLP, emphasized greater effectiveness and satisfaction with personalized feedback and coaching than other intervention components despite generally favorable experiences with web-based advice, active diary self-monitoring, and passive biosensor monitoring. \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e group participants who received feedback and/or actively self-monitored (diaries) rated their intervention as more effective than \u003cem\u003eA\u003c/em\u003e control participants in the exit survey. Topic modeling (LDA) and thematic analysis showed that some participants found web-based advice strategies difficult to implement, and negative themes about web-based advice were associated more with \u003cem\u003eCIAN\u003c/em\u003e participants.\u003c/p\u003e \u003cp\u003eOur study adds insight to extant literature on young adults\u0026rsquo; preferences for interactive digital interventions, including alcohol and sleep treatment. An umbrella overview of meta-analyses and systematic reviews\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e found that digital interventions offering interaction with a health professional or other social support had higher adherence and effectiveness and lower attrition. Mobile alcohol treatment apps providing personalized feedback may be especially engaging\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e. Further, recent research on mobile sleep apps\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e highlights increased engagement and satisfaction with personalized feedback and communication with health professionals. Our study adds findings that individually tailored coaching from a health professional significantly enhanced participants\u0026rsquo; experience interpreting digital health data, which likely supported their understanding. Current research also shows that user-friendliness\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e and evidence-based information\u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e increase engagement, which was consistent with our finding that web-based advice was considered helpful and user-friendly, especially by control groups. Therefore, whereas web-based advice and other automated interventions are likely to be helpful to many young adults, tailored feedback and coaching with individualized data are likely to be preferable in the current digital health market.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eEmpowering Young Adults with Digital Health Tools for General Wellness\u003c/h2\u003e \u003cp\u003eConsistent with prior findings \u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e, our results showed that young adults particularly want to use digital tools to pursue holistic health and wellness goals. Thematic analysis and LDA with interviews converged to show that participants are interested in understanding varied other lifestyle factors that impact sleep beyond alcohol (e.g., caffeine, diet, exercise, stress, environment). Most participants expressed a willingness to use additional digital tools (e.g., more biosensors, additional diary self-monitoring) to attain whole-health, personalized feedback. LDA results highlighted greater perceived awareness and mindfulness of sleep and drinking through diary self-monitoring and/or wearing biosensors. Thus, while young adults are interested in and capable of using varied digital health tools, incorporating appealing treatment foci could be important to increase young adults\u0026rsquo; generally low engagement and adherence\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eOur study suggests that digital interventions improving overall health may represent a more appealing treatment approach to young adults for potentially stigmatizing diagnoses, such as AUDs. Most available digital SUD interventions for young adults involve web-based advice explicitly targeting alcohol use to the exclusion of other substance use or wellness factors\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e. Whereas young adults may not be concerned about alcohol use or seek traditional alcohol treatment \u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e, they are concerned about improving sleep quality \u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e,\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e and other aspects of their general wellness. In this study, LDA with exit interviews showed that participants commonly joined the study to improve their sleep quality rather than address their alcohol use. Further, survey and interview thematic analysis results, respectively, showed that participants with a lifetime history of an MHD/SUD found the current intervention to be especially effective and described intervention helpfulness differently than those with no diagnoses. Therefore, digital interventions, like the current study which focused more explicitly on improving wellness (e.g., enhancing sleep quality), may be especially engaging to young adults with AUDs or other diagnoses who have not presented to treatment.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003eImplications and Recommendations\u003c/h2\u003e \u003cp\u003eOur study has implications for clinical researchers and mobile app designers making user-centered app designs. User experience evaluations of digital health tools should incorporate a hybrid, convergent approach to optimize depth, breadth, and efficiency. Each user experience evaluation method has distinct strengths and potential weaknesses \u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. Thematic analysis of exit interviews can provide detailed information across complex subjective experiences. In the current study, along with post hoc analyses, thematic analysis revealed more distinctions between perceived helpfulness of different intervention components than any other method. However, as sample sizes increase, qualitative thematic analysis can become impractical and unfeasible. Our use of NLP topic modeling (LDA) to derive overarching topics across interview questions enabled us to target and abbreviate our thematic analysis to interview questions focusing on intervention component helpfulness. Exit surveys are also time-efficient like NLP but may be subject to user response bias. NLP, specifically sentiment analysis, helped confirm the generally positive perceptions derived from exit survey analyses.\u003c/p\u003e \u003cp\u003eOur convergent evaluation findings also have specific implications for digital intervention design. Young adults are enthusiastic about receiving a broad, holistic picture of factors impacting their wellness, including different aspects of their lifestyle, in addition to appreciating individualized and specific coaching. Therefore, mobile mental health apps and linked biosensors should have a variety of choices for active and passive monitoring that can be tailored to user feasibility and interest. Digital tools should integrate seamlessly into young adults\u0026rsquo; schedules that often include exercise, which may make ankle transdermal monitors challenging and potentially stigmatizing\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e compared to watch biosensors. Also, digital interventions, including for sleep and alcohol use, should empower young adults to actively explore their wellness with support from a health professional. While participants in the current study valued the mindfulness derived from digital health monitoring, they preferred to also receive personalized data reports with explanations and suggested health tips from a coach. Some individuals may lack the expertise to interpret their health data or to devise action plans for improvement. Thus, digital interventions should include mechanisms for interactive feedback, such as an on-call health provider, pre-scheduled video or text check-ins with a health provider, or a chatbot that provides a spectrum of suggestions and individual data descriptions.\u003c/p\u003e \u003cp\u003eDigital interventions for specific, sensitive clinical issues, like heavy drinking, could engage more young adults via a focus on related general health and wellness behaviors, like sleep. Interventions that increase accessibility and engagement for young adults are important given that only half of young adults with psychiatric disorders currently receive any mental health treatment\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Digital sleep interventions for reduced drinking can target two risky, prevalent, and modifiable issues: drinking and sleep problems. Young adults commonly experience AUDs, binge drinking \u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e, and sleep problems\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e. Further, sleep problems in adolescence lead to heavier drinking and AUDs in adulthood\u003csup\u003e\u003cspan additionalcitationids=\"CR30\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. Notably, young adults who drink heavily may be less concerned about their drinking than their sleep\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e. In general, digital tools that explicitly focus on less sensitive behavioral goals, like improved sleep, with implications for improving potentially stigmatized goals, like reduced drinking, are likely to be more appealing and engaging to young adults.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003eStrengths and Limitations\u003c/h2\u003e \u003cp\u003eDistinct strengths of the current study included a relatively large sample size, especially for a qualitative analysis of exit interviews; generally high adherence across intervention phases and groups; and minor amounts of missing data. Each of these attributes increased the validity of findings. Further, our hybrid, convergent approach incorporating NLP represents an important strength as these methods are promising and relatively new in digital medicine\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. Our research also had notable limitations. Although missing data were minor, it is possible that additional themes would have emerged in a complete set of exit interviews. NLP helped address response bias in exit survey items, but survey responses were not normally distributed and generally skewed towards higher responses. Our sample was primarily college students, so it may not be representative of young adults in general. Further, study participants were paid, so implementation and uptake may differ in clinical or other naturalistic settings.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe current evaluation demonstrates the value of NLP in convergent mixed methods to efficiently capture broad and nuanced user experiences with digital health interventions. Specifically, our results show that digital sleep interventions for heavy drinking may increase appeal and access to alcohol treatment for young adults, especially when they include tailored coaching. Broadly, the current findings emphasize the importance of digital tools for young adults that provide a holistic, dynamic view of health coupled with interactive, individualized feedback. Consistent and precise evaluations that leverage user feedback are critical to support higher uptake and adherence to effective digital health interventions, which can begin to address the mental health treatment demand gap for young adults.\u003c/p\u003e "},{"header":"Methods","content":"\u003cdiv id=\"Sec19\" class=\"Section3\"\u003e \u003ch2\u003eStudy design\u003c/h2\u003e \u003cp\u003eIn the current study, we evaluated user experiences during a randomized clinical trial of \u003cem\u003eCIAN\u003c/em\u003e, a novel, two-session personalized feedback and coaching digital sleep intervention developed for young adults to reduce drinking (\u003cem\u003eN\u003c/em\u003e\u0026thinsp;=\u0026thinsp;120). Participants were randomly assigned to one of three conditions using a 2:1:1 ratio for two weeks: 1) \u003cem\u003eCIAN\u003c/em\u003e: two-session \u0026ldquo;Call it a Night\u0026rdquo; intervention (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;60) or 2) one of two control conditions, \u003cem\u003eA\u003c/em\u003e: brief advice only (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;30) or \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e: brief advice\u0026thinsp;+\u0026thinsp;active sleep diary self-monitoring (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;30). For this outpatient study in participants\u0026rsquo; natural environments, all wore sleep and alcohol biosensors daily for two weeks and viewed a two-module web-based sleep advice program during brief, midweek research appointments. There were no significant differences between the trial groups on any demographic variable (i.e., age, gender, race, ethnicity, student status, lifetime history of psychiatric diagnosis, or baseline alcohol use). Of the participants, 118 completed the exit survey and 107 completed the exit interview with study staff.\u003c/p\u003e \u003cp\u003eTo evaluate user experiences, we used convergent mixed methods with NLP following the two-week intervention phase. Exit surveys and interviews were conducted in parallel with participants across trial conditions to attain both broad and deep perspectives on satisfaction and intervention acceptability and feasibility. Both qualitative thematic analysis and quantitative NLP examined exit interviews. Further, in line with convergent mixed methodology, exit survey results were analyzed using descriptive and predictive analyses to compare these findings with exit interview results. This study was approved by the Institutional Review Board of Yale University (IRB# 2000021048). Additional study details can be found in the published study protocol paper \u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003ch2\u003eRecruitment\u003c/h2\u003e \u003cp\u003eTo be eligible for the \u003cem\u003eCIAN\u003c/em\u003e clinical trial, participants needed to a) be 18\u0026ndash;25 years of age, b) be fluent in English, c) self-report three or more heavy drinking occasions in the last two weeks (i.e., 5 or more drinks on one occasion for men or 4 or more for women), d) score at risk of harm from drinking on the AUDIT-C\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, and d) self-report sleep concerns. Exclusion criteria are described further in the study protocol paper\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e. Participants were recruited using online (e.g., Facebook, Instagram, Snapchat) and in-person advertisements placed around the local community. Online advertisements and flyers directed individuals to a web-based pre-screening survey, and those who met pre-screening eligibility were invited to an intake visit for final eligibility determination.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003eExit Survey and Interview\u003c/h2\u003e \u003cp\u003eUpon completion of the two-week intervention phase, we asked participants to complete a self-report exit survey (see Supplemental Material for exit survey items). The exit survey included Likert-type and yes/no questions to assess user experiences, including overall satisfaction, intervention helpfulness and effectiveness, and intervention feasibility. Participants were also asked to detail areas for improvement in open-ended response items.\u003c/p\u003e \u003cp\u003e Participants across trial groups also completed semi-structured exit interviews to provide information about their subjective user experiences (see Supplemental Material for exit interview protocol). Exit interviews were administered by three research staff members (two men, one woman), who were trained in the protocol and familiar with study interventions. The exit interview protocol included questions about the following topics: participants\u0026rsquo; sleep and alcohol use before and after the study, perceptions of these behaviors relative to their peers, use of sleep hygiene strategies from the intervention, helpfulness of intervention components, the impact of payment on study participation, what drew them to the study, and suggestions for intervention improvement or future studies targeting sleep or alcohol use.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eWe conducted both qualitative thematic analysis and quantitative NLP with exit interviews. For thematic analysis, we used Braun and Clarke\u0026rsquo;s six-step process\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e, during which we used a recursive and iterative team coding process to ensure rigor and trustworthiness in the process of identifying, naming, and categorizing recurrent themes. Two members of the research team completed an initial open coding of the entire text of a large subset of exit interviews (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;80) to derive initial themes. Then, guided by findings from NLP topic modeling analysis, the current thematic analysis targeted key portions of the exit interviews, including questions related to helpfulness of intervention components and participants\u0026rsquo; suggestions. Three research team members engaged in a rigorous team coding process of the entire dataset of completed two-week exit interviews (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;107). Using an initial codebook based on previous first-round, open thematic coding, these three researchers independently coded a random selection of 20% of the entire interview dataset (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;21) to evaluate interrater reliability. All themes had percent agreement of 95% or greater, but an extensive reconciliation meeting was still undertaken to audit the initial codebook and resolve any disagreements until an overall kappa exceeding 0.7 was reached between each pair of coders. The remaining exit interviews were divided among the three researchers to code using the revised codebook. Auditing was used throughout the qualitative process to ensure transparency, including field notes maintained by each coder and reviewed by other coders and ongoing check-ins with the entire research team.\u003c/p\u003e \u003cp\u003eTo visualize the interrelationship between participants and themes, we conducted NMDS with theme frequency counts using the \u003cem\u003evegan\u003c/em\u003e package\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e in the programming language R\u003csup\u003e\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e\u003c/sup\u003e. NMDS is a multivariate ordination method that condenses variation in matrices to a small number of orthogonal dimensions\u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e. We selected NMDS for post-hoc analysis of themes because it does not place assumptions of normality on frequency count data compared to other multidimensional scaling methods \u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e. To assess the goodness of fit when selecting the number of orthogonal dimensions, we used stress level cutoffs and a stress plot (Shepard diagram)\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e. We selected five orthogonal dimensions, resulting in a good to fair stress score of .13, which is likely to result in minimal distortion\u003csup\u003e\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e\u003c/sup\u003e. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, we used a multivariate correlation analysis with \u003cem\u003evegan\u003c/em\u003e in R to fit factors (trial condition and participant gender, race, ethnicity, student status, and diagnosis history) and a vector (participant age) to the NMDS ordination of participants and themes. This enabled us to assess whether participants\u0026rsquo; user experiences (as interview themes) varied significantly with their assigned trial condition or personal characteristics.\u003c/p\u003e \u003cp\u003eFor NLP of exit interviews, we conducted topic modeling \u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e and sentiment analyses\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e with an expanded dataset of all of participants\u0026rsquo; statements during completed exit interviews and feedback session transcripts (\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;112). LDA with the \u003cem\u003etopicmodels\u003c/em\u003e package \u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e in R was used for topic modeling analysis. To boost rigor and ensure the most parsimonious number of topics (\u003cem\u003ek\u003c/em\u003e) were used for LDA, we undertook a preliminary analysis with three model selection methods to test different \u003cem\u003ek\u003c/em\u003e values\u003csup\u003e\u003cspan additionalcitationids=\"CR39\" citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e in the \u003cem\u003eldatuning\u003c/em\u003e package\u003csup\u003e\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e in R and ultimately selected nine topics. Text in participants\u0026rsquo; statements was preprocessed before NLP, including removing punctuation, numbers, capitalization, and English \u0026lsquo;stopwords\u0026rsquo; as well as stemming all remaining words. The LDA determined which of the nine topics was most likely to occur in each document and which terms were most likely to occur in each topic. Topics were systematically named by closely reading five or more interview transcripts determined most likely to contain the topic by LDA. Chi-squared tests determined whether topics varied with participants\u0026rsquo; assigned trial condition.\u003c/p\u003e \u003cp\u003eFor sentiment analysis, we used the AFINN lexicon\u003csup\u003e\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e and the \u003cem\u003esyuzhet\u003c/em\u003e package \u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e in R. The AFINN lexicon is a well-established and commonly used sentiment lexicon \u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e that assigns a valence score from \u0026minus;\u0026thinsp;5 to 5 to selected terms, and documents are scored based on an aggregate of their terms\u0026rsquo; scores. An aggregate AFINN sentiment score of \u0026ldquo;0\u0026rdquo; for a document indicates neutral term valence. Bivariate regression models, including analysis of variance (ANOVA), were used to assess whether participants\u0026rsquo; trial condition or personal characteristics predicted the sentiment of their statements during exit interviews.\u003c/p\u003e \u003cp\u003eDescriptive and predictive analyses were conducted with exit survey results. These included summary statistics of each item and linear modeling to determine if participants\u0026rsquo; trial condition or personal characteristics predicted responses to exit survey items.\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments:\u0026nbsp;\u003c/strong\u003eThis research was supported by the National Institute of Health and the National Institute on Alcohol Abuse and Alcoholism under Grants # R34AA026021, R34AA026021-S1, and R01AA03013 and the National Institute on Drug Abuse under Grant # T32DA019426. Further, we would like to thank Daniel Griffith for his consultation on statistical analyses conducted in R.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests statement:\u0026nbsp;\u003c/strong\u003eAll authors declare no financial or non-financial competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability statement:\u003c/strong\u003e The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode availability statement:\u003c/strong\u003e The underlying code for this study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions statement:\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eFJG: Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing\u0026mdash;original draft, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eGIA: Conceptualization, Methodology, Investigation, Project administration, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eMA: Data curation, Formal analysis, Validation, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eLL: Formal analysis, Validation, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eNV: Formal analysis\u003c/li\u003e\n \u003cli\u003eNSR: Conceptualization, Methodology, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eSSO: Conceptualization, Methodology, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eKSD: Conceptualization, Methodology, Supervision, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n \u003cli\u003eLMF: Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Validation, Writing\u0026mdash;review \u0026amp; editing\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eSolmi M et al. Age at onset of mental disorders worldwide: Large-scale meta-analysis of 192 epidemiological studies. Molecular Psychiatry. 2022;27(1):281\u0026ndash;295. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41380-021-01161-7\u003c/span\u003e\u003cspan address=\"10.1038/s41380-021-01161-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSAMHSA, Center for Behavioral Health Statistics and Quality. \u003cem\u003eResults from the 2022 National Survey on Drug Use and Health.\u003c/em\u003e 2023. Retrieved from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.samhsa.gov/data/report/2022-nsduh-detailed-tables\u003c/span\u003e\u003cspan address=\"https://www.samhsa.gov/data/report/2022-nsduh-detailed-tables\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMoreno C et al. How mental health care should change as a consequence of the COVID-19 pandemic. The Lancet Psychiatry. 2020;7(9):813\u0026ndash;824. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/S2215-0366(20)30307-2\u003c/span\u003e\u003cspan address=\"10.1016/S2215-0366(20)30307-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee MR, Sher KJ. \u0026ldquo;Maturing out\u0026rdquo; of binge and problem drinking. Alcohol Research. 2018;39:31\u0026ndash;42.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSingleton Jr. RA, Wolfson AR. Alcohol consumption, sleep, and academic performance among college students. Journal of Studies on Alcohol and Drugs. 2009;70(3):355\u0026ndash;63. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.15288/jsad.2009.70.355\u003c/span\u003e\u003cspan address=\"10.15288/jsad.2009.70.355\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFucito LM et al. A multimodal mobile sleep intervention for young adults engaged in risky drinking: protocol for a randomized controlled trial. JMIR Research Protocols. 2021;10(2):e26557. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/26557\u003c/span\u003e\u003cspan address=\"10.2196/26557\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSivertsen et al. Sleep patterns and insomnia in young adults: A national survey of Norwegian university students. Journal of Sleep Research. 2019;28(2):e12790. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1111/jsr.12790\u003c/span\u003e\u003cspan address=\"10.1111/jsr.12790\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAsh GI et al. Using web-based social media to recruit heavy-drinking young adults for sleep intervention: Prospective observational study. JMIR. 2020;22(8):e17449. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/17449\u003c/span\u003e\u003cspan address=\"10.2196/17449\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrooks AT, Wallen GR. Sleep disturbances in individuals with alcohol-related disorders: A review of cognitive-behavioral therapy for insomnia (CBT-I) and associated non-pharmacological therapies. Substance Abuse. 2014;8:55\u0026ndash;62. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.4137/SART.S18446\u003c/span\u003e\u003cspan address=\"10.4137/SART.S18446\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDomhardt M. et al. Mechanisms of change in digital health interventions for mental disorders in youth: Systematic review. JMIR. 2021;23(11):e29742. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/29742\u003c/span\u003e\u003cspan address=\"10.2196/29742\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eParadis S, Roussel J, Bosson JL, Kern JB. Use of smartphone health apps among patients aged 18 to 69 years in primary care: Population-based cross-sectional survey. JMIR Formative Research. 2022;6(6):e34882. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/34882\u003c/span\u003e\u003cspan address=\"10.2196/34882\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eClement A, Ravet M, Stanger C, Gabrielli J. Feasibility, usability, and acceptability of MobileCoach-Teen: A smartphone app-based preventative intervention for risky adolescent drinking behavior. Journal of Substance Use and Addiction Treatment. 2024;159:209275. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.josat.2023.209275\u003c/span\u003e\u003cspan address=\"10.1016/j.josat.2023.209275\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarlo AD, Hosseini Ghomi R, Renn BN, Are\u0026aacute;n PA. By the numbers: Ratings and utilization of behavioral health mobile applications. Npj Digital Medicine. 2019;2(1):1\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41746-019-0129-6\u003c/span\u003e\u003cspan address=\"10.1038/s41746-019-0129-6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBaumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: Systematic search and panel-based usage analysis. JMIR. 2019;21(9):e14567. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/14567\u003c/span\u003e\u003cspan address=\"10.2196/14567\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGarrido S et al. What works and what doesn\u0026rsquo;t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Frontiers in Psychiatry. 2019;10. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.3389/fpsyt.2019.00759\u003c/span\u003e\u003cspan address=\"10.3389/fpsyt.2019.00759\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAji M et al. Exploring user needs and preferences for mobile apps for sleep disturbance: Mixed methods study. JMIR Mental Health. 2019;6(5):e13895. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/13895\u003c/span\u003e\u003cspan address=\"10.2196/13895\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrane D, Garnett C, Brown J, West R, Michie S. Factors influencing usability of a smartphone app to reduce excessive alcohol consumption: Think aloud and interview studies. \u003cem\u003eFrontiers in Public Health\u003c/em\u003e. 2017;5:240357. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fpubh.2017.00039\u003c/span\u003e\u003cspan address=\"10.3389/fpubh.2017.00039\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee-Tobin PA, Ogeil RP, Savic M, Lubman DI. Rate my sleep: Examining the information, function, and basis in empirical evidence within sleep applications for mobile devices. \u003cem\u003eJournal of Clinical Sleep Medicine.\u003c/em\u003e 2017;13(11):1349\u0026ndash;1354. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://dx.doi.org/10.5664/jcsm.6814\u003c/span\u003e\u003cspan address=\"10.5664/jcsm.6814\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOyebode O, Alhasani M, Mulchandani D, Olagunju T, Orji R. SleepFit: A persuasive mobile app for improving sleep habits in young adults. \u003cem\u003e2021 IEEE 9th International Conference on Serious Games and Applications for Health (SeGAH). IEEE\u003c/em\u003e. 2021;1\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLitvin S, Saunders R, Maier MA, L\u0026uuml;ttke S. Gamification as an approach to improve resilience and reduce attrition in mobile mental health interventions: A randomized controlled trial. \u003cem\u003ePLOS ONE\u003c/em\u003e. 2020;15(9):e0237220. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1371/journal.pone.0237220\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0237220\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on Digital Mental Health Interventions for Adolescents and Young People: Systematic Overview. \u003cem\u003eJMIR Mental Health\u003c/em\u003e. 2021;8(4). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2196/25847\u003c/span\u003e\u003cspan address=\"10.2196/25847\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePetersen CL et al. (2020). Using natural language processing and sentiment analysis to augment traditional user-centered design: Development and usability study. \u003cem\u003eJMIR Mhealth Uhealth.\u003c/em\u003e 2020;8(8):e16862. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/16862\u003c/span\u003e\u003cspan address=\"10.2196/16862\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSkeen SJ, Jones SS, Cruse CM, Horvath KJ. Integrating natural language processing and interpretive thematic analyses to gain human-centered design insights on HIV mobile health: Proof-of-concept analysis. \u003cem\u003eJMIR Human Factors\u003c/em\u003e. 2022.;9(3):e37350. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.2196/37350\u003c/span\u003e\u003cspan address=\"10.2196/37350\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOksanen J, et al. \u003cem\u003evegan\u003c/em\u003e: Community ecology package. R package version 2.6-4. 2022. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://CRAN.R-project.org/package=vegan\u003c/span\u003e\u003cspan address=\"https://CRAN.R-project.org/package=vegan\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDing CS. Fundamentals of applied multidimensional scaling for educational and psychological research. \u003cem\u003eSpringer International Publishing\u003c/em\u003e. 2018. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-319-78172-3\u003c/span\u003e\u003cspan address=\"10.1007/978-3-319-78172-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGr\u0026uuml;n B, Hornik K. \u003cem\u003etopicmodels: Topic models\u003c/em\u003e. R package version 0.2\u0026ndash;14. 2023. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://CRAN.R-project.org/package=topicmodels\u003c/span\u003e\u003cspan address=\"https://CRAN.R-project.org/package=topicmodels\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMonarque M, Sabetti J, Ferrari M. Digital interventions for substance use disorders in young people: rapid review. Subst Abuse Treat Prev Policy. 2023;18(1):13. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1186/s13011-023-00518-1\u003c/span\u003e\u003cspan address=\"10.1186/s13011-023-00518-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAsh GI et al. Sensitivity, specificity, and tolerability of the BACTrack Skyn compared to other alcohol monitoring approaches among young adults in a field-based setting. \u003cem\u003eAlcoholism: Clinical and Experimental Research.\u003c/em\u003e 2022;46(5):783\u0026ndash;796. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1111/acer.14804\u003c/span\u003e\u003cspan address=\"10.1111/acer.14804\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHasler BP, Martin CS, Wood DS, Rosario B, Clark DB. A longitudinal study of insomnia and other sleep complaints in adolescents with and without alcohol use disorders. \u003cem\u003eAlcoholism: Clinical and Experimental Research.\u003c/em\u003e 2014;38(8):2225\u0026ndash;2233. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1111/acer.12474\u003c/span\u003e\u003cspan address=\"10.1111/acer.12474\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHasler BP, Kirisci L, Clark DB. Restless sleep and variable sleep timing during late childhood accelerate the onset of alcohol and other drug involvement. \u003cem\u003eJournal of Studies on Alcohol and Drugs\u003c/em\u003e 2016;77(4):649\u0026ndash;655. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.15288/jsad.2016.77.649\u003c/span\u003e\u003cspan address=\"10.15288/jsad.2016.77.649\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMike TB, Shaw DS, Forbes EE, Sitnick SL, Hasler BP. The hazards of bad sleep\u0026mdash;Sleep duration and quality as predictors of adolescent alcohol and cannabis use. \u003cem\u003eDrug and Alcohol Dependence.\u003c/em\u003e 2016;168:335\u0026ndash;339. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.drugalcdep.2016.08.009\u003c/span\u003e\u003cspan address=\"10.1016/j.drugalcdep.2016.08.009\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDemartini KS, Carey KB. Correlates of AUDIT risk status for male and female college students. \u003cem\u003eJournal of American College Health\u003c/em\u003e. 2009;58(3):233\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1080/07448480903295342\u003c/span\u003e\u003cspan address=\"10.1080/07448480903295342\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSaunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption II. \u003cem\u003eAddiction\u003c/em\u003e. 1993;88:791\u0026ndash;804. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1111/j.1360-0443.1993.tb02093.x\u003c/span\u003e\u003cspan address=\"10.1111/j.1360-0443.1993.tb02093.x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBraun V, Clarke V. Using thematic analysis in psychology. \u003cem\u003eQualitative Research in Psychology\u003c/em\u003e. 2006;3(2):77\u0026ndash;101. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1191/1478088706qp063oa\u003c/span\u003e\u003cspan address=\"10.1191/1478088706qp063oa\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eR Core Team. R: A language and environment for statistical computing. \u003cem\u003eR Foundation for Statistical Computing.\u003c/em\u003e 2023. Vienna, Austria. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.R-project.org/\u003c/span\u003e\u003cspan address=\"https://www.R-project.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eClarke KR. Non-parametric multivariate analyses of changes in community structure. \u003cem\u003eAust. J. Ecol\u003c/em\u003e. 1993;18:117\u0026ndash;143. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.org/10.1111/j.1442-9993.1993.tb00438.x\u003c/span\u003e\u003cspan address=\"10.1111/j.1442-9993.1993.tb00438.x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJockers ML. syuzhet: Extract sentiment and plot arcs from text. 2015. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/mjockers/syuzhet\u003c/span\u003e\u003cspan address=\"https://github.com/mjockers/syuzhet\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eArun R, Suresh V, Veni Madhavan CE, Narasimha Murthy MN. On finding the natural number of topics with Latent Dirichlet Allocation: Some observations. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (Eds.). \u003cem\u003eAdvances in Knowledge Discovery and Data Mining. PAKDD 2010. Lecture Notes in Computer Science\u003c/em\u003e. 2010;6118. Springer, Berlin, Heidelberg. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-642-13657-3_43\u003c/span\u003e\u003cspan address=\"10.1007/978-3-642-13657-3_43\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGriffiths TL, Steyvers M, Blei DM, Tenenbaum JB. (2004). Integrating topics and syntax. In \u003cem\u003eAdvances in Neural Information Processing Systems\u003c/em\u003e. 2004;17.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCao J, Xia T, Li J, Zhang Y, Tang S. A density-based method for adaptive LDA model selection. \u003cem\u003eNeurocomputing.\u003c/em\u003e 2009;72(7\u0026ndash;9):1775\u0026ndash;1781. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.neucom.2008.06.011\u003c/span\u003e\u003cspan address=\"10.1016/j.neucom.2008.06.011\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNikita M. ldatuning: Tuning of the Latent Dirichlet Allocation models parameters. R package version 1.0.2. 2020. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://CRAN.R-project.org/package=ldatuning\u003c/span\u003e\u003cspan address=\"https://CRAN.R-project.org/package=ldatuning\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNielsen, F. A. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. In \u003cem\u003eProceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big things come in small packages 718 in CEUR Workshop Proceedings.\u003c/em\u003e 2011:93\u0026ndash;98. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/1103.2903\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/1103.2903\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"natural language processing, digital health, sleep, alcohol use, user experience","lastPublishedDoi":"10.21203/rs.3.rs-3977182/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3977182/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEvaluating user experiences with digital interventions is critical to increase uptake and adherence, but traditional methods have limitations. We incorporated natural language processing (NLP) with convergent mixed methods to evaluate a personalized feedback and coaching digital sleep intervention for alcohol risk reduction: \u0026lsquo;Call it a Night\u0026rsquo; (\u003cem\u003eCIAN; N\u003c/em\u003e\u0026thinsp;=\u0026thinsp;120). In this randomized clinical trial with young adults with heavy drinking, control conditions were \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e: web-based advice\u0026thinsp;+\u0026thinsp;active and passive monitoring; and \u003cem\u003eA\u003c/em\u003e: advice\u0026thinsp;+\u0026thinsp;passive monitoring. Findings converged to show that the \u003cem\u003eCIAN\u003c/em\u003e treatment condition group found feedback and coaching most helpful, whereas participants across conditions generally found advice helpful. Further, most participants across groups were interested in varied whole-health sleep-related factors besides alcohol use (e.g., physical activity), and many appreciated increased awareness through monitoring with digital tools. All groups had high adherence, satisfaction, and reported feasibility, but participants in \u003cem\u003eCIAN\u003c/em\u003e and \u003cem\u003eA\u0026thinsp;+\u0026thinsp;SM\u003c/em\u003e reported significantly higher effectiveness than those in \u003cem\u003eA.\u003c/em\u003e NLP corroborated positive sentiments across groups and added critical insight that sleep, not alcohol use, was a main participant motivator. Digital sleep interventions are an acceptable, novel alcohol treatment strategy, and improving sleep and overall wellness may be important motivations for young adults. Further, NLP provides an efficient convergent method for evaluating experiences with digital interventions.\u003c/p\u003e","manuscriptTitle":"Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-28 20:36:44","doi":"10.21203/rs.3.rs-3977182/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"revise","date":"2024-08-01T12:52:39+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-07-17T14:43:03+00:00","index":3,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-07-07T07:21:53+00:00","index":3,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-07-05T21:33:52+00:00","index":2,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-05-29T06:35:21+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-05-20T15:01:51+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-05-17T06:44:05+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewersInvited","content":"","date":"2024-03-25T11:57:49+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-02-23T00:50:39+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-02-22T10:03:30+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Digital Medicine","date":"2024-02-22T00:57:36+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"dda26686-d901-45f1-9f8a-39050bb155cb","owner":[],"postedDate":"March 28th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":29851276,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":29851277,"name":"Scientific community and society/Social sciences/Psychology/Human behaviour"},{"id":29851278,"name":"Health sciences/Risk factors"},{"id":29851279,"name":"Health sciences/Medical research/Translational research"}],"tags":[],"updatedAt":"2024-10-30T19:30:43+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-28 20:36:44","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3977182","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3977182","identity":"rs-3977182","version":["v1"]},"buildId":"FbvkV6FR0MCFSLy54lSbu","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}