Usability and feasibility assessment of the Huntington Support App (HD-eHelp study): a mixed-methods study

Usability and feasibility assessment of the Huntington Support App (HD-eHelp study): a mixed-methods study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Usability and feasibility assessment of the Huntington Support App (HD-eHelp study): a mixed-methods study Pearl J. C. van Lonkhuizen, Leanne Slutter, Eline Meijer, Erik van Duijn, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5440332/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background: eHealth holds great promise for managing rare and complex neurodegenerative diseases, such as Huntington’s disease (HD). To address this potential, the Huntington Support App (HSA) was developed to provide reliable information on HD, news updates, and contact possibilities tailored primarily to the needs of HD gene expansion carriers (HDGECs) and their partners. Building on prior research outlining the HSA’s co-creation process with HDGECs, partners and health care providers (HCPs), the present study offers an in-depth mixed-methods evaluation of the HSA in practice in the Netherlands. Methods: This research comprises a multi-study report assessing the usability of the HSA through a think-aloud method (Study 1) and subsequent feasibility testing in a real-world setting (Study 2). In the first study, 4 HDGECs, 4 partners, and 4 HCPs, who themselves were not involved in the HSA co-design, provided real-time feedback on usability while using the HSA. In the feasibility study the HSA was publicly launched and made available to the larger HD community for feedback (including HDGECs, partners, HCPs and family/friends, n=13). In both studies, the System Usability Scale (SUS) was administered, and quantitative data were triangulated with qualitative findings. Results: Overall, participants were positive about the app’s usability in both studies. App ratings in the first study were notably higher than those in the second study, with participants’ SUS scores in the first study far exceeding the benchmark usually seen in usability testing for healthcare apps. Furthermore, in both studies there were variations in ratings among participant groups, with HDGECs, partners and family/friends viewing the app more favorably than HCPs, who generally expressed a more neutral opinion. Qualitative responses further complemented these findings and highlighted suggestions for the app’s improvement, particularly in terms of content and navigation. Conclusions: The HSA was well-received by members of the HD community, indicating alignment with their needs. Future efforts should address identified areas for improvement of the app. Our findings underscore the importance and feasibility of a participatory, human-centered design approach for complex and rare diseases like HD and highlight its relevance for future co-design initiatives of eHealth services for other rare neurodegenerative diseases. Huntington’s disease rare diseases neurodegenerative diseases telemedicine eHealth human-centered design tele-neurology digital health Figures Figure 1 Figure 2 Figure 3 1 Background eHealth has become increasingly important in the management of chronic conditions (1, 2). By utilizing information and communication technologies (3), eHealth offers a range of possibilities, including remote treatment options, remote monitoring, and information exchange (4, 5), all of which can increase care capacity (4). Moreover, eHealth offers the opportunity for personalized, tailored care from within the patient’s home (4, 5). This is particularly important in neurodegenerative diseases due to their progressive nature and the unique challenges they pose. Previous work has demonstrated the acceptability and effectiveness of eHealth services for several neurodegenerative and neurological diseases such as dementia, Parkinson’s disease, and multiple sclerosis (6-11). Participatory initiatives in the development of such services for these groups have been shown to be beneficial (12-14) and are also recommended for more rare diseases (15, 16), as active engagement of end-users increases the uptake and acceptance of eHealth services later on (5, 17-19). eHealth holds great promise in enhancing care delivery for rare (15, 16) and complex neurodegenerative diseases for which no cure is available, such as Huntington’s disease (HD). HD progressively affects motor, cognitive and psychiatric functioning over time (20-22), having a profound impact on work, family, and social life. Children have a 50% chance of inheriting the gene expansion from their affected parent, and if inherited, the disease usually manifest between 30 and 50 years of age (20, 23). HD gene expansion carriers (HDGECs) go through several disease phases, including the premanifest stage where (subtle) symptoms have not yet occurred (pre-symptomatic) or can begin to occur (prodromal) (23-25). After the onset of manifest disease, where clinical changes in motor functioning are evident (22, 23), life expectancy ranges between 15 to 20 years (21). Given the complex nature of HD and the challenges HDGECs face in managing the disease and its impact on their quality of life (26), eHealth provides a valuable opportunity to increase the availability and accessibility of HD care services across Europe. Previous research has shown that people affected by HD need accessible information about HD and are seeking out knowledge to better understand the disease (27, 28). Due to the absence of eHealth services co-designed with HDGECs to meet their specific needs (29-34), we developed the Huntington Support App (HSA) together with HDGECs, partners and health care providers (HCPs) from various countries (i.e., the Netherlands, Germany, Ireland, Italy and Czech Republic) (35, 36). The HSA is a web-app that offers clear and reliable information on HD, HD-related news and events, and contact possibilities with HCPs via an interactive walk-in hour and appointment booking tool (36). With direct participation of HDGECs, partners and HCPs from the different countries throughout the development process, the HSA is tailored to meet the needs and wishes of these groups. The prototype of the HSA was well-received and positively evaluated by HDGECs and their partners throughout its development phase (36). In this article, we present an in-depth mixed-methods evaluation of the HSA in practice. In the first part of the study, the app was tested on usability following the ‘think-aloud’ method to refine the app for future use. In the second part, we conducted a feasibility study in which the app was launched publicly and evaluated on usability by members of the Dutch HD community. 2 Part I – Think-aloud study 2.1 Methods 2.1.1 Participants and recruitment Dutch HDGECs (i.e., premanifest and manifest), partners and HCPs involved in HD care were recruited, as these same groups actively participated throughout the development process of the HSA (35, 36). It is important to note that all participants were newly recruited and did not have any prior involvement in the development of the HSA. Participants were recruited via the primary and secondary networks of Topaz Overduin, the Huntington Expertise Center that collaborated with the Leiden University Medical Center in co-developing the HSA in the Netherlands (35, 36). For HDGECs and partners, we also recruited via HD-related social media platforms and the Enroll-HD database (37). Eligible participants were aged 18 years or older and proficient in Dutch. HDGECs were residing at home and could participate without a partner. For an overview of all eligibility criteria, see (35). To address recruitment challenges, we broadened the initial inclusion criteria to include HCPs with less than two years of experience, partners living alone, and patient-partner dyads. All eligible participants received an information letter by (e-)mail, dependent on their preference, containing information on the development process of the app as well as information on study participation. 2.1.2 Study design and procedures A mixed-methods approach was used, in which qualitative and quantitative data on usability of the final prototype of the HSA was collected during individual ‘think-aloud’ sessions, and combined. For an example of the home page of the final prototype, see Figure 1. The think aloud sessions were conducted face-to-face at Huntington Expertise Center Topaz Overduin or Leiden University Medical Center (dependent on participants’ choice) in November and December 2022. One session was conducted online as it was more convenient and practically manageable for the participant. Participants were asked to verbalize their thoughts and actions as they interacted with the HSA prototype. This think-aloud method is a common procedure in human-centered design processes and provides in-depth insight into usability-related weaknesses and strengths of a system (38). Participants explored the app in the presence of a researcher on their preferred device (i.e., research laptop or own smartphone) and concurrently ‘thought aloud’ by describing their thoughts (e.g., “Strange picture” ) and feelings, and verbalizing their navigation behavior. The researcher provided prompts to encourage participants to express their thoughts and asked open-ended questions about certain aspects of the HSA (e.g., “How would you describe the app?” or “What was confusing?” ). Additionally, since HDGECs and partners were the primary target group of the HSA, they completed specific navigation tasks (“Where would you find information about symptoms?” ) to observe and evaluate the app’s navigability. The sessions were guided by a protocol (see Supplementary file 1) to standardize the procedure for all participants and conducted by an independent researcher (LS). At the beginning of each session, participants were explicitly informed that the researcher had not been involved in the app’s development. Sessions were audio-recorded and transcribed intelligent verbatim while masking all personally identifiable information. At the end of the sessions, participants completed an online questionnaire on sociodemographic information, digital skills and usability of the app. For the latter, the Dutch version of the System Usability Scale (SUS) was used. The SUS is a widely-used instrument for evaluating the usability of technologies or services (39). The SUS is comprised of ten items that are evaluated on a Likert scale ranging from 1 ‘strongly disagree’ to 5 ‘strongly agree’. The SUS total score ranges from 0 to 100, with higher scores indicating better usability (39). The SUS is considered a valid and reliable tool, and is also suitable for studies with a small sample size (40). This study was part of a larger project (HEALTHE-RND HD-eHelp study) that was cleared for ethics by the medical research ethics committee of Leiden Den Haag Delft in the Netherlands (file number: N20.013) (35). Informed consent was signed by all participants prior to the start of the think-aloud session. 2.1.3 Data analyses Qualitive data were analyzed using qualitative content analysis (41). This method is suitable for analyzing concrete feedback and user experiences due to its focus on the explicit and visible content of textual data (i.e., words, phrases) (41). All transcripts were read by the first (PL) and second author (LS) to become familiar with the data. Both researchers independently developed an inductively derived codebook based on two transcripts and subsequently discussed and merged both codebooks. The second author performed the initial coding of the remaining transcripts using the finalized codebook, allowing room for new codes to emerge during the analysis. The first author subsequently reviewed all transcripts, made revisions where necessary, and refined the overall coding scheme. The codes were then explored and grouped together in case of similar topics. Categories (groups of similar codes) were identified and grouped into broader themes. Transcripts were analyzed using ATLAS.ti version 23. Relevant quotes from participants were translated from Dutch to English by the first author and checked by the third author, thereby keeping close to the original wording and sentence structure used by participants. Descriptive statistics were used to analyze the quantitative data (IBM SPSS Statistics, version 29.0). SUS scores were recoded to obtain a total score, followed by calculating mean scores and the proportion of responses to each item (i.e., % completely disagree, slightly disagree, neither agree nor disagree, slightly agree, completely agree) for all participant groups. Responses to the SUS and the qualitative findings were triangulated. [INSERT FIGURE 1 HERE] Figure 1. Home page of the final prototype of the Huntington Support App Adopted from van Lonkhuizen PJC, Heemskerk A-W, Meijer E, van Duijn E, de Bot ST, Klempir J, Landwehrmeyer GB, Mühlbäck A, Hoblyn J, Squitieri F, Chavannes NH, Vegt NJH, and on behalf of the HEALTHE-RND consortium, 'Development of the Huntington Support App (HD-eHelp study): a human-centered and co-design approach', Frontiers in Neurology, 2024, 1:15:1399126, published under a Creative Commons Attribution License (CC BY). DOI: 10.3389/fneur.2024.1399126. The left part of the original figure has been left out. 2.2 Results In total, 4 HDGECs (2 premanifest and 2 manifest), 4 partners of HDGECs (2 of premanifest and 2 of manifest HDGECs), and 4 HCPs participated in this part of the study. HCPs were from different professions involved in HD care (i.e., elderly care physician, dietician, social worker and psychologist). See Table 1 for more information on participants’ characteristics. Table 1. Part I – Think-aloud study: Participants’ characteristics and mean SUS score at time of think-aloud session HDGECs (n = 4) Partners (n = 4) HCPs (n = 4) Age (mean; range) 48; 40-54 53; 34-64 42; 26-57 Gender (n (%)) Male 0 (0) 2 (50) 1 (25) Female 4 (100) 2 (50) 3 (75) Living situation (n (%)) - Together with partner 1 (25) 3 (75) - Alone 2 (50) 1 (25) - Together with others 1 (25) - - Profession (n (%)) Dietician - - 1 (25) Social worker - - 1 (25) Psychologist - - 1 (25) Elderly care physician - - 1 (25) Years of experience with HD (mean; range) - - 3.7; 1-10 Time since genetic test in years (mean; range) 11; 1-24 16; 6-27 1 - Use of health apps (n (%)) Often - 2 (50) - Sometimes - 2 (50) 2 (50) Rarely 3 (75) - 1 (25) Never 1 (25) - 1 (25) Self-reported digital skills (n (%)) Very experienced 2 (50) 2 (50) 2 (50) Experienced 2 (50) 2 (50) - Neither experienced or inexperienced - - 1 (25) Inexperienced - - - Very inexperienced - - 1 (25) SUS total score (mean; range) 84; 78-90 81; 60-90 65; 35-83 HDGECs: Huntington’s disease gene expansion carriers; N: number of participants; HCPs: Health care providers; SUS: System Usability Scale (SUS). Numbers and percentages are rounded to the nearest whole number. 1 This refers to the genetic test of the affected partner. Of the twelve participants, eleven participants preferred to use their personal smartphones to evaluate the HSA during the think-aloud session. A partner of a premanifest HDGEC preferred to use the research laptop. Overall, three themes related to the app’s usability were identified from the qualitative data, which triangulated and complemented the quantitative descriptive data from the SUS. Each theme is described below. Descriptive statistics for the SUS scores for the total sample can be found in Table 1 and Figure 2 and are reported as part of the three themes where relevant. For the proportion of responses for each item per participant group (i.e., HDGECs, partners and HCPs), the supplementary files can be referenced. 2.2.1 Theme 1: Usability of the Huntington Support App: user friendly and comprehensive, with emotionally sensitive content for some Overall, participants were positive about the app’s usability, as reflected in the total SUS score across all groups (i.e., 76.6), which exceeds the benchmark of 68 commonly found for the usability of health apps (42, 43). Particularly, HDGECs and partners were very positive. Their SUS total score averaged above 80 (i.e., 84 and 81 respectively, see Table 1). HCPs also expressed a positive attitude towards the app during the think-aloud sessions, yet had more feedback on and suggestions for improvement of the app than HDGECs and partners. This was also reflected in their overall lower SUS score of 65, slightly below the benchmark. Overall, participants reported the app to be user friendly and convenient: “[The app is] clear. Clean, I like that. I find it organized. […]. So I find it very user-friendly.” (premanifest HDGEC) Most participants mentioned that the app would especially be helpful in finding information about the disease as it bundles all relevant information about HD. One manifest HDGEC described the app as: “[…] a great improvement towards the digital world, where you can find everything about Huntington’s disease at a glance. From information about Huntington’s disease to making appointments to everything else. I find it very clear.” (manifest HDGEC) Particularly, HDGECs and partners mentioned the information in the app to be relevant and tailored to their needs. Next to providing information, all participants perceived the HSA as an app that provides the latest news updates, fosters connection with peers and HCPs, and facilitates in accessing healthcare services. Participants found the app’s content comprehensive and valuable. According to them, the information is presented concisely and clearly and is written in simple language. This was also observed in participants’ SUS item scores. Most participants found the app straightforward (84%), consistent (83%), and easy to use (75%) (see Figure 2 item 2, 6 and 3 respectively). Primarily HCPs found the HSA not so easy to use and unnecessarily complex. All participants but one HCP felt confident in using the HSA (see Supplementary Figure 1 item 9 in the Supplementary files). Participants who commented on the lay-out of the app especially liked the overall ‘professional’ look, as well as the text size and used colors. They liked that the font size is adjustable, which, according to some, comes in handy when the disease progresses and challenges in motor function arise. Next to outlining the app’s positive aspects, some participants also highlighted areas for improvement. Two premanifest HDGECs and one HCPs found certain information parts of the app somewhat confronting: “No, well, it is what it is, but indeed, there’s really nothing cheerful on it [the app]. And maybe it’s just not cheerful at all [the disease]. But you don’t come here [in the app] for cheerfulness, right?” (premanifest HDGEC) “And I kind of think that if you’re a gene carrier and you go to health [topic on the app], and you see somewhere, or under symptoms, ‘I have swallowing problems’, then you think ‘oh, am I going to get that in the future? That doesn’t make me happy’. Whereas if you already have them [the swallowing problems], and that’s also disease awareness, then it’s very different.” (HCP) Two HCPs suggested rephrasing certain sentences to avoid the first-person perspective (e.g., ‘Sleeping difficulties’ instead of ‘I experience sleeping difficulties’) to soften its directive tone. Similarly, about half of participants (only HCPs and partners) perceived the interactive walk-in hour feature on the homepage as too prominent, particularly for those who choose not to use it. Others did not comment on the directiveness of the feature or found it inviting, thereby lowering the threshold to healthcare. One premanifest HDGEC specifically suggested to give the appointment booking tool a more prominent place on the app as it was somewhat difficult to find. Overall, the majority of HDGECs and premanifest partners were positive about the usability and user friendliness of the interactive features. Regarding the app’s functionality, some usability issues were mentioned primarily by HCPs and some partners. They pointed out that certain functionalities were not working properly yet, such as hyperlinks, the app’s search function, and the location filter for finding offline peer group meetings. See Table 2 for an overview of the most frequently mentioned suggestions for the app’s improvement. 2.2.2 Theme 2: Navigating the Huntington Support App: well-organized and accessible, with room for stronger integration In terms of the app’s navigation, most HDGECs and partners found the app well-organized and easy to navigate. About half of them preferred to use the search bar to find information and liked that information was tailored to specific groups (i.e., being an HDGEC or a partner/family member). All participants were particularly positive about the layered presentation of information and the flexibility to expand relevant sections if needed: “I think it’s quite nice, that short summary. They [HDGECs], when it comes to attention and concentration, that’s one of the early things that become difficult. So, having a brief text that summarizes it allows you to decide whether to continue or not, without having to read through long passages.” (HCP) Positive feedback about the app’s structure was also observed in the quantitative SUS data, as all participants but one HCP found the app convenient to navigate (see Supplementary Figure 1 item 8 in the Supplementary files). A manifest HDGEC and partner expressed their appreciation for the app’s information display, emphasizing the flexibility to selectively choose content to engage with. However, some participants were less pleased with having to click multiple times to access specific content. In terms of redirecting to other sites for more information about certain topics, participants also had differing views. Some found it convenient to have all the information on the app itself, whereas others liked that the app provides the key information and redirects to other sites for more detail about specific topics: “It’s indeed, I find it nice that everything is written quite concisely. But that there might indeed be the option to click through to dive deeper into everything.” (partner of a premanifest HDGEC) Most HCPs and one manifest HDGECs reported some (anticipated) difficulties with navigating back to the app after opening a redirection link, whereas the other HDGECs and partners did not report any difficulties. Moreover, some difficulties were reported with navigating back to main topics within the app itself. These issues with the integration of various functions were also apparent in the SUS item score (see Figure 2 item 5). About one-third of participants reported that the app’s functions were not well integrated (mostly expressed by HCPs; see Supplementary Figure 1 item 5 in the Supplementary files), while the remaining two-third of participants found the integration satisfactory. During the navigation tasks, some HDGECs and partners encountered challenges in navigating to specific topics, often due to differing expectations of the location or confusion between the walk-in hour and appointment booking tool. However, most of the navigation tasks were easy for them to perform. Several recommendations to improve the app’s intuitiveness/navigation have been suggested by participants, which are displayed in Table 2. 2.2.3 Theme 3: Future potential of the Huntington Support App: easy access to care, broad applicability, and practical considerations Many participants (9 out of 12) mentioned that they would recommend the app to others, including HDGECs and their partners, as well as others in the community to aid in understanding the disease: “I think for clients in the early stages and informal caregivers, loved ones, who are looking for information. It might be handy for them too, you get told that you have or will get it [the disease]. People don’t understand it well because no one knows the disease. Then you can say, ‘Yeah, there’s this app, I have it too. You might download it.’ That way, people can read it at their own pace afterwards, and the client doesn’t have to explain everything because I think that’s usually difficult. It [the app] can provide a bit of support for what social work already does." (HCP) Additional target audiences mentioned included HDGECs without a family history of HD, those who have limited to no contact with an HD center, and individuals with early-onset HD. The majority of HCPs and all partners indicated that the app is best suitable for those who are in the early and mid-phases of HD when people have mild to moderate symptoms. Some HCPs and one manifest HDGECs indicated that using and understanding the content of the app would be too difficult in the advanced stages of HD: “The app is not suitable for people who are really in the advanced stages. I think those [advanced patients] are (A) already so experienced that they don't benefit from the content anymore, and B) in practice, their motor skills are significantly affected such that using an app would be very difficult, both on a phone and laptop. They have difficulty with their fine motor skills and experience significant chorea, which makes it challenging for them to operate the app effectively. But I don't think that's a problem. I also don't think you should have a 'one size fits all', so to speak. I think if you focus on people in the early and somewhat advanced stages [of the disease], you're already well on your way." (HCP) Two manifest HDGECs and a manifest partner pointed out that once the disease has progressed, help is likely needed when navigating the app: "And if, as a patient, you can no longer operate it [the app] yourself, then the caregiver will have to do it for you." (manifest HDGEC) Despite the variety in digital skills reported (see Table 1), none of the participants expressed a need for assistance in using the HSA (see Figure 2, item 4). Most participants (92%) stated that prior knowledge is not necessary to use the HSA (Figure 2, item 10), and all participants but two HCPs found the HSA easy to learn (see SUS item 7 in Supplementary Figure 1 in the Supplementary files). All participants (except for one HDGEC) reported that they would use the HSA frequently (Figure 2, item 1). This was also reflected in the qualitative data, as all participants mentioned that they would (occasionally) use the app and its interactive features. Participants mentioned that such features lower the threshold to care by increasing easy access and reducing travel time: "Traveling takes a lot of time. And it's not always necessary, I think, to speak to each other in person. So, I would definitely make use of that [the interactive features]." (partner of a premanifest HDGEC) Only one premanifest HDGEC acknowledged not wanting to use the app at this time, as she preferred to distance herself from the disease for the time being: "Some people have a strong need for information, so they spend the whole day searching the internet and doing all sorts of things. And I really don't want to, to know. Because right now, I don't want it [the disease] to already control my life." (premanifest HDGEC) For this participant at this point in her life, the information in the app appeared too confronting. However, she mentioned that she might want to use the HSA in the future if needed. None of the HCPs reported to use any eHealth services in current HD care. They indicated that they would not use the HSA to find information on HD for themselves, as they are already familiar with most of it. Instead, they would use the HSA to refer HDGECs and their partners for information or for themselves to stay updated on expert centers and colleagues. Regarding the use of the interactive features, most HDGECs and partners reported to be willing to use these functions in the future, whereas most HCPs expressed their reservations especially about using the appointment booking tool as they are concerned about its fit with the current care pathways. They prefer the current with a case manager/social worker as the initial point of contact who refers patients to the appropriate HCPs if necessary. If HDGECs or other app visitors were to schedule appointments directly with them, they are unsure whether they can provide the necessary assistance or if the correct HCP has been selected. Other practical considerations for future use of an appointment booking tool mentioned across participant groups included time management, difficulties around reimbursement by healthcare insurance, and cost-effectiveness. HCPs showed more favorable attitudes towards the online walk-in hour, with many expressing willingness to host such sessions if they meet the general need. Some important requirements for implementation of the walk-in hour were mentioned by HCPs, such as security and privacy of data. Table 2. Part I - Think-aloud study: Design and content recommendations for the HSA Personalization • Present information / interactive elements that fits the needs of a specific user • Add filters so that users can find centers and HCPs that are geographically closest to them Usability • Ensure that all functionalities work properly (e.g., hyperlinks, search bar, filters) • Differentiate more clearly between the information and contact components of the app, and improve findability of the latter • Add explanation to certain terms (e.g., ‘telehealth’, ‘pen pal’, ‘what to expect’) and avoid abbreviations Navigation • Add navigation aids for internal navigation (e.g., category tiles, decision tree) and external redirection (e.g., back button, instructions on how to navigate back) • Add a dropdown menu for easier navigation Comprehensiveness • Increase representation of other HD centers and HCPs across the country • Update current content, list of HCPs, meetings, and news items • Extent the app with more content (including topics as euthanasia, intimacy, financial and practical aspects that come with living with HD) and provide more detailed information on certain topics either within the app or via external references Additional potential features beyond the app’s current scope • Keep track of personal details (logbook, favorites, current location) HCPs: Health care providers; HD: Huntington’s disease. [INSERT FIGURE 2 HERE] Figure 2. Part I - Think-aloud study: Participants’ responses to the positive and negative items of the SUS SUS: System Usability Scale; HSA: Huntington Support App. 3 Part II – Feasibility study 3.1 Methods 3.1.1 Participants and recruitment As the Huntington Support App (HSA) was made openly accessible to everyone in the HD community for the feasibility study, eligibility criteria were not applicable. The HSA was advertised via HD-related social media platforms and through 9 out of 10 approached outpatient clinics at Huntington (expertise) centers in the Netherlands. One approached center chose not to distribute advertisement flyers as, according to them, the app’s content was not representative of all HD centers. Everyone who was interested could voluntarily decide to visit the web app via a link/QR-code that was provided in the advertisement. 3.1.2 Study design and procedure An observational feasibility study was designed to evaluate the usability of the minimal viable product (MVP) of the HSA over a six-week period, with participant feedback rounds at two time points. The MVP, a first version of a digital health product intended for real-world evaluation, was developed based on feedback from the think-aloud study (Part I). It was openly accessible without a pin code and featured a functional search bar. The final content was reviewed by HD experts from Huntington Expertise Center Topaz Overduin (i.e., by a psychiatrist, neuropsychologist, speech therapist, case manager, physiotherapist, dietician, and social worker). It also included additional content, drafted based on input from the think-aloud sessions and subsequently reviewed by the experts, covering topics such as genetic risk for children, the euthanasia process, and a comprehensive list of all Dutch Huntington expertise centers. The MVP of the HSA was launched publicly on February 1 st , 2023. During four weeks in February, the app included an online walk-in hour every Monday with a social worker with expertise in HD to ask HD-related questions. The social worker was asked to collect information on who visited the walk-in hour (i.e., a patient, partner, health care provider, family member or other) and for what reason. Visitors were able to leave their anonymous feedback on the HSA by filling in a questionnaire through the corresponding links/QR-codes in the web-app or advertisement. Visitors could voluntarily decide to leave feedback, and if they chose not to, the app remained fully accessible to them. The feedback questionnaire included quantitative items on sociodemographics (i.e., age categories, gender, living situation, relationship to HD), digital skills, device used, satisfaction, future usage, and the System Usability Scale (SUS) (see Part I). At the end of the questionnaire, respondents could provide qualitative responses, offering suggestions for improving the HSA in open-ended response fields. Additionally, participants were informed that they could evaluate the web-app and its usage again after four weeks by contacting the research team to receive the follow-up questionnaire. However, none of the participants reached out, resulting in no follow-up questionnaires being completed [1] . As the feasibility study was not originally planned within the larger HD-eHelp study, an amendment to the original study protocol (file number: N20.013) (35) was submitted. The amendment was approved by the scientific overview committee of the Department of Public Health and Primary care of Leiden University Medical Center. Given the low-threshold, non-intrusive nature and voluntary basis of the feasibility study, the amendment did not require additional approval from the medical ethics committee. A brief explanation on the study and data anonymity was included in the online questionnaire. Digital informed consent was signed by all participants prior to the start of the questionnaires. 3.1.3 Data analyses Descriptive statistics were used to calculate the mean scores and frequencies/proportions on respondents’ characteristics (sociodemographics, digital skills, device used, satisfaction, future usage) as well as on the SUS scores (IBM SPSS Statistics, version 29.0). For the SUS, the same procedure was followed as previously described in Part I. Responses to the open-ended questions, were extracted and key points were highlighted, with a similar approach intended for the documented reasons for visiting the walk-in hour. 3.2 Results In total, 13 respondents completed the initial feedback questionnaire on the HSA, and none of them reached out to receive the four-week follow-up questionnaire. Respondents consisted of a mixed group from the HD community, including HDGECs (n = 4), partners of manifest HDGECs (n = 3), HCPs (n = 3), family members of manifest HDGECs (n = 2), and other (i.e., person with no relationship to HD, n = 1). Of the HDGECs, two reported to be in the premanifest stage whereas the other two reported to have manifest disease. HCPs were all (case)managers HD. See Table 3 for an overview of the other collected characteristics per respondent group. Table 3. Part II – Feasibility study: Respondents’ characteristics and usability outcomes of the HSA HDGECs (n = 4) Partners (n = 3) HCPs (n = 3) Friends/family (n = 2) Other (n = 1) Respondents’ characteristics Age range category (n (%)) 1940-1960 - 2 (67) - 1 (50) - 1961-1980 3 (75) 1 (33) 3 (100) 1 (50) - 1981-2000 1 (25) - - - 1 (100) Gender (n (%)) Male 2 (50) 1 (33) - 1 (50) - Female 2 (50) 2 (67) 3 (100) 1 (50) 1 (100) Previous involvement app development (n (%)) Yes 1 (25) 3 (100) - - - No 3 (75) - 3 (100) 2 (100) 1 (100) Use of health apps (n (%)) Often - 1 (33) - - - Sometimes 1 (25) 1 (33) 2 (67) 1 (50) 1 (100) Rarely 1 (25) - 1 (33) - - Never 2 (50) 1 (33) - 1 (50) - Self-reported digital skills (n (%)) (Very) experienced 3 (75) 3 (100) 2 (67) 2 (100) 1 (100) Neither experienced or Inexperienced 1 (25) - 1 (33) - - (Very) inexperienced - - - - - Usability outcomes Device used (n (%)) Smartphone 2 (50) 2 (67) - 2 (100) - Computer/laptop 1 (25) 1 (33) 3 (100) - 1 (100) Tablet 1 (25) - - - - Satisfaction with app (mean; range) (0: not satisfied to 10: very satisfied) 7 (6-8) 8 (6-10) 6 (5-7) 9 (8-9) 8 (8-8) Would use app in future (Strongly) agree 3 (75) 3 (100) 1 (33) 2 (100) 1 (100) Neutral 1 (25) - 1 (33) - - (Strongly) disagree - - 1 (33) - - Intended use of app To gather information on HD 2 (50) 3 (100) - - 1 (100) To stay up to date with latest news 2 (50) - - 2 (100) - To find peer support - - - - - To attend the online walk-in hour - - - - - Other - - 3 (100) - - SUS total score (mean; range) 68 (50-95) 71 (58-90) 53 (48-63) 74 (65-83) 78 (78-78) HDGECs: Huntington’s disease gene expansion carriers; N: number of participants; HCPs: Health care providers; SUS: System Usability Scale. Numbers and percentages are rounded to the nearest whole number. The participant in the ‘Other’ group indicated that she did not have any relationship to HD, but expressed a desire to provide feedback on the app. 3.2.1 Usability of the HSA in practice The overall SUS scores across all groups (i.e., 68.8) is comparable to the standard benchmark of 68 commonly observed in usability testing for healthcare apps (42, 43). In particular, partners and friends/family members were very satisfied with the app (rating of 8 and 9 out of 10, respectively) and its usability (i.e., 71 and 74 respectively). HDGECs were also satisfied with the app and its usability (see Table 3). HCPs, however, reported moderate satisfaction with the HSA and their overall usability score (i.e., 53) was below the benchmark. The majority of HDGECs, and all partners and friends/family members reported to be wanting to use the HSA in the future. They would like to use the HSA either for information purposes or to stay updated on the latest news on scientific research. The latter was also what family/friends reported they intend to use the HSA for. Two HCPs indicated that they would use the app for professional purposes; one HCPs reported not to intend using the HSA. With regard to the online walk-in hour, most HDGECs and one partner expressed occasional interest in using it in the future. The other HDGEC, two partners and family/friends indicated that they would rarely use it, whereas all HCPs reported that they would not use it in the future. To note, none of the app’s visitors attended the online walk-in hour sessions with the social worker. When looking at the SUS scores across all groups (Figure 3), participants were mostly neutral to (very) positive. A slight majority of all respondents found the HSA convenient and easy to learn, reported feeling confident using the app, and expressed interest in using it frequently (item 8, 7, 9 and 1 respectively in Figure 3). Most respondents indicated that using the HSA did not require assistance or prior knowledge (item 4 and 10 in Figure 3). However, opinions regarding the ease of use, complexity, consistency, and integration of the HSA’s functions varied among respondents, with approximately half of participants expressing neutral responses (item 3, 2, 6 and 5 respectively in Figure 3). When looking at the SUS item responses per subgroup (see Supplementary Figure 2 in Supplementary files), most of the neutral responses were observed among the HCPs. [INSERT FIGURE 3 HERE] Figure 3. Part II - Feasibility study: HSA visitors’ responses to the positive and negative items of the SUS SUS: System Usability Scale; HSA: Huntington Support App. 3.2.2 Strengths and suggestions for improving the HSA Respondents highlighted several strengths of the HSA in the open-ended response fields of the questionnaire. Participants praised the app for its clarity and user-friendly design, making it easy to navigate and understand. They valued the app’s focus on providing accessible and relatable information. The comprehensive range of topics covered by the HSA was another significant strength noted by respondents, as according to some, it bundles all relevant information on HD. Furthermore, the app’s ability to connect users to existing resources and services was seen as beneficial, facilitating access to relevant support. In addition, respondents provided suggestions for improving the HSA in the future. Some participants suggested including a comprehensive list of all HD experts available in the Netherlands, e.g., case managers were currently not listed in the app. Additionally, participants emphasized the need to increase the representation and information from other HD centers across the country, with some noting that the current focus was too concentrated on the center involved in co-designing the app. Some participants also suggested expanding the app with more detailed information on HD and to cover more topics relevant to users. Lastly, one respondent recommended making the list of HD expertise centers more visually appealing, such as by integrating them into a country-wide map display. 4 Discussion This study provided valuable insights into the usability and feasibility of the Huntington Support App (HSA), a web-app intended to offer clear and reliable information on Huntington’s disease (HD), HD-related news and events and interactive contact possibilities with health care providers (HCPs) (36). Building on prior research outlining the co-creation process of the HSA prototype (36), the present study employed a mixed-methods approach to gain an in-depth evaluation of the HSA in practice. By conducting a ‘think-aloud’ study and subsequent feasibility evaluation of the HSA, we were able to identify several key themes and outcomes that have implications for the future refinements of the HSA and the development of similar digital health services for rare neurodegenerative diseases such as HD. 4.1 Usability and feasibility of the Huntington Support App Overall, the HSA was well-received in both studies. Especially during the think-aloud study, participants were very positive about the app’s usability with their scores on the usability questionnaire far exceeding the benchmark found in usability testing for healthcare apps (42, 43). In the feasibility study, the usability of the HSA was rated lower, yet remained slightly above the standard benchmark for healthcare apps (42, 43). Next to positive evaluations, more neutral responses on the usability of the HSA were observed during the feasibility study. About half of the participants reported being neither satisfied nor dissatisfied with the different usability aspects of the HSA. This may be due to the fully anonymous nature of the feasibility study. Participants had the flexibility to review the app and complete the questionnaire at their convenience without a researcher present, unlike in the think-aloud study. Although it was explicitly stated during the think-aloud session that the researcher was not involved in the app’s development and participants completed the questionnaire independently, the presence of a researcher could have influenced their responses. When looking at each participant group separately, considerable variations in how the HSA was evaluated were observed. HD gene expansion carriers (HDGECs) and partners reported very positive experiences with the HSA in both studies. Although HCPs were generally positive, their views were less favorable compared to those of HDGECs, partners, and family/friends. This is not surprising, as the app’s development primarily focused on the needs of HDGECs and their partners (36). HCPs were involved to a lesser extent throughout the app’s development and were not considered the primary target group (36), likely leading to their needs not being fully addressed within the app. Moreover, HCPs may feel more compelled to express concerns and advocate for improvements when evaluating (future) care services given their professional responsibilities and familiarity with clinical workflows. Furthermore, HCPs may have evaluated the app from the perspective of HDGECs rather than their own, which may have led them to perceive the HSA as more difficult for HDGECs to use. For instance, about half of the HCPs in the think-aloud study reported that they did not find the HSA easy to use, easy to learn or straightforward, whereas HDGECs themselves did not share these concerns. HDGECs, partners, friends/family members, as well as someone with no connection to HD were (very) positive about the app’s usability. 4.2 The importance of human-centered design Participatory design is highly recommended for aligning eHealth solutions with end-users’ needs (5, 17-19) and has shown success in other neurodegenerative and neurological diseases (e.g., dementia, Parkinson’s disease, and multiple sclerosis) (12-14). However, despite its recommendations for rare diseases (15, 16) and its potential for HD, research on participatory design and co-creation of eHealth for HD is limited (30-32, 34, 46, 47). Our findings illustrate the value of adopting a participatory human-centered design (HCD) approach for a rare and complex disease like HD. Throughout the app’s development, HDGECs and their partners were prioritized as the primary end-user groups (36). The positive feedback from especially these groups in the present study underlines the effectiveness of HCD in developing an app that aligns with the needs and expectations of end-users. Participants in the present study were newly recruited and had not been involved in the app’s development. This shows that the app not only fulfills needs from the participants that co-designed the app, but addresses needs commonly present among those affected by HD. Diverse representation in the co-design process of the end-user group, early and consistent involvement throughout the process, and rigorous analysis of what participants say, seem important factors for developing an app that closely addresses the needs of the target audience. Moreover, involving end-users from the beginning not only ensures that the solution meets their needs but also likely enhances its adoption and uptake later on (5, 17, 18, 48-52). The mixed findings regarding the (future) use of the interactive features of the HSA (i.e., appointment booking tool and walk-in hour) further highlights the importance of the HCD design process. While most participants (HDGECs and partners) expressed willingness to use these features in the future during both the development phase (36) and think-aloud study, none of the app’s visitors attended the online walk-in sessions scheduled during the feasibility study. This lack of attendance could be attributed to the overall low visitor rate and/or the study’s short timeframe, with only one scheduled walk-in hour a week, of which the timing (i.e., Monday morning) may not have been convenient for everyone. Moreover, the interactive features may not have closely aligned with the needs of HDGECs and partners, as these features did not emerge from their initial needs assessment across all participating countries (36). The need for these functionalities was mainly expressed by HCPs from the other countries (36). Since this study was conducted in the relatively small Dutch population, it is important to explore how the HSA is evaluated in the other countries. They face greater challenges in bridging geographic distances (36) and have different healthcare organizations with relatively fewer HD services available, making these interactive features more essential. This underscores the importance of considering and designing for different target groups, as their needs and requirements can vary greatly depending on country and care context. Our HCD approach allowed us to develop a customizable app tailored to the diverse and complex needs of various participant groups. By aligning with these needs, the HSA was positively evaluated not only by the primary end-users, but also by others who were indirectly or not involved in the app’s development, such as family members and friends. While it is not possible to meet all needs and preferences with a single eHealth solution (36), the HSA demonstrates that, through a participatory design approach, we were able to address a wide range of expressed needs and wishes. Our study further shows that such an approach allows for the development of an app that can address both current and emerging needs of different groups affected by HD. The HSA was designed to be easily adopted and tailored to specific target groups and different healthcare contexts/language regions, with the possibility to incorporate new features if new needs emerge (36). 4.3 Study’s limitations and strengths Although the findings of our study are promising, they should be interpreted with several limitations in mind. First, the think-aloud study included a relatively well-functioning group of participants in terms of disease characteristics and digital literacy which may have influenced the results toward more favorable usability outcomes. Second, the small sample size in both studies limits the generalizability of our findings to the larger HD community. At the same time, it reflects the challenges of studying a rare condition like HD. We faced some recruitment challenges in the think-aloud study that necessitated broadening our initial inclusion criteria. Additionally, the recruitment strategy for the feasibility study involved leaving flyers at HD expertise centers, and therefore lacked active advertising and tracking. This approach potentially limited our sample size and response rate. As a result, we had little insight into how many individuals noticed the flyers and subsequently viewed and used the HSA during that period. This made it difficult to assess the extent of bias among survey respondents. While our approach was sufficient for collecting initial user feedback from a larger group in the HD community, a more proactive strategy for promoting the flyers and monitoring its distribution, as well as a longer time frame for the study, could have enhanced participant engagement. Despite these limitations, some of the study’s strengths are worth mentioning as well. The mixed-methods approach in the think-aloud study provided both a comprehensive overview of the app’s usability and detailed insights into specific areas for improvement. The quantitative data corroborated the detailed qualitative feedback from participants, strengthening the overall reliability of our findings on the app’s usability. Furthermore, by recruiting new participants who had not been part of earlier phases of the app’s development process, we were able to assess the effectiveness of our HCD approach and identify additional areas for improvement. Lastly, conducting both a think-aloud and feasibility study added significant value. The think-aloud study allowed us to address major issues, such as navigation and content improvements, before the app was publicly tested in the feasibility study. With these key issues already resolved, the feasibility study could provide a more accurate assessment of the app’s overall usability. This broader rollout provided insights into how the app was received by user groups not previously involved in its development, including family, friends, and other stakeholders. It also offered a first understanding of how the app performed in a real world-setting, highlighting directions for future refinement and use. 4.4 Implications for future research and practice This study identified several key areas for improving the HSA, such as navigation challenges, content suggestions and its tone, and differing user needs regarding some of the app’s features. Future refinements of the HSA should address these, while also examining the use and effectiveness of the interactive features (i.e., walk-in hour and appointment booking tool) more thoroughly. We gathered feedback on participants’ willingness to use the appointment booking tool in the future. While testing its actual use was beyond the scope of this study due to necessary changes in the clinical workflow of all centers, this presents a valuable area for future research. For the walk-in hour, it would be beneficial to explore whether offering sessions multiple times a week could enhance its accessibility and user engagement. Moreover, future research should evaluate the app’s usability in other participating countries to account for the varying needs across end-user groups and countries. Given that the app is modular in its design (36), it would be valuable to explore which components are most relevant in specific settings (e.g., country, expertise center). This would allow certain features to be enabled or disabled, tailoring the app to those particular contexts. For instance, while the interactive features may be less relevant in the Dutch context, they could be highly valued in the other participating countries. Moreover, involving a broader range of expertise centers, stakeholders, and a diverse group of HDGECs with varying diseases stages and digital skills, will further ensure the app meets a wider range of user needs. To gain a more comprehensive understanding of the HSA and its implications, future research should address the app’s impact on practical outcomes for HDGECs and their partners (e.g., information provision, travel time, accessibility to care, navigating care options) as well as on care outcomes (e.g., quality of care, knowledge exchange, cost-benefit). With the app’s potential to enhance current care practices, future studies should explore barriers to implementation (e.g., app’s representation of expertise centers across countries, staff availability) and identify effective implementation strategies (e.g., pinpointing key stakeholders and the optimal stages in the patient journey for the app’s introduction). 5 Conclusions The current study provided initial insights into the usability and feasibility of the Huntington Support App (HSA) within both a structured and real-world setting in the Netherlands. While there was considerable variation in how different participant groups evaluated the HSA, it was generally well-received by all. Future efforts should focus on addressing the identified areas for improvement and other key themes to refine the app and its implementation. Our findings highlight the importance and feasibility of a HCD approach in developing digital tools for a disease as rare and complex as HD. By aligning closely with end-users’ needs, the HSA holds promise in addressing challenges related to time, distance and cost in current HD care (19). It has the potential to enhance the quality of life for those affected by HD by improving access to care and reliable information. Given the importance of these factors in the management of long-term complex conditions (53), our approach and findings could offer valuable insights for other initiatives aimed at developing eHealth services to support individuals affected by other rare neurodegenerative diseases as well. Declarations 6.1 Ethics approval and consent to participate The first study was part of a larger project (HEALTHE-RND HD-eHelp study) that was cleared for ethics by the medical research ethics committee of Leiden Den Haag Delft in the Netherlands (file number: N20.013). The amendment for the second study did not require approval from the medical ethics committee, however, it was reviewed and approved by the scientific overview committee of the Department of Public Health and Primary care of Leiden University Medical Center. (Digital) informed consent was signed by all participants prior to the start of the studies. 6.2 Consent for publication Not applicable. 6.3 Availability of data and materials The data will not be shared publicly to protect the anonymity and privacy of participants due to the rarity of HD. Pseudonymized data supporting the findings of this study can be accessed upon reasonable request from the corresponding author (PL). 6.4 Competing interests Susanne T. de Bot: Leiden University Medical Center receives funding from the European Huntington’s Disease Network (EHDN) and Cure HD Initiative (CHDI), is involved in the EU Horizon 2020 project: Innovative Medicines Initiative (IMI) 2 (IDEA_FAST), and takes part in clinical trials sponsored by PRILENIA, PTC Therapeutics, WAVE, and VICO Therapeutics. Susanne T. de Bot is member of the HD Expert Advisory Panel for the PTC518 phase 3 program (PTC Therapeutics). None of these sponsors were involved in the design, execution, interpretation, or writing of this study. The other authors have no conflicts of interest to declare. 6.5 Funding This research was supported by the EU Joint Programme Neurodegenerative Disease Research (JPND) (grant number: 01ED1903) and locally by the Netherlands Organisation for Health Research and Development (ZonMw) (project number: 733051085). The funders had no involvement in the study’s design, data collection, analysis, interpretation, decision to publish, or manuscript preparation. 6.6 Authors’ contributions Funding acquisition: EM, NC, and the HEALTHE-RND consortium; Conceptualization: EM, AH, NV, LS and PL; Data collection: LS. Formal analysis: LS and PL. Writing – original draft: PL. Writing – review & editing: LS, AH, EM, ED, SB, NC, NV, and PL. All authors read and approved the final manuscript. 6.7 Acknowledgments The authors would like to thank all participants in the think aloud study for their active participation and valuable contributions. Their feedback was greatly appreciated. We also wish to thank all app visitors during the feasibility study who reviewed the app and shared their feedback. Additionally, we want to express our gratitude to the HCPs who checked the app’s content to ensure its clinical accuracy. We also like to acknowledge the outpatient clinics of several HD (expertise) centers in the Netherlands for their help in distributing flyers to promote the Huntington Support App for the feasibility study: Land van Horne, Archipel, Willem Drees-Oostpoort, Noorderbreedte, Topaz Overduin, Kloosterhoeve and the university medical centers of Maastricht, Groningen and Leiden. 6.7.1 Collaborating author names HEALTHE-RND consortium The Netherlands: Niels H. Chavannes, Eline Meijer, Anne- Wil Heemskerk, Erik van Duijn, Susanne T. de Bot, Pearl J. C. van Lonkhuizen, Niko Vegt, Leanne Slutter; Germany: G. Bernhard Landwehrmeyer, Alzbeta Mühlbäck, Wiebke Frank, Franziska Steck; Czech Republic: Jiří Klempíř, Romama Konvalinková, Eva Bezuchová, Kristýna Dolečková, Olga Klempířová, Jan Roth, Olga Ulmanová; Italy: Ferdinando Squitieri, Sabrina Maffi, Giulia Giancaterino, Chiara Di Giorgio, Barbara D’Alessio, Melissa Casella; Ireland: Jennifer Hoblyn, Muthukumaran Thangaramanujam, Tom Burke, Emer O’Malley; United Kingdom: Stephen McKenna, Ian McKenna, Jeanette Thorpe, Ellie Johnstone, Isobel Spray, Mariusz Grzeda, Ramona Moldovan, Peter Foley, Jacqueline Kerr. References Ma Y, Zhao C, Zhao Y, Lu J, Jiang H, Cao Y, et al. Telemedicine application in patients with chronic disease: a systematic review and meta-analysis. BMC Med Inform Decis Mak. 2022;22(1):105; doi:10.1186/s12911-022-01845-2. Renzi E, Baccolini V, Migliara G, De Vito C, Gasperini G, Cianciulli A, et al. The Impact of eHealth Interventions on the Improvement of Self-Care in Chronic Patients: An Overview of Systematic Reviews. Life (Basel). 2022;12(8); doi:10.3390/life12081253. Eysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20; doi:10.2196/jmir.3.2.e20. Dorsey ER, Glidden AM, Holloway MR, Birbeck GL, Schwamm LH. Teleneurology and mobile technologies: the future of neurological care. Nat Rev Neurol. 2018;14(5):285-97; doi:10.1038/nrneurol.2018.31. van der Kleij R, Kasteleyn MJ, Meijer E, Bonten TN, Houwink EJF, Teichert M, et al. SERIES: eHealth in primary care. Part 1: Concepts, conditions and challenges. Eur J Gen Pract. 2019;25(4):179-89; doi:10.1080/13814788.2019.1658190. Chirra M, Marsili L, Wattley L, Sokol LL, Keeling E, Maule S, et al. Telemedicine in Neurological Disorders: Opportunities and Challenges. Telemed J E Health. 2019;25(7):541-50; doi:10.1089/tmj.2018.0101. Colon-Semenza C, Latham NK, Quintiliani LM, Ellis TD. Peer Coaching Through mHealth Targeting Physical Activity in People With Parkinson Disease: Feasibility Study. JMIR Mhealth Uhealth. 2018;6(2):e42; doi:10.2196/mhealth.8074. Espay AJ, Bonato P, Nahab FB, Maetzler W, Dean JM, Klucken J, et al. Technology in Parkinson's disease: Challenges and opportunities. Mov Disord. 2016;31(9):1272-82; doi:10.1002/mds.26642. Lakshminarayana R, Wang D, Burn D, Chaudhuri KR, Galtrey C, Guzman NV, et al. Using a smartphone-based self-management platform to support medication adherence and clinical consultation in Parkinson's disease. NPJ Parkinsons Dis. 2017;3:2; doi:10.1038/s41531-016-0003-z. Nissen RM, Hersch G, Tietze M, Chang PJ. Persons With Dementia and Their Caregivers' Perceptions About Occupational Therapy and Telehealth: A Qualitative Descriptive Study. Home Healthc Now. 2018;36(6):369-78; doi:10.1097/NHH.0000000000000697. Rathnayake S, Moyle W, Jones C, Calleja P. mHealth applications as an educational and supportive resource for family carers of people with dementia: An integrative review. Dementia (London). 2019;18(7-8):3091-112; doi:10.1177/1471301218768903. Monje MHG, Grosjean S, Srp M, Antunes L, Bouca-Machado R, Cacho R, et al. Co-Designing Digital Technologies for Improving Clinical Care in People with Parkinson's Disease: What Did We Learn? Sensors (Basel). 2023;23(10); doi:10.3390/s23104957. Russell RD, Black LJ, Purdue J, Daly A, Begley A. A collaborative approach to designing an online nutrition education program for people with multiple sclerosis. Disabil Rehabil. 2024;46(5):947-56; doi:10.1080/09638288.2023.2186499. Wang G, Marradi C, Albayrak A, van der Cammen TJM. Co-designing with people with dementia: A scoping review of involving people with dementia in design research. Maturitas. 2019;127:55-63; doi:10.1016/j.maturitas.2019.06.003. Hatem S, Long JC, Best S, Fehlberg Z, Nic Giolla Easpaig B, Braithwaite J. Mobile Apps for People With Rare Diseases: Review and Quality Assessment Using Mobile App Rating Scale. J Med Internet Res. 2022;24(7):e36691; doi:10.2196/36691. Long JC, Best S, Nic Giolla Easpaig B, Hatem S, Fehlberg Z, Christodoulou J, et al. Needs of people with rare diseases that can be supported by electronic resources: a scoping review. BMJ Open. 2022;12(9):e060394; doi:10.1136/bmjopen-2021-060394. Greenhalgh T, A'Court C, Shaw S. Understanding heart failure; explaining telehealth - a hermeneutic systematic review. BMC Cardiovasc Disord. 2017;17(1):156; doi:10.1186/s12872-017-0594-2. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111; doi:10.2196/jmir.1672. Whitehead L, Seaton P. The Effectiveness of Self-Management Mobile Phone and Tablet Apps in Long-term Condition Management: A Systematic Review. J Med Internet Res. 2016;18(5):e97; doi:10.2196/jmir.4883. Bates GP, Dorsey R, Gusella JF, Hayden MR, Kay C, Leavitt BR, et al. Huntington disease. Nat Rev Dis Primers. 2015;1:15005; doi:10.1038/nrdp.2015.5. Bates GP, Tabrizi SJ, Jones L. Huntington’s Disease. 4 ed. New York, NY: Oxford University Press; 2014. McColgan P, Tabrizi SJ. Huntington's disease: a clinical review. Eur J Neurol. 2018;25(1):24-34; doi:10.1111/ene.13413. Roos RA. Huntington's disease: a clinical review. Orphanet J Rare Dis. 2010;5:40; doi:10.1186/1750-1172-5-40. Reilmann R, Leavitt BR, Ross CA. Diagnostic criteria for Huntington's disease based on natural history. Mov Disord. 2014;29(11):1335-41; doi:10.1002/mds.26011. Ross CA, Reilmann R, Cardoso F, McCusker EA, Testa CM, Stout JC, et al. Movement Disorder Society Task Force Viewpoint: Huntington's Disease Diagnostic Categories. Mov Disord Clin Pract. 2019;6(7):541-6; doi:10.1002/mdc3.12808. van Lonkhuizen PJC, Frank W, Heemskerk AW, van Duijn E, de Bot ST, Muhlback A, et al. Quality of life, health-related quality of life, and associated factors in Huntington's disease: a systematic review. J Neurol. 2023;270(5):2416-37; doi:10.1007/s00415-022-11551-8. Anderson KE, Griffin J, Kinel A, Shaikh AR, Olofintuyi T, Ramirez S, et al. Quality of Care for Huntington's Disease in the United States: Findings from a National Survey of Patients and Caregivers. J Huntingtons Dis. 2019;8(4):509-19; doi:10.3233/JHD-190380. Chase CL, Yashar BM, Swope C, Albin RL, Uhlmann WR. Searching for Answers: Information-Seeking by Young People At-Risk for Huntington's Disease. J Huntingtons Dis. 2022;11(3):337-46; doi:10.3233/JHD-210523. de Man J, de Jager L, Kleiterp J, Engelhard D, Achterberg W. K10 E-health support for 841 community residing HD patients: report of a pilot-project. Journal of Neurology, Neurosurgery & Psychiatry. 2016;87:A82-A3. Bull MT, Darwin K, Venkataraman V, Wagner J, Beck CA, Dorsey ER, et al. A pilot study of virtual visits in Huntington disease. J Huntingtons Dis. 2014;3(2):189-95; doi:10.3233/JHD-140102. Frich JC, Rae D, Roxburgh R, Miedzybrodzka ZH, Edmondson M, Pope EB, et al. Health Care Delivery Practices in Huntington's Disease Specialty Clinics: An International Survey. J Huntingtons Dis. 2016;5(2):207-13; doi:10.3233/JHD-160192. Hawkins AK, Creighton S, Ho A, McManus B, Hayden MR. Providing predictive testing for Huntington disease via telehealth: results of a pilot study in British Columbia, Canada. Clin Genet. 2013;84(1):60-4; doi:10.1111/cge.12033. Hawkins Virani AK, Creighton SM, Hayden MR. Developing a comprehensive, effective patient-friendly website to enhance decision making in predictive testing for Huntington disease. Genet Med. 2013;15(6):466-72; doi:10.1038/gim.2012.149. Mohanty D, Schmitt P, Dixon L, Holiday V, Hedera P. Patient and Caregiver Perspectives on Telehealth Use in a Multidisciplinary Huntington's Disease Clinic: A Single-Institution Experience. J Huntingtons Dis. 2022;11(4):415-9; doi:10.3233/JHD-220547. van Lonkhuizen PJC, Vegt NJH, Meijer E, van Duijn E, de Bot ST, Klempir J, et al. Study Protocol for the Development of a European eHealth Platform to Improve Quality of Life in Individuals With Huntington's Disease and Their Partners (HD-eHelp Study): A User-Centered Design Approach. Front Neurol. 2021;12:719460; doi:10.3389/fneur.2021.719460. van Lonkhuizen PJC, Heemskerk AW, Meijer E, van Duijn E, de Bot ST, Klempir J, et al. Development of the Huntington Support App (HD-eHelp study): a human-centered and co-design approach. Front Neurol. 2024. Landwehrmeyer GB, Fitzer-Attas CJ, Giuliano JD, Goncalves N, Anderson KE, Cardoso F, et al. Data Analytics from Enroll-HD, a Global Clinical Research Platform for Huntington's Disease. Mov Disord Clin Pract. 2017;4(2):212-24; doi:10.1002/mdc3.12388. Arsand E, Demiris G. User-centered methods for designing patient-centric self-help tools. Inform Health Soc Care. 2008;33(3):158-69; doi:10.1080/17538150802457562. Brooke J. SUS: a quick and dirty usability scale. Usability Eval Ind. 1995;189:1-6. Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. In: Kurosu, M. (eds) Human Centered Design. . HCD 2009; Berlin2009. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105-12; doi:10.1016/j.nedt.2003.10.001. Hyzy M, Bond R, Mulvenna M, Bai L, Dix A, Leigh S, et al. System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR Mhealth Uhealth. 2022;10(8):e37290; doi:10.2196/37290. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: A scoping review. Int J Med Inform. 2019;126:95-104; doi:10.1016/j.ijmedinf.2019.03.018. Kelly L, Ziebland S, Jenkinson C. Measuring the effects of online health information: Scale validation for the e-Health Impact Questionnaire. Patient Educ Couns. 2015;98(11):1418-24; doi:10.1016/j.pec.2015.06.008. Neijenhuijs KI, van der Hout A, Veldhuijzen E, Scholten-Peeters GGM, van Uden-Kraan CF, Cuijpers P, et al. Translation of the eHealth Impact Questionnaire for a Population of Dutch Electronic Health Users: Validation Study. J Med Internet Res. 2019;21(8):e13408; doi:10.2196/13408. de Man J, de Jager L, Kleiterp J, Engelhard D, Achterberg W. K10 E-health support for community residing HD patients: report of a pilot-project. Journal of Neurology, Neurosurgery & Psychiatry. 2016;87(Suppl 1):A82-A3; doi:10.1136/jnnp-2016-314597.233. Hawkins AK, Creighton SM, Hayden MR. Developing a comprehensive, effective patient-friendly website to enhance decision making in predictive testing for Huntington disease. Genetics in Medicine. 2013;15(6):466-72. Granja C, Janssen W, Johansen MA. Factors Determining the Success and Failure of eHealth Interventions: Systematic Review of the Literature. J Med Internet Res. 2018;20(5):e10235; doi:10.2196/10235. Nijland N, van Gemert-Pijnen J, Boer H, Steehouder MF, Seydel ER. Evaluation of internet-based technology for supporting self-care: problems encountered by patients and caregivers when using self-care applications. J Med Internet Res. 2008;10(2):e13; doi:10.2196/jmir.957. Ruggiano N, Brown EL, Shaw S, Geldmacher D, Clarke P, Hristidis V, et al. The Potential of Information Technology to Navigate Caregiving Systems: Perspectives from Dementia Caregivers. J Gerontol Soc Work. 2018:1-19; doi:10.1080/01634372.2018.1546786. Tinschert P, Jakob R, Barata F, Kramer JN, Kowatsch T. The Potential of Mobile Apps for Improving Asthma Self-Management: A Review of Publicly Available and Well-Adopted Asthma Apps. JMIR Mhealth Uhealth. 2017;5(8):e113; doi:10.2196/mhealth.7177. Shah SG, Robinson I. User involvement in healthcare technology development and assessment: structured literature review. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2006;19(6-7):500-15; doi:10.1108/09526860610687619. Battersby M, Lawn S, Pols R. Conceptualisation of self-management. . In: Kralik D, Paterson B, Coates V, editors. Translating Chronic Illness Research Into Practice Hoboken, NJ: Wiley-Blackwell; 2010. p. 115-201. Footnote [1] The follow-up questionnaire was intended to assess the usability of the app after a period of four weeks, including items on frequency of use, satisfaction with the app, as well as part 2 of the Dutch version of the eHealth Impact Questionnaire (eHIQ) (44, 45). Part 2 of the eHIQ contains 26 validated items on the attitudes of users towards a specific eHealth application (e.g., “I can easily understand the information on the website” or “ The website gives me the confidence to explain my health concerns to others” ), rated on a five-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’ (44). Additional Declarations No competing interests reported. Supplementary Files Supplementaryfile1Thinkaloudprotocol.docx Supplementary files Supplementary file 1: Think-aloud protocol .jpeg Think-aloud protocol Supplementaryfile2FigureSUSitemsperparticipantgroupthinkaloudstudy.jpg Supplementary file 2: Figure SUS items per participant group think-aloud study .jpeg Supplementary Figure 1. Part I – Think-aloud study: proportions of responses to each SUS item per participant group Supplementaryfile3FigureSUSitemsperparticipantgroupfeasibilitystudy.jpg Supplementary file 3: Figure SUS items per participant group feasibility study .jpeg Supplementary Figure 2. Part II – Feasibility study: proportions of responses to each SUS item per participant group Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 16 Dec, 2024 Reviews received at journal 26 Nov, 2024 Reviews received at journal 25 Nov, 2024 Reviewers agreed at journal 19 Nov, 2024 Reviewers agreed at journal 19 Nov, 2024 Reviewers invited by journal 19 Nov, 2024 Editor assigned by journal 13 Nov, 2024 Submission checks completed at journal 13 Nov, 2024 First submitted to journal 12 Nov, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5440332","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":382942675,"identity":"2510b262-a377-4a9b-9166-3a21eb9bbe2b","order_by":0,"name":"Pearl J. C. van Lonkhuizen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIie3RsUrDQBzH8d9xkC6nWStB+gpXAo3FiA/ikkOwi4GOHSOBuBx2zeYrZHW7cpCpD9DBIRDo3EkymrSVCp5xdbjvkuOSD/8jB9hs/7Po+KQJoBScbr2b77eI6ifkREjOD3u/kK+OZD+Q9ZAAqNBAjwKQp3q+fhcvri7rkOs7NzlTJjJNEBEJPX5LSOrnm63IUM78R67jXJ1HJsIVIgo8kEKRzGM7LTIiJ15HCjDeR25PhLof3tXfJBQHsmmJwxwPPWSaIlpJHt63JPXZWvsZcyYXks/iXJtJMJCiahbDm0LRVc1Kffm61Nths7iOl89yXJkORln7W7pbGP14TQ3fdwQD03CbzWazfesTz5JmLVGjO3IAAAAASUVORK5CYII=","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":true,"prefix":"","firstName":"Pearl","middleName":"J. C. van","lastName":"Lonkhuizen","suffix":""},{"id":382942676,"identity":"d3d6918b-6455-4a8d-bfe3-36b27508747b","order_by":1,"name":"Leanne Slutter","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Leanne","middleName":"","lastName":"Slutter","suffix":""},{"id":382942677,"identity":"9ec50815-0b2a-424c-bee3-28f5938edfda","order_by":2,"name":"Eline Meijer","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Eline","middleName":"","lastName":"Meijer","suffix":""},{"id":382942678,"identity":"587da2e5-60c6-4f84-9da7-b20e64256c66","order_by":3,"name":"Erik van Duijn","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Erik","middleName":"van","lastName":"Duijn","suffix":""},{"id":382942681,"identity":"a7b5b514-8ae0-4cb9-9f0f-f697a1c62b1b","order_by":4,"name":"Susanne T. de Bot","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Susanne","middleName":"T.","lastName":"de Bot","suffix":""},{"id":382942684,"identity":"14b6e6d2-6e58-453e-a8cc-e7a80a68d764","order_by":5,"name":"Niko J. H. Vegt","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Niko","middleName":"J. H.","lastName":"Vegt","suffix":""},{"id":382942685,"identity":"f9328d2f-7230-49b8-bd7e-254038bda937","order_by":6,"name":"Niels H. Chavannes","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Niels","middleName":"H.","lastName":"Chavannes","suffix":""},{"id":382942686,"identity":"f91f9cfc-3f7b-43fd-a987-266b00c8f7d3","order_by":7,"name":"Anne-Wil Heemskerk","email":"","orcid":"","institution":"Leiden University Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Anne-Wil","middleName":"","lastName":"Heemskerk","suffix":""}],"badges":[],"createdAt":"2024-11-12 14:23:35","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5440332/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5440332/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":71693993,"identity":"04871e3f-c153-44b2-834a-ba02b1c9e000","added_by":"auto","created_at":"2024-12-17 19:00:02","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":474700,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eHome page of the final prototype of the Huntington Support App\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAdopted from van Lonkhuizen PJC, Heemskerk A-W, Meijer E, van Duijn E, de Bot ST, Klempir J, Landwehrmeyer GB, Mühlbäck A, Hoblyn J, Squitieri F, Chavannes NH, Vegt NJH, and on behalf of the HEALTHE-RND consortium, 'Development of the Huntington Support App (HD-eHelp study): a human-centered and co-design approach', Frontiers in Neurology, 2024, 1:15:1399126, published under a Creative Commons Attribution License (CC BY). DOI: 10.3389/fneur.2024.1399126. The left part of the original figure has been left out.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure1.png","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/d35009df4906b90c081319f0.png"},{"id":71693992,"identity":"e5080ada-478c-488d-b02b-458ca6a09809","added_by":"auto","created_at":"2024-12-17 19:00:02","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":253186,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePart I - Think-aloud study:\u003c/strong\u003e \u003cstrong\u003eParticipants’ responses to the positive and negative items of the SUS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSUS: System Usability Scale; HSA: Huntington Support App.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure2.png","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/0b46d8618c044dce91634f25.png"},{"id":71694083,"identity":"0c1afcb6-7400-4a65-871f-ea9a671789c1","added_by":"auto","created_at":"2024-12-17 19:08:02","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":206659,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePart II - Feasibility study: HSA visitors’ responses to the positive and negative items of the SUS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSUS: System Usability Scale; HSA: Huntington Support App.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"Figure3.png","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/cd7104df2094a4bc21b2ee37.png"},{"id":71694573,"identity":"4ed6f386-126f-4956-a255-303a1c4bc975","added_by":"auto","created_at":"2024-12-17 19:16:08","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2207933,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/58bde6fc-04c6-4f01-b587-d649fb90a976.pdf"},{"id":71693994,"identity":"fee9bfe7-311f-436e-97d0-d1d368c8ee5d","added_by":"auto","created_at":"2024-12-17 19:00:02","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":130448,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary files\u003c/p\u003e\n\u003cp\u003eSupplementary file 1: Think-aloud protocol\u003c/p\u003e\n\u003cp\u003e.jpeg\u003c/p\u003e\n\u003cp\u003eThink-aloud protocol\u003c/p\u003e","description":"","filename":"Supplementaryfile1Thinkaloudprotocol.docx","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/d32b367c601f9af696a8fddb.docx"},{"id":71694084,"identity":"15004af1-ec5b-4ae5-a724-55c2dc8751e5","added_by":"auto","created_at":"2024-12-17 19:08:02","extension":"jpg","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":628119,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary file 2: Figure SUS items per participant group think-aloud study\u003c/p\u003e\n\u003cp\u003e.jpeg\u003c/p\u003e\n\u003cp\u003eSupplementary Figure 1. Part I – Think-aloud study: proportions of responses to each SUS item per participant group\u003c/p\u003e","description":"","filename":"Supplementaryfile2FigureSUSitemsperparticipantgroupthinkaloudstudy.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/188c852daeae78bb99bdb739.jpg"},{"id":71693996,"identity":"4e60fb14-1e9a-48a1-a07a-239099d2b429","added_by":"auto","created_at":"2024-12-17 19:00:02","extension":"jpg","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":892965,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary file 3: Figure SUS items per participant group feasibility study\u003c/p\u003e\n\u003cp\u003e.jpeg\u003c/p\u003e\n\u003cp\u003eSupplementary Figure 2. Part II – Feasibility study: proportions of responses to each SUS item per participant group\u003c/p\u003e","description":"","filename":"Supplementaryfile3FigureSUSitemsperparticipantgroupfeasibilitystudy.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5440332/v1/93dde358010fffe40a0e9479.jpg"}],"financialInterests":"No competing interests reported.","formattedTitle":"Usability and feasibility assessment of the Huntington Support App (HD-eHelp study): a mixed-methods study","fulltext":[{"header":"1 Background","content":"\u003cp\u003eeHealth has become increasingly important in the management of chronic conditions (1, 2). By utilizing information and communication technologies (3), eHealth offers a range of possibilities, including remote treatment options, remote monitoring, and information exchange (4, 5), all of which can increase care capacity (4). Moreover, eHealth offers the opportunity for personalized, tailored care from within the patient’s home (4, 5). This is particularly important in neurodegenerative diseases due to their progressive nature and the unique challenges they pose. Previous work has demonstrated the acceptability and effectiveness of eHealth services for several\u0026nbsp;neurodegenerative and neurological diseases such as dementia, Parkinson’s disease, and multiple sclerosis (6-11). Participatory initiatives in the development of such services for these groups have been shown to be beneficial (12-14) and are also recommended for more rare diseases (15, 16), as active engagement of end-users increases the uptake and acceptance of eHealth services later on (5, 17-19).\u003c/p\u003e\n\u003cp\u003eeHealth holds great promise in enhancing care delivery for rare (15, 16)\u0026nbsp;and complex neurodegenerative diseases for which no cure is available, such as Huntington’s disease (HD). HD progressively affects motor, cognitive and psychiatric functioning over time (20-22), having a profound impact on work, family, and social life. Children have a 50% chance of inheriting the gene expansion from their affected parent, and if inherited, the disease usually manifest between 30 and 50 years of age (20, 23). HD gene expansion carriers (HDGECs) go through several disease phases, including the premanifest stage where (subtle) symptoms have not yet occurred (pre-symptomatic) or can begin to occur (prodromal) (23-25). After the onset of manifest disease, where clinical changes in motor functioning are evident (22, 23), life expectancy ranges between 15 to 20 years (21).\u003c/p\u003e\n\u003cp\u003eGiven the complex nature of HD and the challenges HDGECs face in managing the disease and its impact on their quality of life (26), eHealth provides a valuable opportunity to increase the availability and accessibility of HD care services across Europe. Previous research has shown that people affected by HD need accessible information about HD and are seeking out knowledge to better understand the disease (27, 28). Due to the absence of eHealth services co-designed with HDGECs to meet their specific needs (29-34), we developed the Huntington Support App (HSA) together with HDGECs, partners and health care providers (HCPs) from various countries (i.e., the Netherlands, Germany, Ireland, Italy and Czech Republic) (35, 36). The HSA is a web-app that offers clear and reliable information on HD, HD-related news and events, and contact possibilities with HCPs via an interactive walk-in hour and appointment booking tool (36). With direct participation of HDGECs, partners and HCPs from the different countries throughout the development process, the HSA is tailored to meet the needs and wishes of these groups.\u003c/p\u003e\n\u003cp\u003eThe prototype of the HSA was well-received and positively evaluated by HDGECs and their partners throughout its development phase (36). In this article, we present an in-depth mixed-methods evaluation of the HSA in practice. In the first part of the study, the app was tested on usability following the ‘think-aloud’ method to refine the app for future use. In the second part, we conducted a feasibility study in which the app was launched publicly and evaluated on usability by members of the Dutch HD community.\u0026nbsp;\u003c/p\u003e"},{"header":"2 Part I – Think-aloud study","content":"\u003ch2\u003e2.1 Methods\u003c/h2\u003e\n\u003ch3\u003e\u003cem\u003e2.1.1 \u0026nbsp;Participants and recruitment\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eDutch HDGECs (i.e., premanifest and manifest), partners and HCPs involved in HD care were recruited, as these same groups actively participated throughout the development process of the HSA (35, 36). It is important to note that all participants were newly recruited and did not have any prior involvement in the development of the HSA. Participants were recruited via the primary and secondary networks of Topaz Overduin, the Huntington Expertise Center that collaborated with the Leiden University Medical Center in co-developing the HSA in the Netherlands (35, 36). For HDGECs and partners, we also recruited via HD-related social media platforms and the Enroll-HD database (37). Eligible participants were aged 18 years or older and proficient in Dutch. HDGECs were residing at home and could participate without a partner. For an overview of all eligibility criteria, see (35). To address recruitment challenges, we broadened the initial inclusion criteria to include HCPs with less than two years of experience, partners living alone, and patient-partner dyads. All eligible participants received an information letter by (e-)mail, dependent on their preference, containing information on the development process of the app as well as information on study participation.\u003c/p\u003e\n\u003ch3\u003e\u003cem\u003e2.1.2 \u0026nbsp;Study design and procedures\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eA mixed-methods approach was used, in which qualitative and quantitative data on usability of the final prototype of the HSA was collected during individual \u0026lsquo;think-aloud\u0026rsquo; sessions, and combined. For an example of the home page of the final prototype, see Figure 1. The think aloud sessions were conducted face-to-face at Huntington Expertise Center Topaz Overduin or Leiden University Medical Center (dependent on participants\u0026rsquo; choice) in November and December 2022. One session was conducted online as it was more convenient and practically manageable for the participant. Participants were asked to verbalize their thoughts and actions as they interacted with the HSA prototype. This think-aloud method is a common procedure in human-centered design processes and provides in-depth insight into usability-related weaknesses and strengths of a system (38). Participants explored the app in the presence of a researcher on their preferred device (i.e., research laptop or own smartphone) and concurrently \u0026lsquo;thought aloud\u0026rsquo; by describing their thoughts (e.g., \u003cem\u003e\u0026ldquo;Strange picture\u0026rdquo;\u003c/em\u003e) and feelings, and verbalizing their navigation behavior. The researcher provided prompts to encourage participants to express their thoughts and asked open-ended questions about certain aspects of the HSA (e.g., \u003cem\u003e\u0026ldquo;How would you describe the app?\u0026rdquo;\u003c/em\u003e or \u003cem\u003e\u0026ldquo;What was confusing?\u0026rdquo;\u003c/em\u003e). Additionally, since HDGECs and partners were the primary target group of the HSA, they completed specific navigation tasks \u003cem\u003e(\u0026ldquo;Where would you find information about symptoms?\u0026rdquo;\u003c/em\u003e) to observe and evaluate the app\u0026rsquo;s navigability. The sessions were guided by a protocol (see Supplementary file 1) to standardize the procedure for all participants and conducted by an independent researcher (LS). At the beginning of each session, participants were explicitly informed that the researcher had not been involved in the app\u0026rsquo;s development. Sessions were audio-recorded and transcribed intelligent verbatim while masking all personally identifiable information.\u003c/p\u003e\n\u003cp\u003eAt the end of the sessions, participants completed an online questionnaire on sociodemographic information, digital skills and usability of the app. For the latter, the Dutch version of the System Usability Scale (SUS) was used. The SUS is a widely-used instrument for evaluating the usability of technologies or services (39). The SUS is comprised of ten items that are evaluated on a Likert scale ranging from 1 \u0026lsquo;strongly disagree\u0026rsquo; to 5 \u0026lsquo;strongly agree\u0026rsquo;. The SUS total score ranges from 0 to 100, with higher scores indicating better usability (39). The SUS is considered a valid and reliable tool, and is also suitable for studies with a small sample size (40).\u003c/p\u003e\n\u003cp\u003eThis study was part of a larger project (HEALTHE-RND HD-eHelp study) that was cleared for ethics by the medical research ethics committee of Leiden Den Haag Delft in the Netherlands (file number: N20.013) (35). Informed consent was signed by all participants prior to the start of the think-aloud session.\u003c/p\u003e\n\u003ch3\u003e\u003cem\u003e2.1.3 \u0026nbsp;Data analyses\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eQualitive data were analyzed using qualitative content analysis (41). This method is suitable for analyzing concrete feedback and user experiences due to its focus on the explicit and visible content of textual data (i.e., words, phrases) (41). All transcripts were read by the first (PL) and second author (LS) to become familiar with the data. Both researchers independently developed an inductively derived codebook based on two transcripts and subsequently discussed and merged both codebooks. The second author performed the initial coding of the remaining transcripts using the finalized codebook, allowing room for new codes to emerge during the analysis. The first author subsequently reviewed all transcripts, made revisions where necessary, and refined the overall coding scheme. The codes were then explored and grouped together in case of similar topics. Categories (groups of similar codes) were identified and grouped into broader themes. Transcripts were analyzed using ATLAS.ti version 23. Relevant quotes from participants were translated from Dutch to English by the first author and checked by the third author, thereby keeping close to the original wording and sentence structure used by participants.\u003c/p\u003e\n\u003cp\u003eDescriptive statistics were used to analyze the quantitative data (IBM SPSS Statistics, version 29.0). SUS scores were recoded to obtain a total score, followed by calculating mean scores and the proportion of responses to each item (i.e., % completely disagree, slightly disagree, neither agree nor disagree, slightly agree, completely agree)\u0026nbsp;for all participant groups. Responses to the SUS and the qualitative findings were triangulated.\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 1 HERE]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 1.\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eHome page of the final prototype of the Huntington Support App\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAdopted from van Lonkhuizen PJC, Heemskerk A-W, Meijer E, van Duijn E, de Bot ST, Klempir J, Landwehrmeyer GB, M\u0026uuml;hlb\u0026auml;ck A, Hoblyn J, Squitieri F, Chavannes NH, Vegt NJH, and on behalf of the HEALTHE-RND consortium, \u0026apos;Development of the Huntington Support App (HD-eHelp study): a human-centered and co-design approach\u0026apos;, Frontiers in Neurology, 2024, 1:15:1399126, published under a Creative Commons Attribution License (CC BY). DOI: 10.3389/fneur.2024.1399126. The left part of the original figure has been left out.\u003c/em\u003e\u003c/p\u003e\n\u003ch2\u003e2.2 \u0026nbsp; \u0026nbsp; Results\u003c/h2\u003e\n\u003cp\u003eIn total, 4 HDGECs (2 premanifest and 2 manifest), 4 partners of HDGECs (2 of premanifest and 2 of manifest HDGECs), and 4 HCPs participated in this part of the study. HCPs were from different professions involved in HD care (i.e., elderly care physician, dietician, social worker and psychologist). See Table 1 for more information on participants\u0026rsquo; characteristics.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1. Part I \u0026ndash; Think-aloud study: Participants\u0026rsquo; characteristics and mean SUS score at time of think-aloud session\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"453\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHDGECs\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 4)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePartners\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 4)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHCPs\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 4)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge\u0026nbsp;\u003c/strong\u003e(mean; range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e48; 40-54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e53; 34-64\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e42; 26-57\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eGender\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Male\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e0 (0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Female\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e4 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLiving situation\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eTogether with partner\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Alone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Together with others\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eProfession\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Dietician\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Social worker\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Psychologist\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Elderly care physician\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eYears of experience with HD\u0026nbsp;\u003c/strong\u003e(mean; range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e3.7; 1-10\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTime since genetic test in years\u0026nbsp;\u003c/strong\u003e(mean; range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e11; 1-24\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e16; 6-27\u003csup\u003e1\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eUse of health apps\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Often\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Sometimes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Rarely\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Never\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSelf-reported digital skills\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Very experienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Experienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Neither experienced or\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; inexperienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eInexperienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Very inexperienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 208px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSUS total score\u0026nbsp;\u003c/strong\u003e(mean; range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e84; 78-90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e81; 60-90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e65; 35-83\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eHDGECs: Huntington\u0026rsquo;s disease gene expansion carriers; N: number of participants; HCPs: Health care providers; SUS: System Usability Scale (SUS). Numbers and percentages are rounded to the nearest whole number.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003csup\u003e1\u003c/sup\u003e\u003c/em\u003e\u003cem\u003e\u0026nbsp;This refers to the genetic test of the affected partner.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eOf the twelve participants, eleven participants preferred to use their personal smartphones to evaluate the HSA during the think-aloud session. A partner of a premanifest HDGEC preferred to use the research laptop.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOverall, three themes related to the app\u0026rsquo;s usability were identified from the qualitative data, which triangulated and complemented the quantitative descriptive data from the SUS. Each theme is described below. Descriptive statistics for the SUS scores for the total sample can be found in Table 1 and Figure 2 and are reported as part of the three themes where relevant. For the proportion of responses for each item per participant group (i.e., HDGECs, partners and HCPs), the supplementary files can be referenced.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e2.2.1 \u0026nbsp;Theme 1: Usability of the Huntington Support App: user friendly and comprehensive, with emotionally sensitive content for some\u003c/h3\u003e\n\u003cp\u003eOverall, participants were positive about the app\u0026rsquo;s usability, as reflected in the total SUS score across all groups (i.e., 76.6), which exceeds the benchmark of 68 commonly found for the usability of health apps (42, 43). Particularly, HDGECs and partners were very positive. Their SUS total score averaged above 80 (i.e., 84 and 81 respectively, see Table 1). HCPs also expressed a positive attitude towards the app during the think-aloud sessions, yet had more feedback on and suggestions for improvement of the app than HDGECs and partners. This was also reflected in their overall lower SUS score of 65, slightly below the benchmark.\u003c/p\u003e\n\u003cp\u003eOverall, participants reported the app to be user friendly and convenient:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;[The app is] clear. Clean, I like that. I find it organized. [\u0026hellip;]. So I find it very user-friendly.\u0026rdquo;\u0026nbsp;\u003c/em\u003e(premanifest HDGEC)\u003c/p\u003e\n\u003cp\u003eMost participants mentioned that the app would especially be helpful in finding information about the disease as it bundles all relevant information about HD. One manifest HDGEC described the app as:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;[\u0026hellip;] a great improvement towards the digital world, where you can find everything about Huntington\u0026rsquo;s disease at a glance. From information about Huntington\u0026rsquo;s disease to making appointments to everything else. I find it very clear.\u0026rdquo;\u0026nbsp;\u003c/em\u003e(manifest HDGEC)\u003c/p\u003e\n\u003cp\u003eParticularly, HDGECs and partners mentioned the information in the app to be relevant and tailored to their needs. Next to providing information, all participants perceived the HSA as an app that provides the latest news updates, fosters connection with peers and HCPs, and facilitates in accessing healthcare services.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eParticipants found the app\u0026rsquo;s content comprehensive and valuable. According to them, the information is presented concisely and clearly and is written in simple language. This was also observed in participants\u0026rsquo; SUS item scores. Most participants found the app straightforward (84%), consistent (83%), and easy to use (75%) (see Figure 2 item 2, 6 and 3 respectively). Primarily HCPs found the HSA not so easy to use and unnecessarily complex. All participants but one HCP felt confident in using the HSA (see Supplementary Figure 1 item 9 in the Supplementary files).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eParticipants who commented on the lay-out of the app especially liked the overall \u0026lsquo;professional\u0026rsquo; look, as well as the text size and used colors. They liked that the font size is adjustable, which, according to some, comes in handy when the disease progresses and challenges in motor function arise.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNext to outlining the app\u0026rsquo;s positive aspects, some participants also highlighted areas for improvement. Two premanifest HDGECs and one HCPs found certain information parts of the app somewhat confronting:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;No, well, it is what it is, but indeed, there\u0026rsquo;s really nothing cheerful on it [the app]. And maybe it\u0026rsquo;s just not cheerful at all [the disease]. But you don\u0026rsquo;t come here [in the app] for cheerfulness, right?\u0026rdquo;\u0026nbsp;\u003c/em\u003e(premanifest HDGEC)\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;And I kind of think that if you\u0026rsquo;re a gene carrier and you go to health [topic on the app], and you see somewhere, or under symptoms, \u0026lsquo;I have swallowing problems\u0026rsquo;, then you think \u0026lsquo;oh, am I going to get that in the future? That doesn\u0026rsquo;t make me happy\u0026rsquo;. Whereas if you already have them [the swallowing problems], and that\u0026rsquo;s also disease awareness, then it\u0026rsquo;s very different.\u0026rdquo;\u0026nbsp;\u003c/em\u003e(HCP)\u003c/p\u003e\n\u003cp\u003eTwo HCPs suggested rephrasing certain sentences to avoid the first-person perspective (e.g., \u0026lsquo;Sleeping difficulties\u0026rsquo; instead of \u0026lsquo;I experience sleeping difficulties\u0026rsquo;) to soften its directive tone. Similarly, about half of participants (only HCPs and partners) perceived the interactive walk-in hour feature on the homepage as too prominent, particularly for those who choose not to use it. Others did not comment on the directiveness of the feature or found it inviting, thereby lowering the threshold to healthcare. One premanifest HDGEC specifically suggested to give the appointment booking tool a more prominent place on the app as it was somewhat difficult to find. Overall, the majority of HDGECs and premanifest partners were positive about the usability and user friendliness of the interactive features.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eRegarding the app\u0026rsquo;s functionality, some usability issues were mentioned primarily by HCPs and some partners. They pointed out that certain functionalities were not working properly yet, such as hyperlinks, the app\u0026rsquo;s search function, and the location filter for finding offline peer group meetings. See Table 2 for an overview of the most frequently mentioned suggestions for the app\u0026rsquo;s improvement.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e2.2.2 \u0026nbsp;Theme 2: Navigating the Huntington Support App: well-organized and accessible, with room for stronger integration \u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eIn terms of the app\u0026rsquo;s navigation, most HDGECs and partners found the app well-organized and easy to navigate. About half of them preferred to use the search bar to find information and liked that information was tailored to specific groups (i.e., being an HDGEC or a partner/family member). All participants were particularly positive about the layered presentation of information and the flexibility to expand relevant sections if needed:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;I think it\u0026rsquo;s quite nice, that short summary. They [HDGECs], when it comes to attention and concentration, that\u0026rsquo;s one of the early things that become difficult. So, having a brief text that summarizes it allows you to decide whether to continue or not, without having to read through long passages.\u0026rdquo;\u0026nbsp;\u003c/em\u003e(HCP)\u003c/p\u003e\n\u003cp\u003ePositive feedback about the app\u0026rsquo;s structure was also observed in the quantitative SUS data, as all participants but one HCP found the app convenient to navigate (see Supplementary Figure 1 item 8 in the Supplementary files). A manifest HDGEC and partner expressed their appreciation for the app\u0026rsquo;s information display, emphasizing the flexibility to selectively choose content to engage with. However, some participants were less pleased with having to click multiple times to access specific content.\u0026nbsp;In terms of redirecting to other sites for more information about certain topics, participants also had differing views. Some found it convenient to have all the information on the app itself, whereas others liked that the app provides the key information and redirects to other sites for more detail about specific topics:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;It\u0026rsquo;s indeed, I find it nice that everything is written quite concisely. But that there might indeed be the option to click through to dive deeper into everything.\u0026rdquo;\u0026nbsp;\u003c/em\u003e(partner of a premanifest HDGEC)\u003c/p\u003e\n\u003cp\u003eMost HCPs and one manifest HDGECs reported some (anticipated) difficulties with navigating back to the app after opening a redirection link, whereas the other HDGECs and partners did not report any difficulties. Moreover, some difficulties were reported with navigating back to main topics within the app itself. These issues with the integration of various functions were also apparent in the SUS item score (see Figure 2 item 5). About one-third of participants reported that the app\u0026rsquo;s functions were not well integrated (mostly expressed by HCPs; see Supplementary Figure 1 item 5 in the Supplementary files), while the remaining two-third of participants found the integration satisfactory.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDuring the navigation tasks, some HDGECs and partners encountered challenges in navigating to specific topics, often due to differing expectations of the location or confusion between the walk-in hour and appointment booking tool. However, most of the navigation tasks were easy for them to perform. Several recommendations to improve the app\u0026rsquo;s intuitiveness/navigation have been suggested by participants, which are displayed in Table 2.\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003e2.2.3 \u0026nbsp;Theme 3: Future potential of the Huntington Support App: easy access to care, broad applicability, and practical considerations\u003c/h3\u003e\n\u003cp\u003eMany participants (9 out of 12) mentioned that they would recommend the app to others, including HDGECs and their partners, as well as others in the community to aid in understanding the disease:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;I think for clients in the early stages and informal caregivers, loved ones, who are looking for information. It might be handy for them too, you get told that you have or will get it [the disease]. People don\u0026rsquo;t understand it well because no one knows the disease. Then you can say, \u0026lsquo;Yeah, there\u0026rsquo;s this app, I have it too. You might download it.\u0026rsquo; That way, people can read it at their own pace afterwards, and the client doesn\u0026rsquo;t have to explain everything because I think that\u0026rsquo;s usually difficult. It [the app] can provide a bit of support for what social work already does.\u0026quot;\u0026nbsp;\u003c/em\u003e(HCP)\u003c/p\u003e\n\u003cp\u003eAdditional target audiences mentioned included HDGECs without a family history of HD, those who have limited to no contact with an HD center, and individuals with early-onset HD.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe majority of HCPs and all partners indicated that the app is best suitable for those who are in the early and mid-phases of HD when people have mild to moderate symptoms. Some HCPs and one manifest HDGECs indicated that using and understanding the content of the app would be too difficult in the advanced stages of HD:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026ldquo;The app is not suitable for people who are really in the advanced stages. I think those [advanced patients] are (A) already so experienced that they don\u0026apos;t benefit from the content anymore, and B) in practice, their motor skills are significantly affected such that using an app would be very difficult, both on a phone and laptop. They have difficulty with their fine motor skills and experience significant chorea, which makes it challenging for them to operate the app effectively. But I don\u0026apos;t think that\u0026apos;s a problem. I also don\u0026apos;t think you should have a \u0026apos;one size fits all\u0026apos;, so to speak. I think if you focus on people in the early and somewhat advanced stages [of the disease], you\u0026apos;re already well on your way.\u0026quot;\u0026nbsp;\u003c/em\u003e(HCP)\u003c/p\u003e\n\u003cp\u003eTwo manifest HDGECs and a manifest partner pointed out that once the disease has progressed, help is likely needed when navigating the app:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026quot;And if, as a patient, you can no longer operate it [the app] yourself, then the caregiver will have to do it for you.\u0026quot;\u0026nbsp;\u003c/em\u003e(manifest HDGEC)\u003c/p\u003e\n\u003cp\u003eDespite the variety in digital skills reported (see Table 1), none of the participants expressed a need for assistance in using the HSA (see Figure 2, item 4). Most participants (92%) stated that prior knowledge is not necessary to use the HSA (Figure 2, item 10), and all participants but two HCPs found the HSA easy to learn (see SUS item 7 in Supplementary Figure 1 in the Supplementary files).\u003c/p\u003e\n\u003cp\u003eAll participants (except for one HDGEC) reported that they would use the HSA frequently (Figure 2, item 1). This was also reflected in the qualitative data, as all participants mentioned that they would (occasionally) use the app and its interactive features. Participants mentioned that such features lower the threshold to care by increasing easy access and reducing travel time:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026quot;Traveling takes a lot of time. And it\u0026apos;s not always necessary, I think, to speak to each other in person. So, I would definitely make use of that [the interactive features].\u0026quot;\u0026nbsp;\u003c/em\u003e(partner of a premanifest HDGEC)\u003c/p\u003e\n\u003cp\u003eOnly one premanifest HDGEC acknowledged not wanting to use the app at this time, as she preferred to distance herself from the disease for the time being:\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026quot;Some people have a strong need for information, so they spend the whole day searching the internet and doing all sorts of things. And I really don\u0026apos;t want to, to know. Because right now, I don\u0026apos;t want it [the disease] to already control my life.\u0026quot;\u0026nbsp;\u003c/em\u003e(premanifest HDGEC)\u003c/p\u003e\n\u003cp\u003eFor this participant at this point in her life, the information in the app appeared too confronting. However, she mentioned that she might want to use the HSA in the future if needed.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNone of the HCPs reported to use any eHealth services in current HD care. They indicated that they would not use the HSA to find information on HD for themselves, as they are already familiar with most of it. Instead, they would use the HSA to refer HDGECs and their partners for information or for themselves to stay updated on expert centers and colleagues.\u003c/p\u003e\n\u003cp\u003eRegarding the use of the interactive features, most HDGECs and partners reported to be willing to use these functions in the future, whereas most HCPs expressed their reservations especially about using the appointment booking tool as they are concerned about its fit with the current care pathways. They prefer the current with a case manager/social worker as the initial point of contact who refers patients to the appropriate HCPs if necessary. If HDGECs or other app visitors were to schedule appointments directly with them, they are unsure whether they can provide the necessary assistance or if the correct HCP has been selected. Other practical considerations for future use of an appointment booking tool mentioned across participant groups included time management, difficulties around reimbursement by healthcare insurance, and cost-effectiveness. HCPs showed more favorable attitudes towards the online walk-in hour, with many expressing willingness to host such sessions if they meet the general need. Some important requirements for implementation of the walk-in hour were mentioned by HCPs, such as security and privacy of data.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2. Part I -\u0026nbsp;Think-aloud study:\u003c/strong\u003e \u003cstrong\u003eDesign and content recommendations for the HSA\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePersonalization\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Present information / interactive elements that fits the needs of a specific user\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Add filters so that users can find centers and HCPs that are geographically closest to them\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eUsability\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Ensure that all functionalities work properly (e.g., hyperlinks, search bar, filters)\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Differentiate more clearly between the information and contact components of the app, and improve findability of the latter\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Add explanation to certain terms (e.g., \u0026lsquo;telehealth\u0026rsquo;, \u0026lsquo;pen pal\u0026rsquo;, \u0026lsquo;what to expect\u0026rsquo;) and avoid abbreviations\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eNavigation\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Add navigation aids for internal navigation (e.g., category tiles, decision tree) and external redirection (e.g., back button, instructions on how to navigate back)\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Add a dropdown menu for easier navigation\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eComprehensiveness\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Increase representation of other HD centers and HCPs across the country\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Update current content, list of HCPs, meetings, and news items\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Extent the app with more content (including topics as euthanasia, intimacy, financial and practical aspects that come with living with HD) and provide more detailed information on certain topics either within the app or via external references\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAdditional potential features beyond the app\u0026rsquo;s current scope\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 548px;\"\u003e\n \u003cp\u003e\u003cb style='text-align: left;color: rgb(32, 33, 36);background-color: rgb(255, 255, 255);font-size: 14px;font-family: \";'\u003e\u0026bull;\u003c/b\u003e Keep track of personal details (logbook, favorites, current location)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eHCPs: Health care providers; HD: Huntington\u0026rsquo;s disease.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 2 HERE]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 2. Part I - Think-aloud study:\u003c/strong\u003e \u003cstrong\u003eParticipants\u0026rsquo; responses to the positive and negative items of the SUS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSUS: System Usability Scale; HSA: Huntington Support App.\u003c/em\u003e\u003c/p\u003e"},{"header":"3 Part II – Feasibility study","content":"\u003ch2\u003e3.1 Methods\u003c/h2\u003e\n\u003ch3\u003e\u003cem\u003e3.1.1 \u0026nbsp;Participants and recruitment\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eAs the Huntington Support App (HSA) was made openly accessible to everyone in the HD community for the feasibility study, eligibility criteria were not applicable. The HSA was advertised via HD-related social media platforms and through 9 out of 10 approached outpatient clinics at Huntington (expertise) centers in the Netherlands. One approached center chose not to distribute advertisement flyers as, according to them, the app\u0026rsquo;s content was not representative of all HD centers. Everyone who was interested could voluntarily decide to visit the web app via a link/QR-code that was provided in the advertisement.\u003c/p\u003e\n\u003ch3\u003e\u003cem\u003e3.1.2 \u0026nbsp;Study design and procedure\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eAn observational feasibility study was designed to evaluate the usability of the minimal viable product (MVP) of the HSA over a six-week period, with participant feedback rounds at two time points. The MVP, a first version of a digital health product intended for real-world evaluation, was developed based on feedback from the think-aloud study (Part I). It was openly accessible without a pin code and featured a functional search bar. The final content was reviewed by HD experts from Huntington Expertise Center Topaz Overduin (i.e., by a psychiatrist, neuropsychologist, speech therapist, case manager, physiotherapist, dietician, and social worker). It also included additional content, drafted based on input from the think-aloud sessions and subsequently reviewed by the experts, covering topics such as genetic risk for children, the euthanasia process, and a comprehensive list of all Dutch Huntington expertise centers.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe MVP of the HSA was launched publicly on February 1\u003csup\u003est\u003c/sup\u003e, 2023. During four weeks in February, the app included an online walk-in hour every Monday with a social worker with expertise in HD to ask HD-related questions. The social worker was asked to collect information on who visited the walk-in hour (i.e., a patient, partner, health care provider, family member or other) and for what reason.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eVisitors were able to leave their anonymous feedback on the HSA by filling in a questionnaire through the corresponding links/QR-codes in the web-app or advertisement. Visitors could voluntarily decide to leave feedback, and if they chose not to, the app remained fully accessible to them. The feedback questionnaire included quantitative items on sociodemographics (i.e., age categories, gender, living situation, relationship to HD), digital skills, device used, satisfaction, future usage, and the System Usability Scale (SUS) (see Part I). At the end of the questionnaire, respondents could provide qualitative responses, offering suggestions for improving the HSA in open-ended response fields. Additionally, participants were informed that they could evaluate the web-app and its usage again after four weeks by contacting the research team to receive the follow-up questionnaire. However, none of the participants reached out, resulting in no follow-up questionnaires being completed\u003csup\u003e[1]\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eAs the feasibility study was not originally planned within the larger HD-eHelp study, an amendment to the original study protocol (file number: N20.013) (35) was submitted. The amendment was approved by the scientific overview committee of the Department of Public Health and Primary care of Leiden University Medical Center. Given the low-threshold, non-intrusive nature and voluntary basis of the feasibility study, the amendment did not require additional approval from the medical ethics committee. A brief explanation on the study and data anonymity was included in the online questionnaire. Digital informed consent was signed by all participants prior to the start of the questionnaires.\u003c/p\u003e\n\u003ch3\u003e\u003cem\u003e3.1.3 \u0026nbsp;Data analyses\u003c/em\u003e\u003c/h3\u003e\n\u003cp\u003eDescriptive statistics were used to calculate the mean scores and frequencies/proportions on respondents\u0026rsquo; characteristics (sociodemographics, digital skills, device used, satisfaction, future usage) as well as on the SUS scores (IBM SPSS Statistics, version 29.0). For the SUS, the same procedure was followed as previously described in Part I. Responses to the open-ended questions, were extracted and key points were highlighted, with a similar approach intended for the documented reasons for visiting the walk-in hour.\u003c/p\u003e\n\u003ch2\u003e3.2 Results\u003c/h2\u003e\n\u003cp\u003eIn total, 13 respondents completed the initial feedback questionnaire on the HSA, and none of them reached out to receive the four-week follow-up questionnaire. Respondents consisted of a mixed group from the HD community, including HDGECs (n = 4), partners of manifest HDGECs (n = 3), HCPs (n = 3), family members of manifest HDGECs (n = 2), and other (i.e., person with no relationship to HD, n = 1). Of the HDGECs, two reported to be in the premanifest stage whereas the other two reported to have manifest disease. HCPs were all (case)managers HD. See Table 3 for an overview of the other collected characteristics per respondent group.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 3. Part II \u0026ndash; Feasibility study: Respondents\u0026rsquo; characteristics and usability outcomes of the HSA\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"652\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHDGECs\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 4)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePartners\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 3)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHCPs\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 3)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFriends/family\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 2)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eOther\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(n = 1)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" valign=\"top\" style=\"width: 652px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eRespondents\u0026rsquo; characteristics\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge range category\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;1940-1960\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e2 (67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; 1961-1980\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;1981-2000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eGender\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Male\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Female\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e2 (67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePrevious involvement app development\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Yes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;No\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e2 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eUse of health apps\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Often\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Sometimes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Rarely\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Never\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e1 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSelf-reported digital skills\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; (Very) experienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e2 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Neither experienced or\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Inexperienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e(Very) inexperienced\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" valign=\"top\" style=\"width: 652px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u003cem\u003eUsability outcomes\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDevice used\u0026nbsp;\u003c/strong\u003e(n (%))\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Smartphone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e2 (67)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e2 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; Computer/laptop\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Tablet\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSatisfaction with app\u0026nbsp;\u003c/strong\u003e(mean; range)\u003cstrong\u003e\u003cbr\u003e\u0026nbsp;\u003c/strong\u003e\u003cem\u003e(0: not satisfied to 10: very satisfied)\u003c/em\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e7 (6-8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003e8 (6-10)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e6 (5-7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 104px;\"\u003e\n \u003cp\u003e9 (8-9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e8 (8-8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eWould use app in future\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; (Strongly) agree\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e2 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;Neutral\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; (Strongly) disagree\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (33)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eIntended use of app\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; To gather information on HD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e1 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp;To stay up to date with latest news\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e2 (50)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e2 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eTo find peer support\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eTo attend the online walk-in hour\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eOther\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85px;\"\u003e\n \u003cp\u003e3 (100)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 217px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSUS total score\u0026nbsp;\u003c/strong\u003e(mean; range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e68 (50-95)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e71 (58-90)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e53 (48-63)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e74 (65-83)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e78 (78-78)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eHDGECs: Huntington\u0026rsquo;s disease gene expansion carriers; N: number of participants; HCPs: Health care providers; SUS: System Usability Scale. Numbers and percentages are rounded to the nearest whole number. The participant in the \u0026lsquo;Other\u0026rsquo; group indicated that she did not have any relationship to HD, but expressed a desire to provide feedback on the app.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003ch3\u003e3.2.1 \u0026nbsp;Usability of the HSA in practice\u003c/h3\u003e\n\u003cp\u003eThe overall SUS scores across all groups (i.e., 68.8) is comparable to the standard benchmark of 68 commonly observed in usability testing for healthcare apps (42, 43). In particular,\u0026nbsp;partners and friends/family members were very satisfied with the app (rating of 8 and 9 out of 10, respectively) and its usability (i.e., 71 and 74 respectively). HDGECs were also satisfied with the app and its usability (see Table 3). HCPs, however, reported moderate satisfaction with the HSA and their overall usability score (i.e., 53) was below the benchmark.\u003c/p\u003e\n\u003cp\u003eThe majority of HDGECs, and all partners and friends/family members reported to be wanting to use the HSA in the future. They would like to use the HSA either for information purposes or to stay updated on the latest news on scientific research. The latter was also what family/friends reported they intend to use the HSA for. Two HCPs indicated that they would use the app for professional purposes; one HCPs reported not to intend using the HSA.\u003c/p\u003e\n\u003cp\u003eWith regard to the online walk-in hour, most HDGECs and one partner expressed occasional interest in using it in the future. The other HDGEC, two partners and family/friends indicated that they would rarely use it, whereas all HCPs reported that they would not use it in the future. To note, none of the app\u0026rsquo;s visitors attended the online walk-in hour sessions with the social worker.\u003c/p\u003e\n\u003cp\u003eWhen looking at the SUS scores across all groups (Figure 3), participants were mostly neutral to (very) positive. A slight majority of all respondents found the HSA convenient and easy to learn, reported feeling confident using the app, and expressed interest in using it frequently (item 8, 7, 9 and 1 respectively in Figure 3). Most respondents indicated that using the HSA did not require assistance or prior knowledge (item 4 and 10 in Figure 3). However, opinions regarding the ease of use, complexity, consistency, and integration of the HSA\u0026rsquo;s functions varied among respondents, with approximately half of participants expressing neutral responses (item 3, 2, 6 and 5 respectively in Figure 3). When looking at the SUS item responses per subgroup (see Supplementary Figure 2 in Supplementary files), most of the neutral responses were observed among the HCPs.\u003c/p\u003e\n\u003cp\u003e[INSERT FIGURE 3 HERE]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 3. Part II - Feasibility study: HSA visitors\u0026rsquo; responses to the positive and negative items of the SUS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSUS: System Usability Scale; HSA: Huntington Support App.\u003c/em\u003e\u003c/p\u003e\n\u003ch3\u003e3.2.2 \u0026nbsp;Strengths and suggestions for improving the HSA\u003c/h3\u003e\n\u003cp\u003eRespondents highlighted several strengths of the HSA in the open-ended response fields of the questionnaire. Participants praised the app for its clarity and user-friendly design, making it easy to navigate and understand. They valued the app\u0026rsquo;s focus on providing accessible and relatable information. The comprehensive range of topics covered by the HSA was another significant strength noted by respondents, as according to some, it bundles all relevant information on HD. Furthermore, the app\u0026rsquo;s ability to connect users to existing resources and services was seen as beneficial, facilitating access to relevant support.\u003c/p\u003e\n\u003cp\u003eIn addition, respondents provided suggestions for improving the HSA in the future. Some participants suggested including a comprehensive list of all HD experts available in the Netherlands, e.g., case managers were currently not listed in the app. Additionally, participants emphasized the need to increase the representation and information from other HD centers across the country, with some noting that the current focus was too concentrated on the center involved in co-designing the app. Some participants also suggested expanding the app with more detailed information on HD and to cover more topics relevant to users. Lastly, one respondent recommended making the list of HD expertise centers more visually appealing, such as by integrating them into a country-wide map display.\u003c/p\u003e"},{"header":"4 Discussion","content":"\u003cp\u003eThis study provided valuable insights into the usability and feasibility of the Huntington Support App (HSA), a web-app intended to offer clear and reliable information on Huntington\u0026rsquo;s disease (HD), HD-related news and events and interactive contact possibilities with health care providers (HCPs) (36). Building on prior research outlining the co-creation process of the HSA prototype (36), the present study employed a mixed-methods approach to gain an in-depth evaluation of the HSA in practice. By conducting a \u0026lsquo;think-aloud\u0026rsquo; study and subsequent feasibility evaluation of the HSA, we were able to identify several key themes and outcomes that have implications for the future refinements of the HSA and the development of similar digital health services for rare neurodegenerative diseases such as HD.\u003c/p\u003e\n\u003ch2\u003e4.1 Usability and feasibility of the Huntington Support App\u003c/h2\u003e\n\u003cp\u003eOverall, the HSA was well-received in both studies. Especially during the think-aloud study, participants were very positive about the app\u0026rsquo;s usability with their scores on the usability questionnaire far exceeding the benchmark found in usability testing for healthcare apps (42, 43). In the feasibility study, the usability of the HSA was rated lower, yet remained slightly above the standard benchmark for healthcare apps\u0026nbsp;(42, 43). Next to positive evaluations, more neutral responses on the usability of the HSA were observed during the feasibility study. About half of the participants reported being neither satisfied nor dissatisfied with the different usability aspects of the HSA. This may be due to the fully anonymous nature of the feasibility study. Participants had the flexibility to review the app and complete the questionnaire at their convenience without a researcher present, unlike in the think-aloud study. Although it was explicitly stated during the think-aloud session that the researcher was not involved in the app\u0026rsquo;s development and participants completed the questionnaire independently, the presence of a researcher could have influenced their responses.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWhen looking at each participant group separately, considerable variations in how the HSA was evaluated were observed. HD gene expansion carriers (HDGECs) and partners reported very positive experiences with the HSA in both studies. Although HCPs were generally positive, their views were less favorable compared to those of HDGECs, partners, and family/friends. This is not surprising, as the app\u0026rsquo;s development primarily focused on the needs of HDGECs and their partners\u0026nbsp;(36). HCPs were involved to a lesser extent throughout the app\u0026rsquo;s development and were not considered the primary target group\u0026nbsp;(36), likely leading to their needs not being fully addressed within the app. Moreover, HCPs may feel more compelled to express concerns and advocate for improvements when evaluating (future) care services given their professional responsibilities and familiarity with clinical workflows. Furthermore, HCPs may have evaluated the app from the perspective of HDGECs rather than their own, which may have led them to perceive the HSA as more difficult for HDGECs to use. For instance, about half of the HCPs in the think-aloud study reported that they did not find the HSA easy to use, easy to learn or straightforward, whereas HDGECs themselves did not share these concerns. HDGECs, partners, friends/family members, as well as someone with no connection to HD were (very) positive about the app\u0026rsquo;s usability.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003e4.2 The importance of human-centered design\u003c/h2\u003e\n\u003cp\u003eParticipatory design is highly recommended for aligning eHealth solutions with end-users\u0026rsquo; needs (5, 17-19) and has shown success in other neurodegenerative and neurological diseases (e.g., dementia, Parkinson\u0026rsquo;s disease, and multiple sclerosis) (12-14). However, despite its recommendations for rare diseases\u0026nbsp;(15, 16) and its potential for HD, research on participatory design and co-creation of eHealth for HD is limited\u0026nbsp;(30-32, 34, 46, 47).\u0026nbsp;Our findings illustrate the value of adopting a participatory human-centered design (HCD) approach for a rare and complex disease like HD. Throughout the app\u0026rsquo;s development, HDGECs and their partners were prioritized as the primary end-user groups (36). The positive feedback from especially these groups in the present study underlines the effectiveness of HCD in developing an app that aligns with the needs and expectations of end-users.\u003c/p\u003e\n\u003cp\u003eParticipants in the present study were newly recruited and had not been involved in the app\u0026rsquo;s development. This shows that the app not only fulfills needs from the participants that co-designed the app, but addresses needs commonly present among those affected by HD. Diverse representation in the co-design process of the end-user group, early and consistent involvement throughout the process, and rigorous analysis of what participants say, seem important factors for developing an app that closely addresses the needs of the target audience. Moreover, involving end-users from the beginning not only ensures that the solution meets their needs but also likely enhances its adoption and uptake later on (5, 17, 18, 48-52).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe mixed findings regarding the (future) use of the interactive features of the HSA (i.e., appointment booking tool and walk-in hour) further highlights the importance of the HCD design process. While most participants (HDGECs and partners) expressed willingness to use these features in the future during both the development phase (36) and think-aloud study, none of the app\u0026rsquo;s visitors attended the online walk-in sessions scheduled during the feasibility study. This lack of attendance could be attributed to the overall low visitor rate and/or the study\u0026rsquo;s short timeframe, with only one scheduled walk-in hour a week, of which the timing (i.e., Monday morning) may not have been convenient for everyone. Moreover, the interactive features may not have closely aligned with the needs of HDGECs and partners, as these features did not emerge from their initial needs assessment across all participating countries\u0026nbsp;(36). The need for these functionalities was mainly expressed by HCPs from the other countries\u0026nbsp;(36). Since this study was conducted in the relatively small Dutch population, it is important to explore how the HSA is evaluated in the other countries. They face greater challenges in bridging geographic distances\u0026nbsp;(36) and have different healthcare organizations with relatively fewer HD services available, making these interactive features more essential. This underscores the importance of considering and designing for different target groups, as their needs and requirements can vary greatly depending on country and care context.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOur HCD approach allowed us to develop a customizable app tailored to the diverse and complex needs of various participant groups. By aligning with these needs, the HSA was positively evaluated not only by the primary end-users, but also by others who were indirectly or not involved in the app\u0026rsquo;s development, such as family members and friends. While it is not possible to meet all needs and preferences with a single eHealth solution\u0026nbsp;(36), the HSA demonstrates that, through a participatory design approach, we were able to address a wide range of expressed needs and wishes. Our study further shows that such an approach allows for the development of an app that can address both current and emerging needs of different groups affected by HD. The HSA was designed to be easily adopted and tailored to specific target groups and different healthcare contexts/language regions, with the possibility to incorporate new features if new needs emerge\u0026nbsp;(36).\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003e4.3 Study\u0026rsquo;s limitations and strengths\u003c/h2\u003e\n\u003cp\u003eAlthough the findings of our study are promising, they should be interpreted with several limitations in mind. First, the think-aloud study included a relatively well-functioning group of participants in terms of disease characteristics and digital literacy which may have influenced the results toward more favorable usability outcomes. Second, the small sample size in both studies limits the generalizability of our findings to the larger HD community. At the same time, it reflects the challenges of studying a rare condition like HD. We faced some recruitment challenges in the think-aloud study that necessitated broadening our initial inclusion criteria. Additionally, the recruitment strategy for the feasibility study involved leaving flyers at HD expertise centers, and therefore lacked active advertising and tracking. This approach potentially limited our sample size and response rate. As a result, we had little insight into how many individuals noticed the flyers and subsequently viewed and used the HSA during that period. This made it difficult to assess the extent of bias among survey respondents. While our approach was sufficient for collecting initial user feedback from a larger group in the HD community, a more proactive strategy for promoting the flyers and monitoring its distribution, as well as a longer time frame for the study, could have enhanced participant engagement.\u003c/p\u003e\n\u003cp\u003eDespite these limitations, some of the study\u0026rsquo;s strengths are worth mentioning as well. The mixed-methods approach in the think-aloud study provided both a comprehensive overview of the app\u0026rsquo;s usability and detailed insights into specific areas for improvement. The quantitative data corroborated the detailed qualitative feedback from participants, strengthening the overall reliability of our findings on the app\u0026rsquo;s usability. Furthermore, by recruiting new participants who had not been part of earlier phases of the app\u0026rsquo;s development process, we were able to assess the effectiveness of our HCD approach and identify additional areas for improvement. Lastly, conducting both a think-aloud and feasibility study added significant value. The think-aloud study allowed us to address major issues, such as navigation and content improvements, before the app was publicly tested in the feasibility study. With these key issues already resolved, the feasibility study could provide a more accurate assessment of the app\u0026rsquo;s overall usability. This broader rollout provided insights into how the app was received by user groups not previously involved in its development, including family, friends, and other stakeholders. It also offered a first understanding of how the app performed in a real world-setting, highlighting directions for future refinement and use.\u003c/p\u003e\n\u003ch2\u003e4.4 Implications for future research and practice\u003c/h2\u003e\n\u003cp\u003eThis study identified several key areas for improving the HSA, such as navigation challenges, content suggestions and its tone, and differing user needs regarding some of the app\u0026rsquo;s features. Future refinements of the HSA should address these, while also examining the use and effectiveness of the interactive features (i.e., walk-in hour and appointment booking tool) more thoroughly. We gathered feedback on participants\u0026rsquo; willingness to use the appointment booking tool in the future. While testing its actual use was beyond the scope of this study due to necessary changes in the clinical workflow of all centers, this presents a valuable area for future research. For the walk-in hour, it would be beneficial to explore whether offering sessions multiple times a week could enhance its accessibility and user engagement. Moreover, future research should evaluate the app\u0026rsquo;s usability in other participating countries to account for the varying needs across end-user groups and countries. Given that the app is modular in its design (36), it would be valuable to explore which components are most relevant in specific settings (e.g., country, expertise center). This would allow certain features to be enabled or disabled, tailoring the app to those particular contexts. For instance, while the interactive features may be less relevant in the Dutch context, they could be highly valued in the other participating countries. Moreover, involving a broader range of expertise centers, stakeholders, and a diverse group of HDGECs with varying diseases stages and digital skills, will further ensure the app meets a wider range of user needs. To gain a more comprehensive understanding of the HSA and its implications, future research should address the app\u0026rsquo;s impact on practical outcomes for HDGECs and their partners (e.g., information provision, travel time, accessibility to care, navigating care options) as well as on care outcomes (e.g., quality of care, knowledge exchange, cost-benefit). With the app\u0026rsquo;s potential to enhance current care practices, future studies should explore barriers to implementation (e.g., app\u0026rsquo;s representation of expertise centers across countries, staff availability) and identify effective implementation strategies (e.g., pinpointing key stakeholders and the optimal stages in the patient journey for the app\u0026rsquo;s introduction).\u003c/p\u003e"},{"header":"5 Conclusions","content":"\u003cp\u003eThe current study provided initial insights into the usability and feasibility of the Huntington Support App (HSA) within both a structured and real-world setting in the Netherlands. While there was considerable variation in how different participant groups evaluated the HSA, it was generally well-received by all. Future efforts should focus on addressing the identified areas for improvement and other key themes to refine the app and its implementation. Our findings highlight the importance and feasibility of a HCD approach in developing digital tools for a disease as rare and complex as HD. By aligning closely with end-users\u0026rsquo; needs, the HSA holds promise in addressing challenges related to time, distance and cost in current HD care (19). It has the potential to enhance the quality of life for those affected by HD by improving access to care and reliable information. Given the importance of these factors in the management of long-term complex conditions (53), our approach and findings could offer valuable insights for other initiatives aimed at developing eHealth services to support individuals affected by other rare neurodegenerative diseases as well.\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003e6.1 Ethics approval and consent to participate\u003c/h2\u003e\n\u003cp\u003eThe first study was part of a larger project (HEALTHE-RND HD-eHelp study) that was cleared for ethics by the medical research ethics committee of Leiden Den Haag Delft in the Netherlands (file number: N20.013). The amendment for the second study did not require approval from the medical ethics committee, however, it was reviewed and approved by the scientific overview committee of the Department of Public Health and Primary care of Leiden University Medical Center. (Digital) informed consent was signed by all participants prior to the start of the studies.\u003c/p\u003e\n\u003ch2\u003e6.2 Consent for publication\u003c/h2\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003ch2\u003e6.3 Availability of data and materials\u003c/h2\u003e\n\u003cp\u003eThe data will not be shared publicly to protect the anonymity and privacy of participants due to the rarity of HD. Pseudonymized data supporting the findings of this study can be accessed upon reasonable request from the corresponding author (PL).\u003c/p\u003e\n\u003ch2\u003e6.4 Competing interests\u003c/h2\u003e\n\u003cp\u003eSusanne T. de Bot: Leiden University Medical Center receives funding from the European Huntington’s Disease Network (EHDN) and Cure HD Initiative (CHDI), is involved in the EU Horizon 2020 project: Innovative Medicines Initiative (IMI) 2 (IDEA_FAST), and takes part in clinical trials sponsored by PRILENIA, PTC Therapeutics, WAVE, and VICO Therapeutics. Susanne T. de Bot is member of the HD Expert Advisory Panel for the PTC518 phase 3 program (PTC Therapeutics). None of these sponsors were involved in the design, execution, interpretation, or writing of this study. The other authors have no conflicts of interest to declare. \u003c/p\u003e\n\u003ch2\u003e6.5 Funding\u003c/h2\u003e\n\u003cp\u003eThis research was supported by the EU Joint Programme Neurodegenerative Disease Research (JPND) (grant number: 01ED1903) and locally by the Netherlands Organisation for Health Research and Development (ZonMw) (project number: 733051085). The funders had no involvement in the study’s design, data collection, analysis, interpretation, decision to publish, or manuscript preparation.\u003c/p\u003e\n\u003ch2\u003e6.6 Authors’ contributions\u003c/h2\u003e\n\u003cp\u003eFunding acquisition: EM, NC, and the HEALTHE-RND consortium; Conceptualization: EM, AH, NV, LS and PL; Data collection: LS. Formal analysis: LS and PL. Writing – original draft: PL. Writing – review \u0026amp; editing: LS, AH, EM, ED, SB, NC, NV, and PL. All authors read and approved the final manuscript.\u003c/p\u003e\n\u003ch2\u003e6.7 Acknowledgments\u003c/h2\u003e\n\u003cp\u003eThe authors would like to thank all participants in the think aloud study for their active participation and valuable contributions. Their feedback was greatly appreciated. We also wish to thank all app visitors during the feasibility study who reviewed the app and shared their feedback. Additionally, we want to express our gratitude to the HCPs who checked the app’s content to ensure its clinical accuracy. We also like to acknowledge the outpatient clinics of several HD (expertise) centers in the Netherlands for their help in distributing flyers to promote the Huntington Support App for the feasibility study: Land van Horne, Archipel, Willem Drees-Oostpoort, Noorderbreedte, Topaz Overduin, Kloosterhoeve and the university medical centers of Maastricht, Groningen and Leiden.\u003c/p\u003e\n\u003ch3\u003e6.7.1 Collaborating author names HEALTHE-RND consortium\u003c/h3\u003e\n\u003cp\u003eThe Netherlands: Niels H. Chavannes, Eline Meijer, Anne- Wil Heemskerk, Erik van Duijn, Susanne T. de Bot, Pearl J. C. van Lonkhuizen, Niko Vegt, Leanne Slutter; Germany: G. Bernhard Landwehrmeyer, Alzbeta Mühlbäck, Wiebke Frank, Franziska Steck; Czech Republic: Jiří Klempíř, Romama Konvalinková, Eva Bezuchová, Kristýna Dolečková, Olga Klempířová, Jan Roth, Olga Ulmanová; Italy: Ferdinando Squitieri, Sabrina Maffi, Giulia Giancaterino, Chiara Di Giorgio, Barbara D’Alessio, Melissa Casella; Ireland: Jennifer Hoblyn, Muthukumaran Thangaramanujam, Tom Burke, Emer O’Malley; United Kingdom: Stephen McKenna, Ian McKenna, Jeanette Thorpe, Ellie Johnstone, Isobel Spray, Mariusz Grzeda, Ramona Moldovan, Peter Foley, Jacqueline Kerr. \u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eMa Y, Zhao C, Zhao Y, Lu J, Jiang H, Cao Y, et al. Telemedicine application in patients with chronic disease: a systematic review and meta-analysis. BMC Med Inform Decis Mak. 2022;22(1):105; doi:10.1186/s12911-022-01845-2.\u003c/li\u003e\n\u003cli\u003eRenzi E, Baccolini V, Migliara G, De Vito C, Gasperini G, Cianciulli A, et al. The Impact of eHealth Interventions on the Improvement of Self-Care in Chronic Patients: An Overview of Systematic Reviews. Life (Basel). 2022;12(8); doi:10.3390/life12081253.\u003c/li\u003e\n\u003cli\u003eEysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20; doi:10.2196/jmir.3.2.e20.\u003c/li\u003e\n\u003cli\u003eDorsey ER, Glidden AM, Holloway MR, Birbeck GL, Schwamm LH. Teleneurology and mobile technologies: the future of neurological care. Nat Rev Neurol. 2018;14(5):285-97; doi:10.1038/nrneurol.2018.31.\u003c/li\u003e\n\u003cli\u003evan der Kleij R, Kasteleyn MJ, Meijer E, Bonten TN, Houwink EJF, Teichert M, et al. SERIES: eHealth in primary care. Part 1: Concepts, conditions and challenges. Eur J Gen Pract. 2019;25(4):179-89; doi:10.1080/13814788.2019.1658190.\u003c/li\u003e\n\u003cli\u003eChirra M, Marsili L, Wattley L, Sokol LL, Keeling E, Maule S, et al. Telemedicine in Neurological Disorders: Opportunities and Challenges. Telemed J E Health. 2019;25(7):541-50; doi:10.1089/tmj.2018.0101.\u003c/li\u003e\n\u003cli\u003eColon-Semenza C, Latham NK, Quintiliani LM, Ellis TD. Peer Coaching Through mHealth Targeting Physical Activity in People With Parkinson Disease: Feasibility Study. JMIR Mhealth Uhealth. 2018;6(2):e42; doi:10.2196/mhealth.8074.\u003c/li\u003e\n\u003cli\u003eEspay AJ, Bonato P, Nahab FB, Maetzler W, Dean JM, Klucken J, et al. Technology in Parkinson\u0026apos;s disease: Challenges and opportunities. Mov Disord. 2016;31(9):1272-82; doi:10.1002/mds.26642.\u003c/li\u003e\n\u003cli\u003eLakshminarayana R, Wang D, Burn D, Chaudhuri KR, Galtrey C, Guzman NV, et al. Using a smartphone-based self-management platform to support medication adherence and clinical consultation in Parkinson\u0026apos;s disease. NPJ Parkinsons Dis. 2017;3:2; doi:10.1038/s41531-016-0003-z.\u003c/li\u003e\n\u003cli\u003eNissen RM, Hersch G, Tietze M, Chang PJ. Persons With Dementia and Their Caregivers\u0026apos; Perceptions About Occupational Therapy and Telehealth: A Qualitative Descriptive Study. Home Healthc Now. 2018;36(6):369-78; doi:10.1097/NHH.0000000000000697.\u003c/li\u003e\n\u003cli\u003eRathnayake S, Moyle W, Jones C, Calleja P. mHealth applications as an educational and supportive resource for family carers of people with dementia: An integrative review. Dementia (London). 2019;18(7-8):3091-112; doi:10.1177/1471301218768903.\u003c/li\u003e\n\u003cli\u003eMonje MHG, Grosjean S, Srp M, Antunes L, Bouca-Machado R, Cacho R, et al. Co-Designing Digital Technologies for Improving Clinical Care in People with Parkinson\u0026apos;s Disease: What Did We Learn? Sensors (Basel). 2023;23(10); doi:10.3390/s23104957.\u003c/li\u003e\n\u003cli\u003eRussell RD, Black LJ, Purdue J, Daly A, Begley A. A collaborative approach to designing an online nutrition education program for people with multiple sclerosis. Disabil Rehabil. 2024;46(5):947-56; doi:10.1080/09638288.2023.2186499.\u003c/li\u003e\n\u003cli\u003eWang G, Marradi C, Albayrak A, van der Cammen TJM. Co-designing with people with dementia: A scoping review of involving people with dementia in design research. Maturitas. 2019;127:55-63; doi:10.1016/j.maturitas.2019.06.003.\u003c/li\u003e\n\u003cli\u003eHatem S, Long JC, Best S, Fehlberg Z, Nic Giolla Easpaig B, Braithwaite J. Mobile Apps for People With Rare Diseases: Review and Quality Assessment Using Mobile App Rating Scale. J Med Internet Res. 2022;24(7):e36691; doi:10.2196/36691.\u003c/li\u003e\n\u003cli\u003eLong JC, Best S, Nic Giolla Easpaig B, Hatem S, Fehlberg Z, Christodoulou J, et al. Needs of people with rare diseases that can be supported by electronic resources: a scoping review. BMJ Open. 2022;12(9):e060394; doi:10.1136/bmjopen-2021-060394.\u003c/li\u003e\n\u003cli\u003eGreenhalgh T, A\u0026apos;Court C, Shaw S. Understanding heart failure; explaining telehealth - a hermeneutic systematic review. BMC Cardiovasc Disord. 2017;17(1):156; doi:10.1186/s12872-017-0594-2.\u003c/li\u003e\n\u003cli\u003evan Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111; doi:10.2196/jmir.1672.\u003c/li\u003e\n\u003cli\u003eWhitehead L, Seaton P. The Effectiveness of Self-Management Mobile Phone and Tablet Apps in Long-term Condition Management: A Systematic Review. J Med Internet Res. 2016;18(5):e97; doi:10.2196/jmir.4883.\u003c/li\u003e\n\u003cli\u003eBates GP, Dorsey R, Gusella JF, Hayden MR, Kay C, Leavitt BR, et al. Huntington disease. Nat Rev Dis Primers. 2015;1:15005; doi:10.1038/nrdp.2015.5.\u003c/li\u003e\n\u003cli\u003eBates GP, Tabrizi SJ, Jones L. Huntington\u0026rsquo;s Disease. 4 ed. New York, NY: Oxford University Press; 2014.\u003c/li\u003e\n\u003cli\u003eMcColgan P, Tabrizi SJ. Huntington\u0026apos;s disease: a clinical review. Eur J Neurol. 2018;25(1):24-34; doi:10.1111/ene.13413.\u003c/li\u003e\n\u003cli\u003eRoos RA. Huntington\u0026apos;s disease: a clinical review. Orphanet J Rare Dis. 2010;5:40; doi:10.1186/1750-1172-5-40.\u003c/li\u003e\n\u003cli\u003eReilmann R, Leavitt BR, Ross CA. Diagnostic criteria for Huntington\u0026apos;s disease based on natural history. Mov Disord. 2014;29(11):1335-41; doi:10.1002/mds.26011.\u003c/li\u003e\n\u003cli\u003eRoss CA, Reilmann R, Cardoso F, McCusker EA, Testa CM, Stout JC, et al. Movement Disorder Society Task Force Viewpoint: Huntington\u0026apos;s Disease Diagnostic Categories. Mov Disord Clin Pract. 2019;6(7):541-6; doi:10.1002/mdc3.12808.\u003c/li\u003e\n\u003cli\u003evan Lonkhuizen PJC, Frank W, Heemskerk AW, van Duijn E, de Bot ST, Muhlback A, et al. Quality of life, health-related quality of life, and associated factors in Huntington\u0026apos;s disease: a systematic review. J Neurol. 2023;270(5):2416-37; doi:10.1007/s00415-022-11551-8.\u003c/li\u003e\n\u003cli\u003eAnderson KE, Griffin J, Kinel A, Shaikh AR, Olofintuyi T, Ramirez S, et al. Quality of Care for Huntington\u0026apos;s Disease in the United States: Findings from a National Survey of Patients and Caregivers. J Huntingtons Dis. 2019;8(4):509-19; doi:10.3233/JHD-190380.\u003c/li\u003e\n\u003cli\u003eChase CL, Yashar BM, Swope C, Albin RL, Uhlmann WR. Searching for Answers: Information-Seeking by Young People At-Risk for Huntington\u0026apos;s Disease. J Huntingtons Dis. 2022;11(3):337-46; doi:10.3233/JHD-210523.\u003c/li\u003e\n\u003cli\u003ede Man J, de Jager L, Kleiterp J, Engelhard D, Achterberg W. K10 E-health support for 841 community residing HD patients: report of a pilot-project. Journal of Neurology, Neurosurgery \u0026amp; Psychiatry. 2016;87:A82-A3.\u003c/li\u003e\n\u003cli\u003eBull MT, Darwin K, Venkataraman V, Wagner J, Beck CA, Dorsey ER, et al. A pilot study of virtual visits in Huntington disease. J Huntingtons Dis. 2014;3(2):189-95; doi:10.3233/JHD-140102.\u003c/li\u003e\n\u003cli\u003eFrich JC, Rae D, Roxburgh R, Miedzybrodzka ZH, Edmondson M, Pope EB, et al. Health Care Delivery Practices in Huntington\u0026apos;s Disease Specialty Clinics: An International Survey. J Huntingtons Dis. 2016;5(2):207-13; doi:10.3233/JHD-160192.\u003c/li\u003e\n\u003cli\u003eHawkins AK, Creighton S, Ho A, McManus B, Hayden MR. Providing predictive testing for Huntington disease via telehealth: results of a pilot study in British Columbia, Canada. Clin Genet. 2013;84(1):60-4; doi:10.1111/cge.12033.\u003c/li\u003e\n\u003cli\u003eHawkins Virani AK, Creighton SM, Hayden MR. Developing a comprehensive, effective patient-friendly website to enhance decision making in predictive testing for Huntington disease. Genet Med. 2013;15(6):466-72; doi:10.1038/gim.2012.149.\u003c/li\u003e\n\u003cli\u003eMohanty D, Schmitt P, Dixon L, Holiday V, Hedera P. Patient and Caregiver Perspectives on Telehealth Use in a Multidisciplinary Huntington\u0026apos;s Disease Clinic: A Single-Institution Experience. J Huntingtons Dis. 2022;11(4):415-9; doi:10.3233/JHD-220547.\u003c/li\u003e\n\u003cli\u003evan Lonkhuizen PJC, Vegt NJH, Meijer E, van Duijn E, de Bot ST, Klempir J, et al. Study Protocol for the Development of a European eHealth Platform to Improve Quality of Life in Individuals With Huntington\u0026apos;s Disease and Their Partners (HD-eHelp Study): A User-Centered Design Approach. Front Neurol. 2021;12:719460; doi:10.3389/fneur.2021.719460.\u003c/li\u003e\n\u003cli\u003evan Lonkhuizen PJC, Heemskerk AW, Meijer E, van Duijn E, de Bot ST, Klempir J, et al. Development of the Huntington Support App (HD-eHelp study): a human-centered and co-design approach. Front Neurol. 2024.\u003c/li\u003e\n\u003cli\u003eLandwehrmeyer GB, Fitzer-Attas CJ, Giuliano JD, Goncalves N, Anderson KE, Cardoso F, et al. Data Analytics from Enroll-HD, a Global Clinical Research Platform for Huntington\u0026apos;s Disease. Mov Disord Clin Pract. 2017;4(2):212-24; doi:10.1002/mdc3.12388.\u003c/li\u003e\n\u003cli\u003eArsand E, Demiris G. User-centered methods for designing patient-centric self-help tools. Inform Health Soc Care. 2008;33(3):158-69; doi:10.1080/17538150802457562.\u003c/li\u003e\n\u003cli\u003eBrooke J. SUS: a quick and dirty usability scale. Usability Eval Ind. 1995;189:1-6.\u003c/li\u003e\n\u003cli\u003eLewis JR, Sauro J. The Factor Structure of the System Usability Scale. In: Kurosu, M. (eds) Human Centered Design. . HCD 2009; Berlin2009.\u003c/li\u003e\n\u003cli\u003eGraneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105-12; doi:10.1016/j.nedt.2003.10.001.\u003c/li\u003e\n\u003cli\u003eHyzy M, Bond R, Mulvenna M, Bai L, Dix A, Leigh S, et al. System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR Mhealth Uhealth. 2022;10(8):e37290; doi:10.2196/37290.\u003c/li\u003e\n\u003cli\u003eMaramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: A scoping review. Int J Med Inform. 2019;126:95-104; doi:10.1016/j.ijmedinf.2019.03.018.\u003c/li\u003e\n\u003cli\u003eKelly L, Ziebland S, Jenkinson C. Measuring the effects of online health information: Scale validation for the e-Health Impact Questionnaire. Patient Educ Couns. 2015;98(11):1418-24; doi:10.1016/j.pec.2015.06.008.\u003c/li\u003e\n\u003cli\u003eNeijenhuijs KI, van der Hout A, Veldhuijzen E, Scholten-Peeters GGM, van Uden-Kraan CF, Cuijpers P, et al. Translation of the eHealth Impact Questionnaire for a Population of Dutch Electronic Health Users: Validation Study. J Med Internet Res. 2019;21(8):e13408; doi:10.2196/13408.\u003c/li\u003e\n\u003cli\u003ede Man J, de Jager L, Kleiterp J, Engelhard D, Achterberg W. K10 E-health support for community residing HD patients: report of a pilot-project. Journal of Neurology, Neurosurgery \u0026amp; Psychiatry. 2016;87(Suppl 1):A82-A3; doi:10.1136/jnnp-2016-314597.233.\u003c/li\u003e\n\u003cli\u003eHawkins AK, Creighton SM, Hayden MR. Developing a comprehensive, effective patient-friendly website to enhance decision making in predictive testing for Huntington disease. Genetics in Medicine. 2013;15(6):466-72.\u003c/li\u003e\n\u003cli\u003eGranja C, Janssen W, Johansen MA. Factors Determining the Success and Failure of eHealth Interventions: Systematic Review of the Literature. J Med Internet Res. 2018;20(5):e10235; doi:10.2196/10235.\u003c/li\u003e\n\u003cli\u003eNijland N, van Gemert-Pijnen J, Boer H, Steehouder MF, Seydel ER. Evaluation of internet-based technology for supporting self-care: problems encountered by patients and caregivers when using self-care applications. J Med Internet Res. 2008;10(2):e13; doi:10.2196/jmir.957.\u003c/li\u003e\n\u003cli\u003eRuggiano N, Brown EL, Shaw S, Geldmacher D, Clarke P, Hristidis V, et al. The Potential of Information Technology to Navigate Caregiving Systems: Perspectives from Dementia Caregivers. J Gerontol Soc Work. 2018:1-19; doi:10.1080/01634372.2018.1546786.\u003c/li\u003e\n\u003cli\u003eTinschert P, Jakob R, Barata F, Kramer JN, Kowatsch T. The Potential of Mobile Apps for Improving Asthma Self-Management: A Review of Publicly Available and Well-Adopted Asthma Apps. JMIR Mhealth Uhealth. 2017;5(8):e113; doi:10.2196/mhealth.7177.\u003c/li\u003e\n\u003cli\u003eShah SG, Robinson I. User involvement in healthcare technology development and assessment: structured literature review. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2006;19(6-7):500-15; doi:10.1108/09526860610687619.\u003c/li\u003e\n\u003cli\u003eBattersby M, Lawn S, Pols R. Conceptualisation of self-management. . In: Kralik D, Paterson B, Coates V, editors. Translating Chronic Illness Research Into Practice Hoboken, NJ: Wiley-Blackwell; 2010. p. 115-201.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Footnote","content":"\u003cp\u003e\u003csup\u003e[1] \u003c/sup\u003eThe follow-up questionnaire was intended to assess the usability of the app after a period of four weeks, including items on frequency of use, satisfaction with the app, as well as part 2 of the Dutch version of the eHealth Impact Questionnaire (eHIQ) (44, 45). Part 2 of the eHIQ contains 26 validated items on the attitudes of users towards a specific eHealth application\u0026nbsp;(e.g., \u003cem\u003e\u0026ldquo;I can easily understand the information on the website\u0026rdquo;\u0026nbsp;\u003c/em\u003eor \u0026ldquo;\u003cem\u003eThe website gives me the confidence to explain my health concerns to others\u0026rdquo;\u003c/em\u003e), rated on a five-point Likert scale ranging from \u0026lsquo;strongly disagree\u0026rsquo; to \u0026lsquo;strongly agree\u0026rsquo; (44).\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Huntington’s disease, rare diseases, neurodegenerative diseases, telemedicine, eHealth, human-centered design, tele-neurology, digital health","lastPublishedDoi":"10.21203/rs.3.rs-5440332/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5440332/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground: \u003c/strong\u003eeHealth holds great promise for managing rare and complex neurodegenerative diseases, such as Huntington’s disease (HD). To address this potential, the Huntington Support App (HSA) was developed to provide reliable information on HD, news updates, and contact possibilities tailored primarily to the needs of HD gene expansion carriers (HDGECs) and their partners. Building on prior research outlining the HSA’s co-creation process with HDGECs, partners and health care providers (HCPs), the present study offers an in-depth mixed-methods evaluation of the HSA in practice in the Netherlands.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003eThis research comprises a multi-study report assessing the usability of the HSA through a think-aloud method (Study 1) and subsequent feasibility testing in a real-world setting (Study 2). In the first study, 4 HDGECs, 4 partners, and 4 HCPs, who themselves were not involved in the HSA co-design, provided real-time feedback on usability while using the HSA. In the feasibility study the HSA was publicly launched and made available to the larger HD community for feedback (including HDGECs, partners, HCPs and family/friends, n=13). In both studies, the System Usability Scale (SUS) was administered, and quantitative data were triangulated with qualitative findings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e Overall, participants were positive about the app’s usability in both studies. App ratings in the first study were notably higher than those in the second study, with participants’ SUS scores in the first study far exceeding the benchmark usually seen in usability testing for healthcare apps. Furthermore, in both studies there were variations in ratings among participant groups, with HDGECs, partners and family/friends viewing the app more favorably than HCPs, who generally expressed a more neutral opinion. Qualitative responses further complemented these findings and highlighted suggestions for the app’s improvement, particularly in terms of content and navigation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003eThe HSA was well-received by members of the HD community, indicating alignment with their needs. Future efforts should address identified areas for improvement of the app. Our findings underscore the importance and feasibility of a participatory, human-centered design approach for complex and rare diseases like HD and highlight its relevance for future co-design initiatives of eHealth services for other rare neurodegenerative diseases.\u003c/p\u003e","manuscriptTitle":"Usability and feasibility assessment of the Huntington Support App (HD-eHelp study): a mixed-methods study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-17 18:59:57","doi":"10.21203/rs.3.rs-5440332/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-12-16T10:02:22+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-26T14:42:58+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-25T11:29:59+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"214804997449869897691912263477318911474","date":"2024-11-19T11:23:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"302277407619240453811559249683031228230","date":"2024-11-19T10:44:16+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-11-19T09:19:36+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-11-13T14:27:51+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-11-13T12:05:21+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Digital Health","date":"2024-11-12T14:18:44+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"639aad0c-1c5c-413c-8afc-6f864cb1899a","owner":[],"postedDate":"December 17th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-06-02T13:08:16+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-17 18:59:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5440332","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5440332","identity":"rs-5440332","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}