Appropriateness and Utility of a Clinical Decision Support System at the Digital Front Door

Appropriateness and Utility of a Clinical Decision Support System at the Digital Front Door | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Appropriateness and Utility of a Clinical Decision Support System at the Digital Front Door Andreia Pimenta, Nisha Kini, Fabienne Cotte, Filipa Dias Lourenço, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8157860/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background: Digital front doors that combine symptom assessment and self-triage are increasingly used to guide patients to appropriate care, yet real-world evidence on safety, performance, and workflow impact remains limited and heterogeneous. This study reports a post-market clinical follow-up (PMCF) evaluation of a clinical decision support system (CDSS), integrated into routine care within Portugal’s largest private healthcare network (CUF) to determine whether it can deliver appropriate urgency advice and clinically useful reports, and whether its use is associated with improved preparedness and consultation efficiency in real-world practice. Methods: This was a prospective, observational study. Adults aged ≥ 18 years completed a full symptom assessment via the myCUF application before consultation. Participants provided informed consent and completed a post-assessment survey; treating physicians completed structured post-consultation surveys. Primary endpoints were the appropriateness of urgency advice and of the assessment report, including the reasonableness of suggested conditions, completeness of symptom history, concordance of the main problem with the consultation focus, and inclusion of the physician’s final diagnosis. Secondary endpoints included consultation efficiency and time saving, clinician preparedness, confidence in diagnosis, and perceived usefulness of the assessment report information. Analyses used descriptive statistics, confidence intervals, risk differences and risk ratios, with subgroup comparisons by specialty, sex, and age. Results: A total of 1,470 participants completed the post-assessment survey. Physicians returned 163 consultation surveys, of which 60 were based on reports reviewed before the encounter and formed the basis for efficiency and preparedness analyses. Urgency advice was judged appropriate in most cases (top advice level: 74.1%, 120/161; 95% CI 67.4–80.1; all advice levels: 77.6%, 125/161; 95% CI 70.1–83.9), with no significant differences by specialty, sex, or age. In cases where advice was not considered appropriate (8.7%; 95% CI 5.0–13.9), all disagreements reflected conservative, over-cautious advice; no instances of under-triage were identified. Report appropriateness was high: conditions provided as suggestions were judged reasonable in 73.0% (119/163; 95% CI 65.7–79.5), the symptom list was considered complete in 78.4% (127/162; 95% CI 71.2–84.4), the main problem discussed in the consultation matched the information in the report in 82.8% (135/163; 95% CI 76.1–88.2), and the clinician’s most likely diagnosis was included among the suggested conditions in 80.4% (131/163; 95% CI 73.5–86.0). Physicians reported efficiency improvements in 66.7% (40/60; 95% CI 53.3–78.3) and time savings in 66.1% (39/59; 95% CI 52.3–77.6), most often during the consultation itself. Increased preparedness for the consultation was reported in 71.7% (43/60; 95% CI 58.6–82.5) and was strongly associated with the perceived usefulness of the condition suggestions. Among participants, 80.5% (1,183/1,470; 95% CI 78.4–82.5) reported that all symptoms were captured in the assessment report, 71.2% (1,046/1,470; 95% CI 68.7–73.6) felt more prepared for the consultation, and 33.0% (485/1,470; 95% CI 30.5–35.6) reported reduced anxiety after assessment completion. Completeness and preparedness were associated with lower anxiety. Conclusion: In routine practice, an integrated symptom assessment device provided appropriate urgency advice and clinically useful reports, and was associated with greater clinician preparedness, perceived efficiency improvements, and positive patient experience. These PMCF findings support continued safe use as a digital front door and contribute real-world evidence for ongoing conformity assessment of Ada, a clinical decision support system and a Class IIa medical device under EU MDR 2017/745. Health sciences/Diseases Health sciences/Health care Health sciences/Medical research digital front door symptom assessment self-triage urgency advice clinical workflow consultation efficiency clinician preparedness patient experience post-market clinical follow-up Figures Figure 1 Introduction Health systems face mounting pressure from population ageing, multimorbidity, and persistent workforce shortages. These trends increase demand while restricting the supply of clinical time, lengthening queues, and threatening timely access to appropriate care. Delayed or misdirected access has been associated with poorer outcomes, including avoidable hospitalisations and excess mortality, underscoring the need for scalable approaches that preserve both quality and safety. 1 , 2 Efforts to address these pressures can be grouped across three phases of the care pathway. Before the visit, patients require support to determine whether care is needed, where to seek it, and how urgently; in some cases, empowering them with reliable information may enable safe self-management. During the visit, clinicians benefit from tools that improve efficiency and quality of the encounter. High documentation burden remains a persistent challenge: in acute care, overcrowding, often driven by non-urgent presentations, forces clinicians to take and record histories under time pressure, with frequent interruptions that increase the risk of omissions. 3 Documentation quality varies with experience: junior physicians may take three times longer to complete histories, while senior physicians are faster but sometimes less complete. 4 Overstretching under these conditions is linked to higher error rates. 5 Structured, interactive, and pre-populated records have therefore been proposed as part of the solution, with digital history-taking tools offering the potential to improve clarity, accuracy, and completeness while reducing burden. 6 After the visit, technologies for monitoring and asynchronous communication can reduce unnecessary attendances, support adherence, and promote continuity. The shared goal across phases is to optimise use of healthcare resources, enhance the experience of both patients and professionals, and improve outcomes. Technology is increasingly being explored as a lever across these phases. AI-based scribe tools aim to reduce documentation burden and free time for patient interaction. Machine learning has been applied to clinical decision support and error reduction. 7 Large language models are being evaluated for summarisation, triage, and diagnostic support, though careful integration is required to ensure reliability. 8 A particular class of tools with relevance at the entry point to care are the so-called digital front doors (DFD). These patient-facing, digitally enabled platforms combine symptom assessment, self-triage, booking, and sometimes remote consultations to guide people to the right service at the right time. 9 , 10 While conceptually spanning all three phases, most implementations focus on the pre-visit stage and preparation for the consultation. The evidence on DFDs based on CDSS remains mixed. Systematic reviews report wide variation in diagnostic accuracy and appropriateness of urgency advice, with performance influenced by presentation type and evaluation method. 11 , 12 Much of the literature is based on vignette studies or specialty-specific pilots, limiting generalisability. Real-world evaluations across broader patient populations and specialties remain scarce. Against this backdrop, the ESSENCE (E-health Self Symptom assEssmeNt as a front door and facilitator of CarE) post-market clinical follow-up study evaluates a regulated symptom assessment device integrated into routine practice in Portugal. In contrast to our companion analysis focusing on health-seeking behaviour, this paper examines the appropriateness of the information provided and its impact on consultation quality and effectiveness, asking whether the tool supports safe and appropriate guidance, strengthens decision-making and care navigation, and enables more prepared and efficient consultations. Objectives To determine whether a CDSS integrated as DFD can deliver appropriate urgency advice and clinically useful reports, and whether it is associated with improved preparedness and consultation efficiency in real-world practice. Methods Study Design and Setting This paper reports results from ESSENCE, a prospective quality-improvement evaluation of a digital clinical decision support system developed by Ada Health (Berlin, Germany) and implemented in Portugal’s largest private healthcare network (CUF), integrated through the myCUF application. ‘Ada Assess’ (commonly known as ‘Ada’), manufactured by Ada Health GmbH (Berlin, Germany), is a CE-marked Class IIa medical device under the EU MDR 2017/745. Adults (≥ 18 years) using the myCUF app to complete a full assessment for themselves were eligible. Inclusion required completion of pre- and post-assessment questions, informed consent, and report sharing with a healthcare provider. Participants were instructed not to use the CDSS in emergency situations as per device intended purpose as described in the instructions for use and disclaimers. Enrolment ran from November 2023 to October 2024. Data Collection Procedures The full ESSENCE protocol has been reported previously. 13 In brief, participants responded to an intended question about their planned healthcare-seeking behaviour before completing the self-assessment through the CDSS. The assessment generated a structured report including suggested conditions based on user provided information, a summary of entered symptoms, and urgency advice. Each suggested condition was paired with a condition-specific urgency recommendation, expressed using eight-point scale (Self-care, Self-care with pharmacy support, Primary care in 2–3 weeks, Primary care in 2–3 days, Primary care on the same day, Primary care within 4 hours, Emergency care, and Call ambulance). Physicians were asked to rate the appropriateness of urgency advice at two levels: (i) whether, based on the patient’s symptoms, the recommended advice for the most likely (top suggested) condition was reasonable, and (ii) whether the urgency advice levels across all suggested conditions were reasonable. In addition, the system generated an overall advice level (highest acuity across all advice suggestions), which was linked to the CUF care navigation options (Call Ambulance, Emergency Care, Video call with a General Practitioner). This overall advice level represented the guidance presented to the patient as the appropriate next step in care. Immediately after the assessment, participants completed a survey, which captured completeness of symptom entry, preparedness for the upcoming consultation, and perceived psychological effects (e.g. anxiety). Following the consultation, treating physicians were instructed to complete a structured survey immediately after seeing the patient, or at the latest by the end of the same working day. Where this was not possible, surveys were completed retrospectively using the patient record. In these retrospective cases, only a reduced set of items was addressed, focusing on the appropriateness of the report information (e.g. urgency advice, reasonableness of suggested conditions, completeness of the symptom list, and concordance of the report with the consultation focus and final diagnosis), whereas questions relating to the effect of the report on the consultation process itself were excluded. Physicians also reported whether the report prompted additional diagnostic tests or flagged critical findings. Further items evaluated consultation efficiency, time savings in specific activities, and perceptions of patient engagement and being informed. Participant and physician surveys combined Likert-type scales, categorical response options, and free-text fields. Full survey instruments are available in Appendix 2. Data Analysis The ESSENCE study had two predefined primary endpoints: (i) the appropriateness of the urgency advice, assessed through two survey items in which physicians rated whether the advice for the most probable condition and overall advice levels were reasonable; and (ii) the appropriateness of the assessment report, evaluated across multiple dimensions including the reasonableness of suggested conditions, completeness of the symptom list, concordance of the main problem with the consultation focus, inclusion of the physician’s most likely diagnosis among the suggested conditions, and whether the report provided useful information, prompted additional tests, or identified critical findings. Survey responses were captured on a five-point Likert scale from Strongly disagree to Strongly agree and dichotomised for analysis, with Agree and Strongly agree considered appropriate. Relationships between survey measures such as completeness, reasonableness, and perceived usefulness and physician-reported outcomes (efficiency, preparedness, perceived patient effects) were examined descriptively or, where appropriate, using the tests described above. For participant-reported outcomes, associations between completeness, preparedness, and anxiety reduction were evaluated similarly. Missing data were handled by excluding cases on a per-item basis. Continuous variables were summarised as means with standard deviations or as medians with interquartile ranges, depending on distribution, and categorical variables were described using frequencies and percentages. Comparisons across subgroups (specialty, sex, and age group) were performed using Pearson’s χ² test or Fisher’s exact test, as appropriate, with a two-sided significance level of 0.05. For specialty-level analyses, specialties with fewer than ten consultations were grouped under Other to enable meaningful comparison. Ninety-five per cent confidence intervals for proportions were calculated using standard binomial methods. Confidence intervals for risk differences and risk ratios were reported where relevant. All analyses were conducted in Python (v3.10). Data Management and Ethics Data was securely stored in a validated electronic system (Teamscope) using unique identifiers. Ethical approval was granted by the Comissão de Ética para a Investigação Clínica (No. 2204JJ351 and No. 2309JJ660 ). The study was registered at ClinicalTrials.gov ( NCT06846957 ) - registry date: February, 26th, 2025, and complied with the Declaration of Helsinki and ISO 14155:2020 guidelines. STROBE guidelines were followed throughout. Results Study Population A total of 1,470 participants were enrolled, with a mean age of 38.5 years (SD 12.5, range 18–83), of whom 57.7% (848/1,470) were female. Participants initially entered a median of two symptoms (IQR 1–3). Cases were grouped by clinical specialty according to their presentation. Participants completed a post-assessment survey, which provided information on the completeness of symptom entry, preparedness for consultation, and psychological effects of the assessment. Physician surveys were completed for 163 consultations, in different settings: 13 in-person visits in the emergency department and 150 teleconsultations. Of the total, 103 surveys were completed retrospectively, where the physician had not reviewed the report before the consultation, and 60 were completed immediately after consultations in which the report was reviewed in advance. The latter group (n = 60) formed the basis for analyses of consultation efficiency and preparedness. The participant inclusion process is outlined in a STROBE flow diagram (Fig. 1 ). Reasonability of urgency advice Physicians agreed that the urgency advice level associated with the most probable condition was reasonable in 74.1% of cases (120/161; 95% CI 67.4–80.1). No differences were observed by case specialty (p = 0.535), sex (p = 0.237) or age group (p = 1.000) (Table 1 , Appendix Table 1A and 1C). For each case, physicians were asked whether all urgency advice levels provided by the assessment report, i.e. condition-specific urgency recommendations, were reasonable; in 77.6% (125/161; 95% CI 70.1–83.9) of cases the recommendations were considered reasonable. No significant differences were found with case specialty (p = 0.107), sex (p = 0.710), or age group (p = 0.685) (Table 1 , Appendix Tables 1B and 1C). Table 1 Appropriateness of urgency advice (* Agree = Strongly Agree or Agree). Values are numbers and percentages of cases judged appropriate by physicians for both the top (most probable condition) and all urgency advice levels generated by the CDSS. Comparisons across specialties, sex, and age groups were performed using Pearson’s χ² or Fisher’s exact test as appropriate. Most probable condition advice recommendation All condition-specific recommendations n/N (% agree)* 95% CI n/N (% agree)* 95% CI Speciality p = 0.535 (top), 0.107 (all) Dermatology 7/10 (70.0) 34.8–93.3 5/10 (50.0) 23.7–76.3 Gastroenterology 15/23 (65.2) 43.4–83.0 16/22 (72.7) 49.8–89.3 Gynaecology 22/26 (84.6) 65.1–95.6 23/26 (88.5) 69.8–97.6 Orthopaedics / Trauma 13/20 (65.0) 40.8–84.6 14/20 (70.0) 45.7–88.1 Otorhinolaryngology / Ear Nose Throat 35/47 (74.5) 59.7–86.1 38/48 (79.2) 64.0–90.0 Pulmonology 10/11 (90.9) 58.7–99.8 11/11 (100.0) 71.5–100 Other 18/23 (78.3) 56.3–92.5 18/24 (75.0) 53.3–90.2 Total 120/161 (74.1) 67.4–80.1 125/161 (77.6) 70.1–83.9 Sex p = 0.237 (top), 0.710 (all) Female 76/97 (78.4) 68.8–86.1 76/96 (79.2) 69.6–86.9 Male 44/64 (68.8) 56.1–79.9 49/65 (75.4) 63.3–85.2 Age p = 1.000 (top), 0.685 (all) 18–64 115/154 (74.7) 67.0–81.4 118/153 (77.1) 69.4–83.7 ≥ 65 5/7 (71.4) 29.0–96.3 7/8 (87.5) 47.3–99.7 The appropriateness of the care recommendations and comparison with actual participant behaviour were analysed by overall advice level, defined as the highest-acuity recommendation generated by the CDSS for each case (for example, if both self-care and emergency care were suggested, the case was categorised under emergency care). Table 2 summarises these results. Physicians rated the overall urgency advice as appropriate in 77.6% (95% CI 70.1–83.9), neutral in 13.7% (95% CI 8.5–20.2), and not appropriate in 8.7% (95% CI 5.0–13.9) (14/161). To understand whether advice judged not appropriate nonetheless resulted in safe user behaviour, the 14 cases in which the CDSS advice was rated not appropriate were examined and compared participants’ actual care-seeking actions. In five of these cases (35.7%; 95% CI 12.8–64.9), participants’ actions matched what physicians considered appropriate, indicating that some users sought care consistent with clinical expectations even when the CDSS advice was inaccurate (Table 2 ). Table 2 Appropriateness of care recommendation with actual participant behaviour comparison. Cases were grouped by the highest-acuity recommendation (overall advice level) generated by the CDSS. Values show physician ratings of appropriateness (appropriate, neutral, not appropriate) and, for cases judged not appropriate, the patient’s actual care-seeking behaviour and whether that behaviour was considered appropriate. Percentages are within each overall-advice category. Overall advice level Appropriate n (%) 95% CI Neutral n (%) 95% CI Not appropriate n (%) 95% CI In not-appropriate cases: Participant behaviour rated appropriate n (%) † Self-care 3 (42.9) 9.9–81.6 4 (57.1) 18.4–90.1 0 (0.0) 0–35.4 – Self-care + pharmacy support 19 (86.4) 65.1–97.1 3 (13.6) 2.9–34.9 0 (0.0) 0–14.8 – Primary care in 2–3 weeks 3 (33.3) 7.5–70.1 5 (55.6) 21.2–86.3 1 (11.1) 0.3–48.2 Telemedicine (1): 0 (0.0) Primary care in 2–3 days 33 (82.5) 67.2–92.7 2 (5.0) 0.6–16.9 5 (12.5) 4.2–26.8 Telemedicine (5): 2 (40.0) Primary care same day 49 (81.7) 70.3–90.2 6 (10.0) 4.0–19.5 5 (8.3) 2.8–18.4 Telemedicine (5): 2 (40.0) Primary care within 4 h 6 (100.0) 61.0–100 0 (0.0) 0–39.0 0 (0.0) 0–39.0 – Emergency care 9 (81.8) 48.2–97.7 1 (9.1) 0.2–41.3 1 (9.1) 0.2–41.3 Telemedicine (1): 0 (0.0) Call ambulance 3 (50.0) 11.8–88.2 1 (16.7) 0.4–64.1 2 (33.3) 4.3–77.7 Telemedicine (2): 1 (50.0) Total (n = 161) 125 (77.6) 70.1–83.9 22 (13.7) 8.5–20.2 14 (8.7) 5.0–13.9 5 (35.7) (95% CI 12.8–64.9) †Alignment compares the action taken (Service Visited) with the physician’s judgement of appropriate behaviour (Behaviour Appropriateness) for that case. To further characterise the nature of the inappropriate advice, within the same 14 cases, the CDSS’s highest (overall) advice level was compared with the physician’s recommended care-seeking behaviour. Physician responses for what the appropriate behaviour in 3 levels (“Managing symptoms at home or consulting a pharmacist”, “Consulting a GP” or “Consulting a specialist” and “Emergency Room / Emergency”) and were mapped onto the same eight-level CDSS advice scale (Appendix Table 2A). The CDSS recommendation coincided with the physician’s behaviour recommendation in nine cases (64.3%; 95% CI 35.1–87.2) and was more cautious in five (35.7%; 95% CI 12.8–64.9). Importantly, no instances were identified in which the CDSS provided a less urgent recommendation than physicians considered appropriate, indicating that all discrepancies reflected conservative rather than unsafe guidance. It should be noted that physicians’ recommended behaviours were captured using a simplified three-level categorisation (self-care, primary care, or emergency care), whereas the CDSS generated advice across an eight-level scale. Consequently, apparent agreements may partly reflect differences in scale granularity rather than true clinical agreement. Appropriateness of the assessment report Appropriateness of the assessment report was evaluated across multiple dimensions, including the reasonableness of suggested conditions, completeness of the symptom list, concordance of the main problem with the consultation focus, inclusion of the physician’s most probable diagnosis, provision of additional useful information, and whether the report prompted further tests or flagged critical findings (Table 3 ). Table 3 Physician-reported survey outcomes on assessment report appropriateness and consultation impact. Values are numbers and percentages of physicians selecting each response option. Comparisons across subgroups (specialty, sex, age) used Pearson’s χ² or Fisher’s exact test where applicable. Item Response n/N (%) 95% CI (%) Appropriateness of the Assessment Report Conditions reasonability All reasonable 119/163 (73.0) 65.7–79.5 Some reasonable 38/163 (23.3) 17.2–30.5 None reasonable 6/163 (3.7) 1.4–7.9 Symptom list complete Strongly agree 57/162 (35.2) 28.0–43.0 Agree 70/162 (43.2) 35.4–51.3 Neutral 18/162 (11.1) 6.7–17.0 Disagree 16/162 (9.9) 5.8–15.5 Strongly disagree 1/162 (0.6) 0.0–3.4 Main problem matched consultation Yes 135/163 (82.8) 76.1–88.2 No 28/163 (17.2) 11.8–23.9 Most probable diagnosis included Included 131/163 (80.4) 73.5–86.0 Not included 32/163 (19.6) 14.0–26.5 Additional useful information Yes 107/153 (69.9) 62.1–77.0 No 46/153 (30.1) 23.0–37.9 Additional tests prompted Yes 31/159 (19.5) 13.8–26.5 No 128/159 (80.5) 73.5–86.2 Critical findings identified Yes 52/157 (33.1) 26.0–41.0 No 105/157 (66.9) 59.0–74.0 Impact of the Assessment Report in the Consultation Usefulness of condition suggestions Very useful 28/60 (46.7) 33.7–60.0 Somewhat useful 23/60 (38.3) 26.1–51.6 Not useful 9/60 (15.0) 7.1–26.6 Efficiency improvement Strongly agree 14/60 (23.3) 13.4–36.0 Agree 26/60 (43.3) 30.6–56.8 Neutral 11/60 (18.3) 9.5–30.4 Disagree 7/60 (11.7) 4.8–22.6 Strongly disagree 2/60 (3.3) 0.6–11.3 Physician preparedness for consultation Strongly agree 9/60 (15.0) 7.1–26.6 Agree 34/60 (56.7) 43.2–69.4 Neutral 9/60 (15.0) 7.1–26.6 Disagree 4/60 (6.7) 1.8–16.2 Strongly disagree 4/60 (6.7) 1.8–16.2 Confidence in diagnosis Strongly agree 14/60 (23.3) 13.4–36.0 Agree 23/60 (38.3) 26.1–51.6 Neutral 12/60 (20.0) 10.7–32.6 Disagree 8/60 (13.3) 6.0–24.6 Strongly disagree 3/60 (5.0) 1.0–13.9 Patient more engaged in consultation Strongly agree 10/58 (17.2) 8.5–29.4 Agree 35/58 (60.3) 46.6–73.0 Neutral 7/58 (12.1) 5.0–23.3 Disagree 6/58 (10.3) 3.9–21.2 Strongly disagree 0/58 (0.0) 0–6.2 Patient more informed in consultation Strongly agree 14/59 (23.7) 13.7–36.6 Agree 30/59 (50.8) 37.6–64.0 Neutral 10/59 (16.9) 8.5–29.0 Disagree 5/59 (8.5) 2.8–18.7 Strongly disagree 0/59 (0.0) 0–6.1 In 73.0% of consultations (119/163; 95% CI 65.7–79.5), all suggested conditions were judged reasonable, while in a further 23.3% (38/163; 95% CI 17.2–30.5) at least some suggestions were considered reasonable; only 3.7% (6/163; 95% CI 1.4–7.9) were judged not reasonable. The symptom list was considered complete in 78.4% (127/162; 95% CI 71.2–84.4), and the main medical problem matched the consultation focus in 82.8% (135/163; 95% CI 76.1–88.2). Additional useful patient information was reported in 69.9% (107/153; 95% CI 62.1–77.0), the report prompted further diagnostic tests in 19.5% (31/159; 95% CI 13.8–26.5), and critical findings were flagged in 33.1% (52/157; 95% CI 26.0–41.0). The physician’s most probable diagnosis was included among the suggested conditions in 80.4% (131/163; 95% CI 73.5–86.0), with significant differences by specialty (p = 0.035), but no significant variation by sex (p = 1.000) or age group (p = 0.191) (Appendix Table 3A). Across most specialties, results were close to or above 80%, indicating generally strong correspondence between the CDSS suggestions and physicians’ clinical reasoning. Physicians were more likely to find the diagnosis included in the report when the consultation focused on the same symptoms or medical problem as those stated in the assessment: most probable diagnosis included in 87.4% (118/135; 95% CI 80.7–92.6) of such cases, compared with 46.4% (13/28; 95% CI 27.5–66.1) when the consultation addressed a different problem (p < 0.00001). This indicates that discrepancies in most likely diagnosis often arose when patients presented a different medical problem in the consultation than the one they had reported during the assessment. Consultation efficiency and time The mean consultation length, based on 59 valid responses, was 13.0 minutes (SD 5.4; median 14; range 4–28). Time savings during the consultation were reported in 66.1% (39/59; 95% CI 52.3–77.6), while 33.9% reported no time saved (20/59; 95% CI 22.4–47.7). Time was most often saved during the consultation itself (29/39; 74.4%; 95% CI 59.7–84.7). Time saving was significantly (p = 0.033) more often observed in clinical cases judged as easy (11/15; 73.3%; 95% CI 44.9–92.2) compared with moderate or difficult ones (10/26; 38.5%; 95% CI 20.2–59.4). Physicians rated the usefulness of the condition suggestions for the consultation work favourably. Across 60 valid responses, 46.7% (28/60; 95% CI 33.7–60.0) judged the report to be very useful, 38.3% (23/60; 95% CI 26.1–51.6) as somewhat useful, and 15.0% (9/60; 95% CI 7.1–26.6) as not useful. Efficiency improvement was reported by 66.7% (40/60; 95% CI 53.3–78.3) (Table 3 ). Efficiency improvements were more frequent when all suggested conditions were judged reasonable (72.3%, 34/47; 95% CI 57.4–84.4), compared with 60.0% (6/10; 95% CI 26.2–87.8) when only some were reasonable and 0.0% (0/3; 95% CI 0–70.8) when none were reasonable, although this trend did not reach statistical significance (p = 0.066). In contrast, perceived usefulness of the condition suggestions was a strong predictor of efficiency: improvements were reported in all cases rated very useful (100%, 28/28; 95% CI 87.9–100), compared with 52.2% (12/23; 95% CI 30.6–73.2) when somewhat useful and 0.0% (0/9; 95% CI 0–33.6) when not useful (p < 0.0001) (Appendix Table 4A). Preparedness for the consultation was reported by 71.7% (43/60; 95% CI 58.6–82.5). This outcome was significantly associated with perceived usefulness of the suggested conditions (p < 0.0002) but not with their reasonableness (p = 0.56) (Appendix Table 4B). Participant survey findings In the post-assessment participant survey, across the three participant-reported endpoints (symptom capture in report, preparedness for consultation, and reduction of anxiety), a joint analysis was possible for those with non-missing answers, i.e. participants without skipped questions to all three items (1,383/1,470). In this cohort, 95.7% (95% CI 94.6–96.6) of participants reported at least one positive effect (22.2% [95% CI 19.9–24.7] in one domain, 41.1% [95% CI 38.4–43.8] in two, and 32.3% [95% CI 29.8–35.0] in all three), while 4.3% (95% CI 3.4–5.4) reported no positive effect on any endpoint. Overall, 80.5% (1,183/1,470; 95% CI 78.4–82.5) reported that all symptoms they wished to include were captured in the assessment, and 71.2% (1,046/1,470; 95% CI 68.7–73.6) felt prepared for their consultation (Appendix Table 5A). Preparedness was more common among those who reported all their symptoms (80.4% vs 65.7%; risk difference + 14.7 pp, 95% CI 7.4–21.9; RR 1.22, 95% CI 1.10–1.36) (Appendix Table 5B). One-third of participants (33.0%, 485/1,470; 95% CI 30.5–35.6) reported feeling less anxious about their health after using the CDSS, while 30.9% (95% CI 28.4–33.4) reported not feeling less anxious and 29.0% (95% CI 26.6–31.5) reported no difference. No significant differences in anxiety reduction were observed by sex or age group (p = 0.49 and p = 0.46, respectively). Reporting all symptoms was associated with a higher likelihood of feeling less anxious after using the CDSS (37.8% vs 21.6%; risk difference + 16.2 pp, 95% CI 9.7–22.7; RR 1.75, 95% CI 1.32–2.32). Similarly, those who felt prepared for the consultation after completing an assessment felt less anxious (43.9% vs 8.0%; risk difference + 35.9 pp, 95% CI 31.6–40.3; RR 5.51, 95% CI 3.70–8.21) (Appendix Table 5B). Changes in intended care were related to changes in anxiety. Reduced anxiety was most common among those who de-escalated intended care (48.1%, 95% CI 43.2–53.0) and those who escalated care (44.0%, 95% CI 38.7–49.5), compared with 33.1% (95% CI 29.5–36.8) in the no-change group. Participants in the made-certain group reported reduced anxiety in 38.7% (95% CI 30.8–47.1) of cases, while those in the made-uncertain and stayed-uncertain groups reported lower rates (24.3% [95% CI 17.1–33.1] and 26.7% [95% CI 17.3–38.8], respectively). Overall differences between groups were not statistically significant (p = 0.141). Discussion This study evaluated a regulated digital clinical decision support system integrated into Portugal’s largest private healthcare network. In real-world use, the CDSS produced clinically appropriate reports, supported consultations, and was associated with positive patient experiences. Physicians rated the urgency advice as appropriate in the great majority of cases, indicating that the system effectively guided patients to a suitable level of care and performed safely within its intended purpose. Diagnostic agreement was also high, with physicians’ most probable diagnoses commonly included among the system’s suggested conditions, confirming the clinical validity of the differential lists and their relevance to real patient presentations. The only clear exception was dermatology, where inclusion was considerably lower, a finding consistent with the fact that dermatological diagnosis depends largely on visual pattern recognition rather than symptom description. This difference highlights both the strengths and inherent limitations of a text-based DDSS, which performs best when diagnostic reasoning is driven by the patient’s reported history rather than visual assessment. Both physicians and patients reported feeling better prepared for the consultation, reflecting the system’s value in improving the structure and completeness of information at the point of care. These findings indicate that a CDSS can generate actionable information, combining appropriate care-level advice with plausible diagnostic hypotheses, while also structuring the underlying symptom and history information in ways that promote safer navigation and more efficient doctor–patient interaction. The report produced by the CDSS represents a structured account of the patient’s history, bridging the pre-consultation and consultation phases. Its value lies in capturing the breadth of symptoms, translating them into plausible conditions, and presenting a concise problem summary for clinical review. When the consultation addressed the same problem reported by the patient in the assessment, the physician’s final diagnosis was commonly included among the suggestions. Evidence from computer-assisted history taking and patient-entered questionnaires shows that such structures improve completeness and data quality compared with unstructured note taking. 6 , 14 They can also help clinicians identify relevant but unspoken symptoms, supporting diagnostic reasoning. By providing a ready-made overview, the CDSS effectively front-loads part of the information-gathering task, reducing redundancy during the visit and clarifying what remains to be verified or explored. Clinical consultations follow a predictable but time-pressured sequence: history taking, examination, and synthesis, often interrupted by documentation tasks and parallel digital systems. In this context, structured digital inputs can alleviate cognitive load and documentation burden. 15 , 16 In this study, physicians most often saved time during the encounter itself. Beyond efficiency, the CDSS appeared to strengthen diagnostic confidence. Associations between the perceived usefulness of condition suggestions and both preparedness and efficiency suggest that clinicians approached the consultation with a clearer differential. For junior doctors in particular, structured data capture can compensate for limited experience in eliciting and synthesising patient information; studies show that novice practitioners rely more on analytical and externally guided reasoning, often gathering broader but less synthesized information, while experts depend on pattern recognition and faster intuitive processing. 17 , 18 Structured digital tools, such as electronic templates and decision-support systems, have been shown to improve completeness and confidence among junior primary care clinicians, effectively scaffolding their reasoning and narrowing the experience gap. 19 , 20 Despite these benefits, only a minority of physicians in this study cohort opened and reviewed the report before consultations (60 out of 163), revealing how challenging it is to change established clinical habits. Behavioral barriers such as perceived time cost or lack of integration into the electronic record are common across digital health implementations. Practical levers for improvement include automatic insertion of the report into the electronic medical record, reminders within scheduling software that alert clinicians when a patient has completed a pre-assessment and prompt report review before the consultation, training actions, and peer examples demonstrating efficiency gains. 21 , 22 From the patient perspective, the CDSS improved both preparedness and led one-third of participants to feel less anxious. Participants who felt prepared were more likely to report reduced anxiety, and physicians noted that these patients arrived more engaged and informed. This sequence, from complete self-report to preparedness and lower anxiety, suggests that structured symptom assessment can foster confidence rather than amplify concern. These findings contrast with evidence linking unfiltered internet searches to increased anxiety and cyberchondria 23 , indicating that clinically validated pre-consultation assessment can help counter misinformation by contextualising symptoms and clarifying appropriate next steps. Together, these findings show that CDSS before the clinical encounter can add value across the care pathway: for clinicians, they enhance preparedness, confidence and safety vigilance; for patients, they promote reassurance and engagement. Realising these benefits at scale will require embedding such tools into daily workflows and maintaining systematic feedback loops for post-market learning. The ESSENCE study, integrating real-world evidence and clinician feedback, offers a blueprint for responsible evolution of digital support within healthcare systems. Limitations & Strengths Several limitations should be considered. Participants were adults using the myCUF application within a private healthcare network, likely favouring individuals with higher digital access and health literacy. Certain specialties were represented by small numbers, and older groups may have been under-represented since ≥ 65 year old subgroup comprised few participants. These factors may limit the extent to which the findings reflect all patient groups. In addition, results are drawn from a single, well-resourced private system with specific workflows. Performance and workflow impact may differ in public or resource-constrained settings. Replication in diverse health systems will be important to assess generalisability. Nonetheless, the consistency of results across multiple endpoints supports confidence in the conclusions. The ESSENCE study is one of the first prospective, real-world evaluations of a CE-marked symptom assessment tool integrated into routine care pathways of a major healthcare system. Strengths include the dual perspective of patients and clinicians, enabling robust triangulation of outcomes across appropriateness, efficiency, preparedness, and patient reassurance. Conducted in Portugal’s largest private healthcare network, the study encompassed diverse specialties and care contexts, while predefined endpoints such as urgency advice appropriateness, diagnosis inclusion, and consultation efficiency provide structured evidence of performance. The prospective PMCF design, aligned with ISO 14155:2020, ensures regulatory and methodological rigour, strengthening the reliability and generalisability of the findings for assessing the safety, performance, and benefit of a CDSS in practice. Conclusion The ESSENCE study provides real world evidence that a CDSS can effectively support safe, appropriate, and more confident decision making at the point of entry to care. In routine practice, urgency advice was judged appropriate, the assessment report aligned with clinical reasoning, and physicians reported better preparation for consultations with meaningful efficiency improvements. Participants described feeling prepared and, for a substantial proportion, less anxious after using the CDSS. This paper reports the primary endpoints on appropriateness of urgency advice and of the assessment report, and the secondary endpoints included consultation efficiency and time saving, clinician preparedness, confidence in diagnosis, and perceived usefulness of the assessment report information. Taken together, these findings show that the value of CDSSs in this context lies not only in what it predicts but in how it structures information so that patients and clinicians can act on it with clarity and confidence. In summary, integrating a clinically validated CDSS as a digital front door can enhance preparedness, support safer navigation, and improve consultation efficiency in everyday care. The findings from this Post-Market Clinical Follow-up study strengthen the evidence base for the Ada CDSS, demonstrating safe and effective use within healthcare systems and outlining priorities for continued optimisation and post-market surveillance. Declarations Contributors FC, SG, PF and MSM were responsible for conceptualisation and methodology. PF, MPP, FDL, FC, NK, TM, AP and MS contributed to investigation, project administration, supervision, validation, software and data curation. AP led writing the original draft in collaboration with all co-authors. AL contributed to project administration, supervision and writing - review and editing. All authors had full access to aggregated data in the study and contributed to the decision to submit the manuscript for publication. Declaration of Interest AL is employed by Ada Health. AP, NK, MS and TM are former employees of Ada Health. SG and FC are consultants for Ada Health, and both SG, TM, AP and FC hold share options in the company. SG declares a nonfinancial interest as an Advisory Group member of the EY-coordinated “Study on Regulatory Governance and Innovation in the field of Medical Devices” conducted on behalf of the Directorate-General for Health and Food Safety (SANTE) of the European Commission. SG declares the following competing financial interests: he has or has had consulting relationships with Una Health GmbH, Lindus Health Ltd., Flo Ltd, ICURA ApS, Rock Health Inc., Thymia Ltd., FORUM Institut für Management GmbH, High-Tech Gründerfonds Management GmbH, Prova Health Ltd, Directorate-General for Research and Innovation Of the European Commission. TM declares the following competing financial interests: he has or has had consulting relationships with Suvera & iPlato Healthcare. Data Sharing Data collected for the study, including individual participant data and a data dictionary defining each field in the set, will be made available to others after publication upon reasonable request, subject to approval. Requests for access should be made to the study team at Ada Health ( [email protected] ). After approval, a signed data sharing agreement will be required before data release. Acknowledgements This work was supported by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung) through the European Union-financed NextGenerationEU program under grant number 16KISA100K, project PATH—“Personal Mastery of Health and Wellness Data.” References OECD & European Union. Health at a Glance: Europe 2020: State of Health in the EU Cycle . (OECD, 2020). doi: 10.1787/82129230-en . Oosterhoff, M. et al. Estimating the health impact of delayed elective care during the COVID – 19 pandemic in the Netherlands. Soc. Sci. Med. 320, 115658 (2023). Schmiedhofer, M. et al. Patient motives behind low-acuity visits to the emergency department in Germany: a qualitative study comparing urban and rural sites. BMJ Open 6, e013323 (2016). Lai, F. W. et al. Variables associated with completeness of medical record documentation in the emergency department. Emerg. Med. Australas. 31, 632–638 (2019). Zavala, A. M., Day, G. E., Plummer, D. & Bamford-Wade, A. Decision-making under pressure: medical errors in uncertain and dynamic environments. Aust. Health Rev. 42, 395 (2018). Huffman, K. C. The Effects of a Targeted History Question on Patient-Triage Nurse Communication. Nurs. Clin. North Am. 54, 33–51 (2019). Korom, R. et al. AI-based Clinical Decision Support for Primary Care: A Real-World Study. Preprint at https://doi.org/10.48550/arXiv.2507.16947 (2025). Yuan, M. et al. Large language models illuminate a progressive pathway to artificial intelligent healthcare assistant. Med. Plus 1, 100030 (2024). McMahon, B. & McInerney, D. Right Care, Right Place, First Time: How AI Is Improving National Virtual Front Doors. NEJM AI 2, (2025). Churruca, K. et al. The place of digital triage in a complex healthcare system: An interview study with key stakeholders in Australia’s national provider. Digit. Health 9, 20552076231181201 (2023). Wallace, W. et al. The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review. Npj Digit. Med. 5, 118 (2022). Riboli-Sasco, E. et al. Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review. J. Med. Internet Res. 25, e43803 (2023). Cotte, F. et al. From Advice to Action: Real-World Behaviour of Patients Using an Integrated Clinical AI for Navigating the Healthcare System. Preprint at https://doi.org/10.1101/2025.07.24.25332138 (2025). Albrink, K., Schröder, D., Joos, C., Müller, F. & Noack, E. M. Usability of an App for Medical History Taking in General Practice From the Patients’ Perspective: Cross-Sectional Study. JMIR Hum. Factors 11, e47755 (2024). Shah, S. J. et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J. Am. Med. Inform. Assoc. 32, 375–380 (2025). Andis Robeznieks. The overlooked benefits of medical scribes. American Medical Association (2018). Cuthbert, C. et al. Expert/Novice differences in Diagnostic Medical Cognition-A Review of the Literature. (1999). Crebbin, W. et al. Judgement: clinical decision-making as a core surgical competency. ANZ J. Surg. 89, 760–763 (2019). Clarke, M., Belden, J. L. & Kim, M. S. Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians: in Proceedings of the International Conference on Health Informatics 304–311 (SCITEPRESS - Science and and Technology Publications, Lisbon, Portugal, 2015). doi: 10.5220/0005188503040311 . Tran, M., Rhee, J., Blazek, K., Balasooriya, C. & Vuong, K. Digital health technology use in Australian general practice (GP) consultations: a cross-sectional analysis of the medicine in Australia: balancing employment and life study. Prim. Health Care Res. Dev. 26, e19 (2025). Ghatnekar, S., Faletsky, A. & Nambudiri, V. E. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol. 11, 803–809 (2021). Bhyat, R., Hagens, S., Bryski, K. & Kohlmaier, J. F. Digital Health Value Realization Through Active Change Efforts. Front. Public Health 9, 741424 (2021). McMullan, R. D., Berle, D., Arnáez, S. & Starcevic, V. The relationships between health anxiety, online health information seeking, and cyberchondria: Systematic review and meta-analysis. J. Affect. Disord. 245, 270–278 (2019). Additional Declarations No competing interests reported. Supplementary Files Appendix1.docx Appendix2.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 25 Feb, 2026 Reviews received at journal 21 Feb, 2026 Reviews received at journal 11 Feb, 2026 Reviewers agreed at journal 21 Jan, 2026 Reviewers agreed at journal 21 Jan, 2026 Reviewers invited by journal 05 Jan, 2026 Editor assigned by journal 24 Nov, 2025 Submission checks completed at journal 24 Nov, 2025 First submitted to journal 19 Nov, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8157860","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":570036277,"identity":"eaeb80d9-930f-4e68-967c-59d22f862391","order_by":0,"name":"Andreia Pimenta","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Andreia","middleName":"","lastName":"Pimenta","suffix":""},{"id":570036279,"identity":"4f99ed82-4a18-49e7-ad55-fac50f59b9e0","order_by":1,"name":"Nisha Kini","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Nisha","middleName":"","lastName":"Kini","suffix":""},{"id":570036281,"identity":"1211e93b-abec-48e8-9fe1-e4b323d727f9","order_by":2,"name":"Fabienne Cotte","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Fabienne","middleName":"","lastName":"Cotte","suffix":""},{"id":570036282,"identity":"809d04d3-6a05-426e-846d-b02ad8ddfed9","order_by":3,"name":"Filipa Dias Lourenço","email":"","orcid":"","institution":"CUF - Gestão de clientes e de serviços de saúde, S.A., Avenida do Forte, N 3, Edifício Suécia III, 2790-073","correspondingAuthor":false,"prefix":"","firstName":"Filipa","middleName":"Dias","lastName":"Lourenço","suffix":""},{"id":570036283,"identity":"ad07c0e0-0696-45cb-8df4-b0334cdb8743","order_by":4,"name":"Miguel Paiva Pereira","email":"","orcid":"","institution":"CUF - Gestão de clientes e de serviços de saúde, S.A., Avenida do Forte, N 3, Edifício Suécia III, 2790-073","correspondingAuthor":false,"prefix":"","firstName":"Miguel","middleName":"Paiva","lastName":"Pereira","suffix":""},{"id":570036284,"identity":"652c6b3a-43d4-465c-90ae-3f9263ce8130","order_by":5,"name":"Marcel Schmude","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Marcel","middleName":"","lastName":"Schmude","suffix":""},{"id":570036285,"identity":"b3de3170-618a-4b2e-b300-814b493dd799","order_by":6,"name":"Athena Lemesiou","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Athena","middleName":"","lastName":"Lemesiou","suffix":""},{"id":570036286,"identity":"d34419ff-a200-4923-b8df-7478e6e31999","order_by":7,"name":"Stephen Gilbert","email":"","orcid":"","institution":"Else Kröner-Fresenius Center for Digital Health, TUD Dresden University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Stephen","middleName":"","lastName":"Gilbert","suffix":""},{"id":570036287,"identity":"d04c12a7-fbd5-46a0-b758-19091e31e188","order_by":8,"name":"Tauseef Mehrali","email":"","orcid":"","institution":"Ada Health GmbH, Neue Grünstraße 17, 10179","correspondingAuthor":false,"prefix":"","firstName":"Tauseef","middleName":"","lastName":"Mehrali","suffix":""},{"id":570036288,"identity":"99c42d8f-e349-49f1-9784-c318429fa90a","order_by":9,"name":"Micaela Seemann Monteiro","email":"","orcid":"","institution":"CUF - Gestão de clientes e de serviços de saúde, S.A., Avenida do Forte, N 3, Edifício Suécia III, 2790-073","correspondingAuthor":false,"prefix":"","firstName":"Micaela","middleName":"Seemann","lastName":"Monteiro","suffix":""},{"id":570036289,"identity":"7b0765e1-daa3-4d78-8588-f663aa645e22","order_by":10,"name":"Pedro Flores","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6ElEQVRIiWNgGAWjYHACAwjFzNzAwFDBwMDeQLwWRqDaMwwMPAcYGAnpgmoBKWRsI0KLOXvztg8fGA4zyLcztkn8nGeX2MPAe/wBPi2WPceKZ84AajE4zNgm2bstGaiFLxGvLQY3coyZeRjSGAyYGZsNeLcdSNzPwGOIX8v9NxAt8s2MzYZ/5xwA2kJIyw0ekBYbBobDjI2PeRuI0GLZk1bMOMPAhscApEXmWLJxDzNf4gx8WszZD29m+FAhISfff/jAwTc1drI97L0HPuB1GJTkQQgx8+BQi6IFFRDQMgpGwSgYBSMOAAC+BkKYkhbX9AAAAABJRU5ErkJggg==","orcid":"","institution":"CUF - Gestão de clientes e de serviços de saúde, S.A., Avenida do Forte, N 3, Edifício Suécia III, 2790-073","correspondingAuthor":true,"prefix":"","firstName":"Pedro","middleName":"","lastName":"Flores","suffix":""}],"badges":[],"createdAt":"2025-11-19 17:53:22","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8157860/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8157860/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99814783,"identity":"530bfb46-9812-4496-99e6-c2be6fc689df","added_by":"auto","created_at":"2026-01-08 14:42:49","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1892236,"visible":true,"origin":"","legend":"","description":"","filename":"ESSENCEappropriatenesspublicationnpjDigitalMedicine1.docx","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/254d63416ef97321081971d5.docx"},{"id":99814880,"identity":"c68496a8-ffbb-46d7-a6a6-e3893805587a","added_by":"auto","created_at":"2026-01-08 14:43:03","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14018,"visible":true,"origin":"","legend":"","description":"","filename":"a7798adfd61c4a49a6c7419b1defa057.json","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/0bac99dac5639f93e625ecd4.json"},{"id":99814692,"identity":"e428ff54-a5a8-402b-b3e6-634d2b1f3c43","added_by":"auto","created_at":"2026-01-08 14:42:35","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1698303,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix1.docx","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/9485060bd6d6778fb933b4c3.docx"},{"id":99814694,"identity":"c6e37ab5-fa73-4bc8-8154-a3b1dfc3a5e7","added_by":"auto","created_at":"2026-01-08 14:42:35","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1917054,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix2.docx","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/d0c1b6ed6ab753ea746942b9.docx"},{"id":99814806,"identity":"21199c4c-ee1d-4be0-aa57-90c0bda1d246","added_by":"auto","created_at":"2026-01-08 14:42:52","extension":"xml","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":108240,"visible":true,"origin":"","legend":"","description":"","filename":"a7798adfd61c4a49a6c7419b1defa0571enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/173a11a257ffb961e6061963.xml"},{"id":99814755,"identity":"0d1eed71-76f2-4c23-aae2-1a30bbb5954f","added_by":"auto","created_at":"2026-01-08 14:42:44","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":175582,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/3fd4e7e848015e3609bd424d.png"},{"id":99814790,"identity":"ae3faac3-e2f2-459a-abc7-47892d717864","added_by":"auto","created_at":"2026-01-08 14:42:49","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":70365,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/1e76dde3faba216d4959bbd7.png"},{"id":99814712,"identity":"17f5f0db-9e4d-4032-8733-57012fa2748a","added_by":"auto","created_at":"2026-01-08 14:42:38","extension":"xml","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":108175,"visible":true,"origin":"","legend":"","description":"","filename":"a7798adfd61c4a49a6c7419b1defa0571structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/0f7690067cb8686256f65613.xml"},{"id":99814621,"identity":"7f5b90f7-958a-482b-91b5-aa68eceb54a0","added_by":"auto","created_at":"2026-01-08 14:42:22","extension":"html","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":118477,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/ebec21d380623af30846b76e.html"},{"id":99814780,"identity":"103130d3-5437-4bad-81d6-c5c446de2ab4","added_by":"auto","created_at":"2026-01-08 14:42:48","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":180380,"visible":true,"origin":"","legend":"\u003cp\u003eSTROBE flow diagram with participant inclusion\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/fa1de23ca3ce7c4b123363bc.png"},{"id":99816773,"identity":"4953fb7e-69cd-41d5-9c2a-14299bf0c0b7","added_by":"auto","created_at":"2026-01-08 14:47:46","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1148297,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/83397485-3525-4448-8201-43cc225de2b7.pdf"},{"id":99814588,"identity":"c1ac9537-8040-4b12-95bf-d1332880b469","added_by":"auto","created_at":"2026-01-08 14:42:06","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1698303,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix1.docx","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/a5e7aafb8ae2aa2c2ecac402.docx"},{"id":99814744,"identity":"e2a66433-9c1a-48c9-86f3-b2394f416fde","added_by":"auto","created_at":"2026-01-08 14:42:42","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1917054,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix2.docx","url":"https://assets-eu.researchsquare.com/files/rs-8157860/v1/485a4299fe0d9c2a4e1db251.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Appropriateness and Utility of a Clinical Decision Support System at the Digital Front Door","fulltext":[{"header":"Introduction","content":"\u003cp\u003eHealth systems face mounting pressure from population ageing, multimorbidity, and persistent workforce shortages. These trends increase demand while restricting the supply of clinical time, lengthening queues, and threatening timely access to appropriate care. Delayed or misdirected access has been associated with poorer outcomes, including avoidable hospitalisations and excess mortality, underscoring the need for scalable approaches that preserve both quality and safety.\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e,\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eEfforts to address these pressures can be grouped across three phases of the care pathway. Before the visit, patients require support to determine whether care is needed, where to seek it, and how urgently; in some cases, empowering them with reliable information may enable safe self-management. During the visit, clinicians benefit from tools that improve efficiency and quality of the encounter. High documentation burden remains a persistent challenge: in acute care, overcrowding, often driven by non-urgent presentations, forces clinicians to take and record histories under time pressure, with frequent interruptions that increase the risk of omissions.\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e Documentation quality varies with experience: junior physicians may take three times longer to complete histories, while senior physicians are faster but sometimes less complete.\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e Overstretching under these conditions is linked to higher error rates.\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e Structured, interactive, and pre-populated records have therefore been proposed as part of the solution, with digital history-taking tools offering the potential to improve clarity, accuracy, and completeness while reducing burden.\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e After the visit, technologies for monitoring and asynchronous communication can reduce unnecessary attendances, support adherence, and promote continuity. The shared goal across phases is to optimise use of healthcare resources, enhance the experience of both patients and professionals, and improve outcomes.\u003c/p\u003e \u003cp\u003eTechnology is increasingly being explored as a lever across these phases. AI-based scribe tools aim to reduce documentation burden and free time for patient interaction. Machine learning has been applied to clinical decision support and error reduction.\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e Large language models are being evaluated for summarisation, triage, and diagnostic support, though careful integration is required to ensure reliability.\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eA particular class of tools with relevance at the entry point to care are the so-called digital front doors (DFD). These patient-facing, digitally enabled platforms combine symptom assessment, self-triage, booking, and sometimes remote consultations to guide people to the right service at the right time.\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e,\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e While conceptually spanning all three phases, most implementations focus on the pre-visit stage and preparation for the consultation.\u003c/p\u003e \u003cp\u003eThe evidence on DFDs based on CDSS remains mixed. Systematic reviews report wide variation in diagnostic accuracy and appropriateness of urgency advice, with performance influenced by presentation type and evaluation method.\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e,\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e Much of the literature is based on vignette studies or specialty-specific pilots, limiting generalisability. Real-world evaluations across broader patient populations and specialties remain scarce.\u003c/p\u003e \u003cp\u003eAgainst this backdrop, the ESSENCE (E-health Self Symptom assEssmeNt as a front door and facilitator of CarE) post-market clinical follow-up study evaluates a regulated symptom assessment device integrated into routine practice in Portugal. In contrast to our companion analysis focusing on health-seeking behaviour, this paper examines the appropriateness of the information provided and its impact on consultation quality and effectiveness, asking whether the tool supports safe and appropriate guidance, strengthens decision-making and care navigation, and enables more prepared and efficient consultations.\u003c/p\u003e\n\u003ch3\u003eObjectives\u003c/h3\u003e\n\u003cp\u003eTo determine whether a CDSS integrated as DFD can deliver appropriate urgency advice and clinically useful reports, and whether it is associated with improved preparedness and consultation efficiency in real-world practice.\u003c/p\u003e "},{"header":"Methods","content":" \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy Design and Setting\u003c/h2\u003e \u003cp\u003eThis paper reports results from ESSENCE, a prospective quality-improvement evaluation of a digital clinical decision support system developed by Ada Health (Berlin, Germany) and implemented in Portugal\u0026rsquo;s largest private healthcare network (CUF), integrated through the myCUF application. \u0026lsquo;Ada Assess\u0026rsquo; (commonly known as \u0026lsquo;Ada\u0026rsquo;), manufactured by Ada Health GmbH (Berlin, Germany), is a CE-marked Class IIa medical device under the EU MDR 2017/745.\u003c/p\u003e \u003cp\u003eAdults (\u0026ge;\u0026thinsp;18 years) using the myCUF app to complete a full assessment for themselves were eligible. Inclusion required completion of pre- and post-assessment questions, informed consent, and report sharing with a healthcare provider. Participants were instructed not to use the CDSS in emergency situations as per device intended purpose as described in the instructions for use and disclaimers. Enrolment ran from November 2023 to October 2024.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eData Collection Procedures\u003c/h3\u003e\n\u003cp\u003eThe full ESSENCE protocol has been reported previously.\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e In brief, participants responded to an intended question about their planned healthcare-seeking behaviour before completing the self-assessment through the CDSS. The assessment generated a structured report including suggested conditions based on user provided information, a summary of entered symptoms, and urgency advice. Each suggested condition was paired with a condition-specific urgency recommendation, expressed using eight-point scale (Self-care, Self-care with pharmacy support, Primary care in 2\u0026ndash;3 weeks, Primary care in 2\u0026ndash;3 days, Primary care on the same day, Primary care within 4 hours, Emergency care, and Call ambulance). Physicians were asked to rate the appropriateness of urgency advice at two levels: (i) whether, based on the patient\u0026rsquo;s symptoms, the recommended advice for the most likely (top suggested) condition was reasonable, and (ii) whether the urgency advice levels across all suggested conditions were reasonable. In addition, the system generated an overall advice level (highest acuity across all advice suggestions), which was linked to the CUF care navigation options (Call Ambulance, Emergency Care, Video call with a General Practitioner). This overall advice level represented the guidance presented to the patient as the appropriate next step in care.\u003c/p\u003e \u003cp\u003eImmediately after the assessment, participants completed a survey, which captured completeness of symptom entry, preparedness for the upcoming consultation, and perceived psychological effects (e.g. anxiety).\u003c/p\u003e \u003cp\u003eFollowing the consultation, treating physicians were instructed to complete a structured survey immediately after seeing the patient, or at the latest by the end of the same working day. Where this was not possible, surveys were completed retrospectively using the patient record. In these retrospective cases, only a reduced set of items was addressed, focusing on the appropriateness of the report information (e.g. urgency advice, reasonableness of suggested conditions, completeness of the symptom list, and concordance of the report with the consultation focus and final diagnosis), whereas questions relating to the effect of the report on the consultation process itself were excluded. Physicians also reported whether the report prompted additional diagnostic tests or flagged critical findings. Further items evaluated consultation efficiency, time savings in specific activities, and perceptions of patient engagement and being informed.\u003c/p\u003e \u003cp\u003eParticipant and physician surveys combined Likert-type scales, categorical response options, and free-text fields. Full survey instruments are available in Appendix 2.\u003c/p\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eThe ESSENCE study had two predefined primary endpoints: (i) the appropriateness of the urgency advice, assessed through two survey items in which physicians rated whether the advice for the most probable condition and overall advice levels were reasonable; and (ii) the appropriateness of the assessment report, evaluated across multiple dimensions including the reasonableness of suggested conditions, completeness of the symptom list, concordance of the main problem with the consultation focus, inclusion of the physician\u0026rsquo;s most likely diagnosis among the suggested conditions, and whether the report provided useful information, prompted additional tests, or identified critical findings.\u003c/p\u003e \u003cp\u003eSurvey responses were captured on a five-point Likert scale from \u003cem\u003eStrongly disagree\u003c/em\u003e to \u003cem\u003eStrongly agree\u003c/em\u003e and dichotomised for analysis, with \u003cem\u003eAgree\u003c/em\u003e and \u003cem\u003eStrongly agree\u003c/em\u003e considered appropriate. Relationships between survey measures such as completeness, reasonableness, and perceived usefulness and physician-reported outcomes (efficiency, preparedness, perceived patient effects) were examined descriptively or, where appropriate, using the tests described above. For participant-reported outcomes, associations between completeness, preparedness, and anxiety reduction were evaluated similarly. Missing data were handled by excluding cases on a per-item basis.\u003c/p\u003e \u003cp\u003eContinuous variables were summarised as means with standard deviations or as medians with interquartile ranges, depending on distribution, and categorical variables were described using frequencies and percentages. Comparisons across subgroups (specialty, sex, and age group) were performed using Pearson\u0026rsquo;s χ\u0026sup2; test or Fisher\u0026rsquo;s exact test, as appropriate, with a two-sided significance level of 0.05. For specialty-level analyses, specialties with fewer than ten consultations were grouped under \u003cem\u003eOther\u003c/em\u003e to enable meaningful comparison. Ninety-five per cent confidence intervals for proportions were calculated using standard binomial methods. Confidence intervals for risk differences and risk ratios were reported where relevant. All analyses were conducted in Python (v3.10).\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eData Management and Ethics\u003c/h3\u003e\n\u003cp\u003eData was securely stored in a validated electronic system (Teamscope) using unique identifiers. Ethical approval was granted by the Comiss\u0026atilde;o de \u0026Eacute;tica para a Investiga\u0026ccedil;\u0026atilde;o Cl\u0026iacute;nica (No. \u003cb\u003e2204JJ351\u003c/b\u003e and No. \u003cb\u003e2309JJ660\u003c/b\u003e). The study was registered at ClinicalTrials.gov (\u003cb\u003eNCT06846957\u003c/b\u003e) - registry date: February, 26th, 2025, and complied with the Declaration of Helsinki and ISO 14155:2020 guidelines. STROBE guidelines were followed throughout.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eStudy Population\u003c/h2\u003e \u003cp\u003eA total of 1,470 participants were enrolled, with a mean age of 38.5 years (SD 12.5, range 18\u0026ndash;83), of whom 57.7% (848/1,470) were female. Participants initially entered a median of two symptoms (IQR 1\u0026ndash;3). Cases were grouped by clinical specialty according to their presentation. Participants completed a post-assessment survey, which provided information on the completeness of symptom entry, preparedness for consultation, and psychological effects of the assessment.\u003c/p\u003e \u003cp\u003ePhysician surveys were completed for 163 consultations, in different settings: 13 in-person visits in the emergency department and 150 teleconsultations. Of the total, 103 surveys were completed retrospectively, where the physician had not reviewed the report before the consultation, and 60 were completed immediately after consultations in which the report was reviewed in advance. The latter group (n\u0026thinsp;=\u0026thinsp;60) formed the basis for analyses of consultation efficiency and preparedness.\u003c/p\u003e \u003cp\u003eThe participant inclusion process is outlined in a STROBE flow diagram (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eReasonability of urgency advice\u003c/h3\u003e\n\u003cp\u003ePhysicians agreed that the urgency advice level associated with the most probable condition was reasonable in 74.1% of cases (120/161; 95% CI 67.4\u0026ndash;80.1). No differences were observed by case specialty (p\u0026thinsp;=\u0026thinsp;0.535), sex (p\u0026thinsp;=\u0026thinsp;0.237) or age group (p\u0026thinsp;=\u0026thinsp;1.000) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, Appendix Table\u0026nbsp;1A and 1C). For each case, physicians were asked whether all urgency advice levels provided by the assessment report, i.e. condition-specific urgency recommendations, were reasonable; in 77.6% (125/161; 95% CI 70.1\u0026ndash;83.9) of cases the recommendations were considered reasonable. No significant differences were found with case specialty (p\u0026thinsp;=\u0026thinsp;0.107), sex (p\u0026thinsp;=\u0026thinsp;0.710), or age group (p\u0026thinsp;=\u0026thinsp;0.685) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, Appendix Tables\u0026nbsp;1B and 1C).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAppropriateness of urgency advice (* Agree\u0026thinsp;=\u0026thinsp;Strongly Agree or Agree). Values are numbers and percentages of cases judged appropriate by physicians for both the top (most probable condition) and all urgency advice levels generated by the CDSS. Comparisons across specialties, sex, and age groups were performed using Pearson\u0026rsquo;s χ\u0026sup2; or Fisher\u0026rsquo;s exact test as appropriate.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c4\" namest=\"c3\"\u003e \u003cp\u003eMost probable condition advice recommendation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eAll condition-specific recommendations\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003en/N (% agree)*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003en/N (% agree)*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"7\" rowspan=\"8\"\u003e \u003cp\u003e\u003cb\u003eSpeciality\u003c/b\u003e\u003c/p\u003e \u003cp\u003ep\u0026thinsp;=\u0026thinsp;0.535 (top), 0.107 (all)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDermatology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7/10 (70.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e34.8\u0026ndash;93.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e5/10 (50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e23.7\u0026ndash;76.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGastroenterology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e15/23 (65.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e43.4\u0026ndash;83.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e16/22 (72.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e49.8\u0026ndash;89.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGynaecology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e22/26 (84.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e65.1\u0026ndash;95.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e23/26 (88.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e69.8\u0026ndash;97.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOrthopaedics / Trauma\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e13/20 (65.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e40.8\u0026ndash;84.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e14/20 (70.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e45.7\u0026ndash;88.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOtorhinolaryngology / Ear Nose Throat\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e35/47 (74.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e59.7\u0026ndash;86.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e38/48 (79.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e64.0\u0026ndash;90.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePulmonology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10/11 (90.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e58.7\u0026ndash;99.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e11/11 (100.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e71.5\u0026ndash;100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOther\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e18/23 (78.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e56.3\u0026ndash;92.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e18/24 (75.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e53.3\u0026ndash;90.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003eTotal\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e120/161 (74.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e67.4\u0026ndash;80.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e125/161 (77.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e70.1\u0026ndash;83.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eSex\u003c/b\u003e\u003c/p\u003e \u003cp\u003ep\u0026thinsp;=\u0026thinsp;0.237 (top), 0.710 (all)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFemale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e76/97 (78.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e68.8\u0026ndash;86.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e76/96 (79.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e69.6\u0026ndash;86.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e44/64 (68.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e56.1\u0026ndash;79.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e49/65 (75.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e63.3\u0026ndash;85.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eAge\u003c/b\u003e\u003c/p\u003e \u003cp\u003ep\u0026thinsp;=\u0026thinsp;1.000 (top), 0.685 (all)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e18\u0026ndash;64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e115/154 (74.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e67.0\u0026ndash;81.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e118/153 (77.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e69.4\u0026ndash;83.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026ge;\u0026thinsp;65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5/7 (71.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e29.0\u0026ndash;96.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e7/8 (87.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e47.3\u0026ndash;99.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe appropriateness of the care recommendations and comparison with actual participant behaviour were analysed by overall advice level, defined as the highest-acuity recommendation generated by the CDSS for each case (for example, if both self-care and emergency care were suggested, the case was categorised under emergency care). Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e summarises these results. Physicians rated the overall urgency advice as appropriate in 77.6% (95% CI 70.1\u0026ndash;83.9), neutral in 13.7% (95% CI 8.5\u0026ndash;20.2), and not appropriate in 8.7% (95% CI 5.0\u0026ndash;13.9) (14/161). To understand whether advice judged not appropriate nonetheless resulted in safe user behaviour, the 14 cases in which the CDSS advice was rated not appropriate were examined and compared participants\u0026rsquo; actual care-seeking actions. In five of these cases (35.7%; 95% CI 12.8\u0026ndash;64.9), participants\u0026rsquo; actions matched what physicians considered appropriate, indicating that some users sought care consistent with clinical expectations even when the CDSS advice was inaccurate (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAppropriateness of care recommendation with actual participant behaviour comparison. Cases were grouped by the highest-acuity recommendation (overall advice level) generated by the CDSS. Values show physician ratings of appropriateness (appropriate, neutral, not appropriate) and, for cases judged not appropriate, the patient\u0026rsquo;s actual care-seeking behaviour and whether that behaviour was considered appropriate. Percentages are within each overall-advice category.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOverall advice level\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAppropriate n (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNeutral n (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNot appropriate n (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003e95% CI\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eIn not-appropriate cases: Participant behaviour rated appropriate n (%) \u0026dagger;\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSelf-care\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3 (42.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9.9\u0026ndash;81.6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e4 (57.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e18.4\u0026ndash;90.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0\u0026ndash;35.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSelf-care\u0026thinsp;+\u0026thinsp;pharmacy support\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e19 (86.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e65.1\u0026ndash;97.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3 (13.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2.9\u0026ndash;34.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0\u0026ndash;14.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrimary care in 2\u0026ndash;3 weeks\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3 (33.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7.5\u0026ndash;70.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5 (55.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e21.2\u0026ndash;86.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1 (11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.3\u0026ndash;48.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eTelemedicine (1): 0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrimary care in 2\u0026ndash;3 days\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e33 (82.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e67.2\u0026ndash;92.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2 (5.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.6\u0026ndash;16.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e5 (12.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e4.2\u0026ndash;26.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eTelemedicine (5): 2 (40.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrimary care same day\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e49 (81.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e70.3\u0026ndash;90.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e6 (10.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e4.0\u0026ndash;19.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e5 (8.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e2.8\u0026ndash;18.4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eTelemedicine (5): 2 (40.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrimary care within 4 h\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e6 (100.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e61.0\u0026ndash;100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0\u0026ndash;39.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0\u0026ndash;39.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u0026ndash;\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEmergency care\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e9 (81.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e48.2\u0026ndash;97.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1 (9.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.2\u0026ndash;41.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1 (9.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.2\u0026ndash;41.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eTelemedicine (1): 0 (0.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCall ambulance\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3 (50.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11.8\u0026ndash;88.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1 (16.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.4\u0026ndash;64.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e2 (33.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e4.3\u0026ndash;77.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eTelemedicine (2): 1 (50.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eTotal (n\u0026thinsp;=\u0026thinsp;161)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e125 (77.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e70.1\u0026ndash;83.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e22 (13.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e8.5\u0026ndash;20.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e14 (8.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e5.0\u0026ndash;13.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e5 (35.7) (95% CI 12.8\u0026ndash;64.9)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u0026dagger;Alignment compares the action taken (Service Visited) with the physician\u0026rsquo;s judgement of appropriate behaviour (Behaviour Appropriateness) for that case.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTo further characterise the nature of the inappropriate advice, within the same 14 cases, the CDSS\u0026rsquo;s highest (overall) advice level was compared with the physician\u0026rsquo;s recommended care-seeking behaviour. Physician responses for what the appropriate behaviour in 3 levels (\u0026ldquo;Managing symptoms at home or consulting a pharmacist\u0026rdquo;, \u0026ldquo;Consulting a GP\u0026rdquo; or \u0026ldquo;Consulting a specialist\u0026rdquo; and \u0026ldquo;Emergency Room / Emergency\u0026rdquo;) and were mapped onto the same eight-level CDSS advice scale (Appendix Table\u0026nbsp;2A). The CDSS recommendation coincided with the physician\u0026rsquo;s behaviour recommendation in nine cases (64.3%; 95% CI 35.1\u0026ndash;87.2) and was more cautious in five (35.7%; 95% CI 12.8\u0026ndash;64.9). Importantly, no instances were identified in which the CDSS provided a less urgent recommendation than physicians considered appropriate, indicating that all discrepancies reflected conservative rather than unsafe guidance. It should be noted that physicians\u0026rsquo; recommended behaviours were captured using a simplified three-level categorisation (self-care, primary care, or emergency care), whereas the CDSS generated advice across an eight-level scale. Consequently, apparent agreements may partly reflect differences in scale granularity rather than true clinical agreement.\u003c/p\u003e\n\u003ch3\u003eAppropriateness of the assessment report\u003c/h3\u003e\n\u003cp\u003eAppropriateness of the assessment report was evaluated across multiple dimensions, including the reasonableness of suggested conditions, completeness of the symptom list, concordance of the main problem with the consultation focus, inclusion of the physician\u0026rsquo;s most probable diagnosis, provision of additional useful information, and whether the report prompted further tests or flagged critical findings (Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePhysician-reported survey outcomes on assessment report appropriateness and consultation impact. Values are numbers and percentages of physicians selecting each response option. Comparisons across subgroups (specialty, sex, age) used Pearson\u0026rsquo;s χ\u0026sup2; or Fisher\u0026rsquo;s exact test where applicable.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eItem\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResponse\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003en/N (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e95% CI (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eAppropriateness of the Assessment Report\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eConditions reasonability\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAll reasonable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e119/163 (73.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e65.7\u0026ndash;79.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSome reasonable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e38/163 (23.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e17.2\u0026ndash;30.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNone reasonable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6/163 (3.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.4\u0026ndash;7.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003eSymptom list complete\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e57/162 (35.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e28.0\u0026ndash;43.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e70/162 (43.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e35.4\u0026ndash;51.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e18/162 (11.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e6.7\u0026ndash;17.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e16/162 (9.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5.8\u0026ndash;15.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1/162 (0.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.0\u0026ndash;3.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eMain problem matched consultation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e135/163 (82.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e76.1\u0026ndash;88.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e28/163 (17.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e11.8\u0026ndash;23.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eMost probable diagnosis included\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncluded\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e131/163 (80.4)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e73.5\u0026ndash;86.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNot included\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e32/163 (19.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e14.0\u0026ndash;26.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eAdditional useful information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e107/153 (69.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e62.1\u0026ndash;77.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e46/153 (30.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e23.0\u0026ndash;37.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eAdditional tests prompted\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e31/159 (19.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.8\u0026ndash;26.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e128/159 (80.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e73.5\u0026ndash;86.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eCritical findings identified\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e52/157 (33.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e26.0\u0026ndash;41.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e105/157 (66.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e59.0\u0026ndash;74.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eImpact of the Assessment Report in the Consultation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eUsefulness of condition suggestions\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVery useful\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e28/60 (46.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e33.7\u0026ndash;60.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSomewhat useful\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e23/60 (38.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e26.1\u0026ndash;51.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNot useful\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e9/60 (15.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e7.1\u0026ndash;26.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003eEfficiency improvement\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e14/60 (23.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.4\u0026ndash;36.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e26/60 (43.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e30.6\u0026ndash;56.8\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e11/60 (18.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e9.5\u0026ndash;30.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7/60 (11.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.8\u0026ndash;22.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2/60 (3.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.6\u0026ndash;11.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003ePhysician preparedness for consultation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e9/60 (15.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e7.1\u0026ndash;26.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e34/60 (56.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e43.2\u0026ndash;69.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e9/60 (15.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e7.1\u0026ndash;26.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4/60 (6.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.8\u0026ndash;16.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4/60 (6.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.8\u0026ndash;16.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003eConfidence in diagnosis\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e14/60 (23.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.4\u0026ndash;36.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e23/60 (38.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e26.1\u0026ndash;51.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12/60 (20.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e10.7\u0026ndash;32.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8/60 (13.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e6.0\u0026ndash;24.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3/60 (5.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.0\u0026ndash;13.9\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003ePatient more engaged in consultation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10/58 (17.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e8.5\u0026ndash;29.4\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e35/58 (60.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e46.6\u0026ndash;73.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e7/58 (12.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5.0\u0026ndash;23.3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e6/58 (10.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3.9\u0026ndash;21.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0/58 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u0026ndash;6.2\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"4\" rowspan=\"5\"\u003e \u003cp\u003ePatient more informed in consultation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly agree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e14/59 (23.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e13.7\u0026ndash;36.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e30/59 (50.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e37.6\u0026ndash;64.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeutral\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e10/59 (16.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e8.5\u0026ndash;29.0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDisagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5/59 (8.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2.8\u0026ndash;18.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStrongly disagree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0/59 (0.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u0026ndash;6.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eIn 73.0% of consultations (119/163; 95% CI 65.7\u0026ndash;79.5), all suggested conditions were judged reasonable, while in a further 23.3% (38/163; 95% CI 17.2\u0026ndash;30.5) at least some suggestions were considered reasonable; only 3.7% (6/163; 95% CI 1.4\u0026ndash;7.9) were judged not reasonable. The symptom list was considered complete in 78.4% (127/162; 95% CI 71.2\u0026ndash;84.4), and the main medical problem matched the consultation focus in 82.8% (135/163; 95% CI 76.1\u0026ndash;88.2). Additional useful patient information was reported in 69.9% (107/153; 95% CI 62.1\u0026ndash;77.0), the report prompted further diagnostic tests in 19.5% (31/159; 95% CI 13.8\u0026ndash;26.5), and critical findings were flagged in 33.1% (52/157; 95% CI 26.0\u0026ndash;41.0).\u003c/p\u003e \u003cp\u003eThe physician\u0026rsquo;s most probable diagnosis was included among the suggested conditions in 80.4% (131/163; 95% CI 73.5\u0026ndash;86.0), with significant differences by specialty (p\u0026thinsp;=\u0026thinsp;0.035), but no significant variation by sex (p\u0026thinsp;=\u0026thinsp;1.000) or age group (p\u0026thinsp;=\u0026thinsp;0.191) (Appendix Table\u0026nbsp;3A). Across most specialties, results were close to or above 80%, indicating generally strong correspondence between the CDSS suggestions and physicians\u0026rsquo; clinical reasoning.\u003c/p\u003e \u003cp\u003ePhysicians were more likely to find the diagnosis included in the report when the consultation focused on the same symptoms or medical problem as those stated in the assessment: most probable diagnosis included in 87.4% (118/135; 95% CI 80.7\u0026ndash;92.6) of such cases, compared with 46.4% (13/28; 95% CI 27.5\u0026ndash;66.1) when the consultation addressed a different problem (p\u0026thinsp;\u0026lt;\u0026thinsp;0.00001). This indicates that discrepancies in most likely diagnosis often arose when patients presented a different medical problem in the consultation than the one they had reported during the assessment.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eConsultation efficiency and time\u003c/h2\u003e \u003cp\u003eThe mean consultation length, based on 59 valid responses, was 13.0 minutes (SD 5.4; median 14; range 4\u0026ndash;28). Time savings during the consultation were reported in 66.1% (39/59; 95% CI 52.3\u0026ndash;77.6), while 33.9% reported no time saved (20/59; 95% CI 22.4\u0026ndash;47.7). Time was most often saved during the consultation itself (29/39; 74.4%; 95% CI 59.7\u0026ndash;84.7). Time saving was significantly (p\u0026thinsp;=\u0026thinsp;0.033) more often observed in clinical cases judged as easy (11/15; 73.3%; 95% CI 44.9\u0026ndash;92.2) compared with moderate or difficult ones (10/26; 38.5%; 95% CI 20.2\u0026ndash;59.4).\u003c/p\u003e \u003cp\u003ePhysicians rated the usefulness of the condition suggestions for the consultation work favourably. Across 60 valid responses, 46.7% (28/60; 95% CI 33.7\u0026ndash;60.0) judged the report to be very useful, 38.3% (23/60; 95% CI 26.1\u0026ndash;51.6) as somewhat useful, and 15.0% (9/60; 95% CI 7.1\u0026ndash;26.6) as not useful. Efficiency improvement was reported by 66.7% (40/60; 95% CI 53.3\u0026ndash;78.3) (Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eEfficiency improvements were more frequent when all suggested conditions were judged reasonable (72.3%, 34/47; 95% CI 57.4\u0026ndash;84.4), compared with 60.0% (6/10; 95% CI 26.2\u0026ndash;87.8) when only some were reasonable and 0.0% (0/3; 95% CI 0\u0026ndash;70.8) when none were reasonable, although this trend did not reach statistical significance (p\u0026thinsp;=\u0026thinsp;0.066). In contrast, perceived usefulness of the condition suggestions was a strong predictor of efficiency: improvements were reported in all cases rated very useful (100%, 28/28; 95% CI 87.9\u0026ndash;100), compared with 52.2% (12/23; 95% CI 30.6\u0026ndash;73.2) when somewhat useful and 0.0% (0/9; 95% CI 0\u0026ndash;33.6) when not useful (p\u0026thinsp;\u0026lt;\u0026thinsp;0.0001) (Appendix Table\u0026nbsp;4A).\u003c/p\u003e \u003cp\u003ePreparedness for the consultation was reported by 71.7% (43/60; 95% CI 58.6\u0026ndash;82.5). This outcome was significantly associated with perceived usefulness of the suggested conditions (p\u0026thinsp;\u0026lt;\u0026thinsp;0.0002) but not with their reasonableness (p\u0026thinsp;=\u0026thinsp;0.56) (Appendix Table\u0026nbsp;4B).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eParticipant survey findings\u003c/h2\u003e \u003cp\u003eIn the post-assessment participant survey, across the three participant-reported endpoints (symptom capture in report, preparedness for consultation, and reduction of anxiety), a joint analysis was possible for those with non-missing answers, i.e. participants without skipped questions to all three items (1,383/1,470). In this cohort, 95.7% (95% CI 94.6\u0026ndash;96.6) of participants reported at least one positive effect (22.2% [95% CI 19.9\u0026ndash;24.7] in one domain, 41.1% [95% CI 38.4\u0026ndash;43.8] in two, and 32.3% [95% CI 29.8\u0026ndash;35.0] in all three), while 4.3% (95% CI 3.4\u0026ndash;5.4) reported no positive effect on any endpoint.\u003c/p\u003e \u003cp\u003eOverall, 80.5% (1,183/1,470; 95% CI 78.4\u0026ndash;82.5) reported that all symptoms they wished to include were captured in the assessment, and 71.2% (1,046/1,470; 95% CI 68.7\u0026ndash;73.6) felt prepared for their consultation (Appendix Table\u0026nbsp;5A). Preparedness was more common among those who reported all their symptoms (80.4% vs 65.7%; risk difference\u0026thinsp;+\u0026thinsp;14.7 pp, 95% CI 7.4\u0026ndash;21.9; RR 1.22, 95% CI 1.10\u0026ndash;1.36) (Appendix Table\u0026nbsp;5B).\u003c/p\u003e \u003cp\u003eOne-third of participants (33.0%, 485/1,470; 95% CI 30.5\u0026ndash;35.6) reported feeling less anxious about their health after using the CDSS, while 30.9% (95% CI 28.4\u0026ndash;33.4) reported not feeling less anxious and 29.0% (95% CI 26.6\u0026ndash;31.5) reported no difference. No significant differences in anxiety reduction were observed by sex or age group (p\u0026thinsp;=\u0026thinsp;0.49 and p\u0026thinsp;=\u0026thinsp;0.46, respectively).\u003c/p\u003e \u003cp\u003eReporting all symptoms was associated with a higher likelihood of feeling less anxious after using the CDSS (37.8% vs 21.6%; risk difference\u0026thinsp;+\u0026thinsp;16.2 pp, 95% CI 9.7\u0026ndash;22.7; RR 1.75, 95% CI 1.32\u0026ndash;2.32). Similarly, those who felt prepared for the consultation after completing an assessment felt less anxious (43.9% vs 8.0%; risk difference\u0026thinsp;+\u0026thinsp;35.9 pp, 95% CI 31.6\u0026ndash;40.3; RR 5.51, 95% CI 3.70\u0026ndash;8.21) (Appendix Table\u0026nbsp;5B).\u003c/p\u003e \u003cp\u003eChanges in intended care were related to changes in anxiety. Reduced anxiety was most common among those who de-escalated intended care (48.1%, 95% CI 43.2\u0026ndash;53.0) and those who escalated care (44.0%, 95% CI 38.7\u0026ndash;49.5), compared with 33.1% (95% CI 29.5\u0026ndash;36.8) in the no-change group. Participants in the made-certain group reported reduced anxiety in 38.7% (95% CI 30.8\u0026ndash;47.1) of cases, while those in the made-uncertain and stayed-uncertain groups reported lower rates (24.3% [95% CI 17.1\u0026ndash;33.1] and 26.7% [95% CI 17.3\u0026ndash;38.8], respectively). Overall differences between groups were not statistically significant (p\u0026thinsp;=\u0026thinsp;0.141).\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study evaluated a regulated digital clinical decision support system integrated into Portugal\u0026rsquo;s largest private healthcare network. In real-world use, the CDSS produced clinically appropriate reports, supported consultations, and was associated with positive patient experiences. Physicians rated the urgency advice as appropriate in the great majority of cases, indicating that the system effectively guided patients to a suitable level of care and performed safely within its intended purpose. Diagnostic agreement was also high, with physicians\u0026rsquo; most probable diagnoses commonly included among the system\u0026rsquo;s suggested conditions, confirming the clinical validity of the differential lists and their relevance to real patient presentations. The only clear exception was dermatology, where inclusion was considerably lower, a finding consistent with the fact that dermatological diagnosis depends largely on visual pattern recognition rather than symptom description. This difference highlights both the strengths and inherent limitations of a text-based DDSS, which performs best when diagnostic reasoning is driven by the patient\u0026rsquo;s reported history rather than visual assessment. Both physicians and patients reported feeling better prepared for the consultation, reflecting the system\u0026rsquo;s value in improving the structure and completeness of information at the point of care. These findings indicate that a CDSS can generate actionable information, combining appropriate care-level advice with plausible diagnostic hypotheses, while also structuring the underlying symptom and history information in ways that promote safer navigation and more efficient doctor\u0026ndash;patient interaction.\u003c/p\u003e \u003cp\u003eThe report produced by the CDSS represents a structured account of the patient\u0026rsquo;s history, bridging the pre-consultation and consultation phases. Its value lies in capturing the breadth of symptoms, translating them into plausible conditions, and presenting a concise problem summary for clinical review. When the consultation addressed the same problem reported by the patient in the assessment, the physician\u0026rsquo;s final diagnosis was commonly included among the suggestions. Evidence from computer-assisted history taking and patient-entered questionnaires shows that such structures improve completeness and data quality compared with unstructured note taking.\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e,\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e They can also help clinicians identify relevant but unspoken symptoms, supporting diagnostic reasoning. By providing a ready-made overview, the CDSS effectively front-loads part of the information-gathering task, reducing redundancy during the visit and clarifying what remains to be verified or explored.\u003c/p\u003e \u003cp\u003eClinical consultations follow a predictable but time-pressured sequence: history taking, examination, and synthesis, often interrupted by documentation tasks and parallel digital systems. In this context, structured digital inputs can alleviate cognitive load and documentation burden.\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e,\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e In this study, physicians most often saved time \u003cem\u003eduring\u003c/em\u003e the encounter itself. Beyond efficiency, the CDSS appeared to strengthen diagnostic confidence. Associations between the perceived usefulness of condition suggestions and both preparedness and efficiency suggest that clinicians approached the consultation with a clearer differential. For junior doctors in particular, structured data capture can compensate for limited experience in eliciting and synthesising patient information; studies show that novice practitioners rely more on analytical and externally guided reasoning, often gathering broader but less synthesized information, while experts depend on pattern recognition and faster intuitive processing.\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e,\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e Structured digital tools, such as electronic templates and decision-support systems, have been shown to improve completeness and confidence among junior primary care clinicians, effectively scaffolding their reasoning and narrowing the experience gap.\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e,\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eDespite these benefits, only a minority of physicians in this study cohort opened and reviewed the report before consultations (60 out of 163), revealing how challenging it is to change established clinical habits. Behavioral barriers such as perceived time cost or lack of integration into the electronic record are common across digital health implementations. Practical levers for improvement include automatic insertion of the report into the electronic medical record, reminders within scheduling software that alert clinicians when a patient has completed a pre-assessment and prompt report review before the consultation, training actions, and peer examples demonstrating efficiency gains.\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e,\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eFrom the patient perspective, the CDSS improved both preparedness and led one-third of participants to feel less anxious. Participants who felt prepared were more likely to report reduced anxiety, and physicians noted that these patients arrived more engaged and informed. This sequence, from complete self-report to preparedness and lower anxiety, suggests that structured symptom assessment can foster confidence rather than amplify concern. These findings contrast with evidence linking unfiltered internet searches to increased anxiety and cyberchondria\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e, indicating that clinically validated pre-consultation assessment can help counter misinformation by contextualising symptoms and clarifying appropriate next steps.\u003c/p\u003e \u003cp\u003eTogether, these findings show that CDSS before the clinical encounter can add value across the care pathway: for clinicians, they enhance preparedness, confidence and safety vigilance; for patients, they promote reassurance and engagement. Realising these benefits at scale will require embedding such tools into daily workflows and maintaining systematic feedback loops for post-market learning. The ESSENCE study, integrating real-world evidence and clinician feedback, offers a blueprint for responsible evolution of digital support within healthcare systems.\u003c/p\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eLimitations \u0026amp; Strengths\u003c/h2\u003e \u003cp\u003eSeveral limitations should be considered. Participants were adults using the myCUF application within a private healthcare network, likely favouring individuals with higher digital access and health literacy. Certain specialties were represented by small numbers, and older groups may have been under-represented since \u0026ge;\u0026thinsp;65 year old subgroup comprised few participants. These factors may limit the extent to which the findings reflect all patient groups. In addition, results are drawn from a single, well-resourced private system with specific workflows. Performance and workflow impact may differ in public or resource-constrained settings. Replication in diverse health systems will be important to assess generalisability. Nonetheless, the consistency of results across multiple endpoints supports confidence in the conclusions.\u003c/p\u003e \u003cp\u003eThe ESSENCE study is one of the first prospective, real-world evaluations of a CE-marked symptom assessment tool integrated into routine care pathways of a major healthcare system. Strengths include the dual perspective of patients and clinicians, enabling robust triangulation of outcomes across appropriateness, efficiency, preparedness, and patient reassurance. Conducted in Portugal\u0026rsquo;s largest private healthcare network, the study encompassed diverse specialties and care contexts, while predefined endpoints such as urgency advice appropriateness, diagnosis inclusion, and consultation efficiency provide structured evidence of performance. The prospective PMCF design, aligned with ISO 14155:2020, ensures regulatory and methodological rigour, strengthening the reliability and generalisability of the findings for assessing the safety, performance, and benefit of a CDSS in practice.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe ESSENCE study provides real world evidence that a CDSS can effectively support safe, appropriate, and more confident decision making at the point of entry to care. In routine practice, urgency advice was judged appropriate, the assessment report aligned with clinical reasoning, and physicians reported better preparation for consultations with meaningful efficiency improvements. Participants described feeling prepared and, for a substantial proportion, less anxious after using the CDSS.\u003c/p\u003e \u003cp\u003eThis paper reports the primary endpoints on appropriateness of urgency advice and of the assessment report, and the secondary endpoints included consultation efficiency and time saving, clinician preparedness, confidence in diagnosis, and perceived usefulness of the assessment report information. Taken together, these findings show that the value of CDSSs in this context lies not only in what it predicts but in how it structures information so that patients and clinicians can act on it with clarity and confidence.\u003c/p\u003e \u003cp\u003eIn summary, integrating a clinically validated CDSS as a digital front door can enhance preparedness, support safer navigation, and improve consultation efficiency in everyday care. The findings from this Post-Market Clinical Follow-up study strengthen the evidence base for the Ada CDSS, demonstrating safe and effective use within healthcare systems and outlining priorities for continued optimisation and post-market surveillance.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eContributors\u003c/h2\u003e\n\u003cp\u003eFC, SG, PF and MSM were responsible for conceptualisation and methodology. PF, MPP, FDL, FC, NK, TM, AP and MS contributed to investigation, project administration, supervision, validation, software and data curation. AP led writing the original draft in collaboration with all co-authors. AL contributed to project administration, supervision and writing - review and editing. All authors had full access to aggregated data in the study and contributed to the decision to submit the manuscript for publication.\u003c/p\u003e\n\u003ch2\u003eDeclaration of Interest\u003c/h2\u003e\n\u003cp\u003eAL is employed by Ada Health. AP, NK, MS and TM are former employees of Ada Health. SG and FC are consultants for Ada Health, and both SG, TM, AP and FC hold share options in the company. SG declares a nonfinancial interest as an Advisory Group member of the EY-coordinated \u0026ldquo;Study on Regulatory Governance and Innovation in the field of Medical Devices\u0026rdquo; conducted on behalf of the Directorate-General for Health and Food Safety (SANTE) of the European Commission. SG declares the following competing financial interests: he has or has had consulting relationships with Una Health GmbH, Lindus Health Ltd., Flo Ltd, ICURA ApS, Rock Health Inc., Thymia Ltd., FORUM Institut für Management GmbH, High-Tech Gründerfonds Management GmbH, Prova Health Ltd, Directorate-General for Research and Innovation Of the European Commission. \u0026nbsp;TM declares the following competing financial interests: he has or has had consulting relationships with Suvera \u0026amp; iPlato Healthcare.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eData Sharing\u003c/h2\u003e\n\u003cp\u003eData collected for the study, including individual participant data and a data dictionary defining each field in the set, will be made available to others after publication upon reasonable request, subject to approval. Requests for access should be made to the study team at Ada Health ([email protected]). After approval, a signed data sharing agreement will be required before data release.\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003cp\u003eThis work was supported by the Federal Ministry of Education and Research (Bundesministerium f\u0026uuml;r Bildung und Forschung) through the European Union-financed NextGenerationEU program under grant number 16KISA100K, project PATH\u0026mdash;\u0026ldquo;Personal Mastery of Health and Wellness Data.\u0026rdquo;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eOECD \u0026amp; European Union. \u003cem\u003eHealth at a Glance: Europe 2020: State of Health in the EU Cycle\u003c/em\u003e. (OECD, 2020). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1787/82129230-en\u003c/span\u003e\u003cspan address=\"10.1787/82129230-en\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOosterhoff, M. \u003cem\u003eet al.\u003c/em\u003e Estimating the health impact of delayed elective care during the COVID \u0026ndash;\u0026thinsp;19 pandemic in the Netherlands. \u003cem\u003eSoc. Sci. Med.\u003c/em\u003e 320, 115658 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchmiedhofer, M. \u003cem\u003eet al.\u003c/em\u003e Patient motives behind low-acuity visits to the emergency department in Germany: a qualitative study comparing urban and rural sites. \u003cem\u003eBMJ Open\u003c/em\u003e 6, e013323 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLai, F. W. \u003cem\u003eet al.\u003c/em\u003e Variables associated with completeness of medical record documentation in the emergency department. \u003cem\u003eEmerg. Med. Australas.\u003c/em\u003e 31, 632\u0026ndash;638 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZavala, A. M., Day, G. E., Plummer, D. \u0026amp; Bamford-Wade, A. Decision-making under pressure: medical errors in uncertain and dynamic environments. \u003cem\u003eAust. Health Rev.\u003c/em\u003e 42, 395 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuffman, K. C. The Effects of a Targeted History Question on Patient-Triage Nurse Communication. \u003cem\u003eNurs. Clin. North Am.\u003c/em\u003e 54, 33\u0026ndash;51 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKorom, R. \u003cem\u003eet al.\u003c/em\u003e AI-based Clinical Decision Support for Primary Care: A Real-World Study. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2507.16947\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2507.16947\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYuan, M. \u003cem\u003eet al.\u003c/em\u003e Large language models illuminate a progressive pathway to artificial intelligent healthcare assistant. \u003cem\u003eMed. Plus\u003c/em\u003e 1, 100030 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcMahon, B. \u0026amp; McInerney, D. Right Care, Right Place, First Time: How AI Is Improving National Virtual Front Doors. \u003cem\u003eNEJM AI\u003c/em\u003e 2, (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChurruca, K. \u003cem\u003eet al.\u003c/em\u003e The place of digital triage in a complex healthcare system: An interview study with key stakeholders in Australia\u0026rsquo;s national provider. \u003cem\u003eDigit. Health\u003c/em\u003e 9, 20552076231181201 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWallace, W. \u003cem\u003eet al.\u003c/em\u003e The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review. \u003cem\u003eNpj Digit. Med.\u003c/em\u003e 5, 118 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRiboli-Sasco, E. \u003cem\u003eet al.\u003c/em\u003e Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review. \u003cem\u003eJ. Med. Internet Res.\u003c/em\u003e 25, e43803 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCotte, F. \u003cem\u003eet al.\u003c/em\u003e From Advice to Action: Real-World Behaviour of Patients Using an Integrated Clinical AI for Navigating the Healthcare System. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1101/2025.07.24.25332138\u003c/span\u003e\u003cspan address=\"10.1101/2025.07.24.25332138\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlbrink, K., Schr\u0026ouml;der, D., Joos, C., M\u0026uuml;ller, F. \u0026amp; Noack, E. M. Usability of an App for Medical History Taking in General Practice From the Patients\u0026rsquo; Perspective: Cross-Sectional Study. \u003cem\u003eJMIR Hum. Factors\u003c/em\u003e 11, e47755 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShah, S. J. \u003cem\u003eet al.\u003c/em\u003e Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. \u003cem\u003eJ. Am. Med. Inform. Assoc.\u003c/em\u003e 32, 375\u0026ndash;380 (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndis Robeznieks. The overlooked benefits of medical scribes. \u003cem\u003eAmerican Medical Association\u003c/em\u003e (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCuthbert, C. \u003cem\u003eet al.\u003c/em\u003e Expert/Novice differences in Diagnostic Medical Cognition-A Review of the Literature. (1999).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrebbin, W. \u003cem\u003eet al.\u003c/em\u003e Judgement: clinical decision-making as a core surgical competency. \u003cem\u003eANZ J. Surg.\u003c/em\u003e 89, 760\u0026ndash;763 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eClarke, M., Belden, J. L. \u0026amp; Kim, M. S. Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians: in \u003cem\u003eProceedings of the International Conference on Health Informatics\u003c/em\u003e 304\u0026ndash;311 (SCITEPRESS - Science and and Technology Publications, Lisbon, Portugal, 2015). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.5220/0005188503040311\u003c/span\u003e\u003cspan address=\"10.5220/0005188503040311\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTran, M., Rhee, J., Blazek, K., Balasooriya, C. \u0026amp; Vuong, K. Digital health technology use in Australian general practice (GP) consultations: a cross-sectional analysis of the medicine in Australia: balancing employment and life study. \u003cem\u003ePrim. Health Care Res. Dev.\u003c/em\u003e 26, e19 (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGhatnekar, S., Faletsky, A. \u0026amp; Nambudiri, V. E. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. \u003cem\u003eHealth Technol.\u003c/em\u003e 11, 803\u0026ndash;809 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBhyat, R., Hagens, S., Bryski, K. \u0026amp; Kohlmaier, J. F. Digital Health Value Realization Through Active Change Efforts. \u003cem\u003eFront. Public Health\u003c/em\u003e 9, 741424 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcMullan, R. D., Berle, D., Arn\u0026aacute;ez, S. \u0026amp; Starcevic, V. The relationships between health anxiety, online health information seeking, and cyberchondria: Systematic review and meta-analysis. \u003cem\u003eJ. Affect. Disord.\u003c/em\u003e 245, 270\u0026ndash;278 (2019).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"digital front door, symptom assessment, self-triage, urgency advice, clinical workflow, consultation efficiency, clinician preparedness, patient experience, post-market clinical follow-up","lastPublishedDoi":"10.21203/rs.3.rs-8157860/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8157860/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground:\u003c/h2\u003e \u003cp\u003eDigital front doors that combine symptom assessment and self-triage are increasingly used to guide patients to appropriate care, yet real-world evidence on safety, performance, and workflow impact remains limited and heterogeneous. This study reports a post-market clinical follow-up (PMCF) evaluation of a clinical decision support system (CDSS), integrated into routine care within Portugal\u0026rsquo;s largest private healthcare network (CUF) to determine whether it can deliver appropriate urgency advice and clinically useful reports, and whether its use is associated with improved preparedness and consultation efficiency in real-world practice.\u003c/p\u003e\u003ch2\u003eMethods:\u003c/h2\u003e \u003cp\u003eThis was a prospective, observational study. Adults aged\u0026thinsp;\u0026ge;\u0026thinsp;18 years completed a full symptom assessment via the myCUF application before consultation. Participants provided informed consent and completed a post-assessment survey; treating physicians completed structured post-consultation surveys. Primary endpoints were the appropriateness of urgency advice and of the assessment report, including the reasonableness of suggested conditions, completeness of symptom history, concordance of the main problem with the consultation focus, and inclusion of the physician\u0026rsquo;s final diagnosis. Secondary endpoints included consultation efficiency and time saving, clinician preparedness, confidence in diagnosis, and perceived usefulness of the assessment report information. Analyses used descriptive statistics, confidence intervals, risk differences and risk ratios, with subgroup comparisons by specialty, sex, and age.\u003c/p\u003e\u003ch2\u003eResults:\u003c/h2\u003e \u003cp\u003eA total of 1,470 participants completed the post-assessment survey. Physicians returned 163 consultation surveys, of which 60 were based on reports reviewed before the encounter and formed the basis for efficiency and preparedness analyses. Urgency advice was judged appropriate in most cases (top advice level: 74.1%, 120/161; 95% CI 67.4\u0026ndash;80.1; all advice levels: 77.6%, 125/161; 95% CI 70.1\u0026ndash;83.9), with no significant differences by specialty, sex, or age. In cases where advice was not considered appropriate (8.7%; 95% CI 5.0\u0026ndash;13.9), all disagreements reflected conservative, over-cautious advice; no instances of under-triage were identified. Report appropriateness was high: conditions provided as suggestions were judged reasonable in 73.0% (119/163; 95% CI 65.7\u0026ndash;79.5), the symptom list was considered complete in 78.4% (127/162; 95% CI 71.2\u0026ndash;84.4), the main problem discussed in the consultation matched the information in the report in 82.8% (135/163; 95% CI 76.1\u0026ndash;88.2), and the clinician\u0026rsquo;s most likely diagnosis was included among the suggested conditions in 80.4% (131/163; 95% CI 73.5\u0026ndash;86.0). Physicians reported efficiency improvements in 66.7% (40/60; 95% CI 53.3\u0026ndash;78.3) and time savings in 66.1% (39/59; 95% CI 52.3\u0026ndash;77.6), most often during the consultation itself. Increased preparedness for the consultation was reported in 71.7% (43/60; 95% CI 58.6\u0026ndash;82.5) and was strongly associated with the perceived usefulness of the condition suggestions. Among participants, 80.5% (1,183/1,470; 95% CI 78.4\u0026ndash;82.5) reported that all symptoms were captured in the assessment report, 71.2% (1,046/1,470; 95% CI 68.7\u0026ndash;73.6) felt more prepared for the consultation, and 33.0% (485/1,470; 95% CI 30.5\u0026ndash;35.6) reported reduced anxiety after assessment completion. Completeness and preparedness were associated with lower anxiety.\u003c/p\u003e\u003ch2\u003eConclusion:\u003c/h2\u003e \u003cp\u003eIn routine practice, an integrated symptom assessment device provided appropriate urgency advice and clinically useful reports, and was associated with greater clinician preparedness, perceived efficiency improvements, and positive patient experience. These PMCF findings support continued safe use as a digital front door and contribute real-world evidence for ongoing conformity assessment of Ada, a clinical decision support system and a Class IIa medical device under EU MDR 2017/745.\u003c/p\u003e","manuscriptTitle":"Appropriateness and Utility of a Clinical Decision Support System at the Digital Front Door","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-08 14:16:41","doi":"10.21203/rs.3.rs-8157860/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-02-25T20:25:50+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-21T18:47:20+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-11T22:30:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"330268422070430219982651375834699069255","date":"2026-01-21T23:11:13+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"76385605065200463795655800191632756795","date":"2026-01-21T20:39:40+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-05T16:52:36+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-11-25T01:46:41+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-11-24T12:11:53+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Digital Medicine","date":"2025-11-19T17:42:52+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"85c31f07-cdbb-430a-af2b-1a0ccb810d0a","owner":[],"postedDate":"January 8th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":60654929,"name":"Health sciences/Diseases"},{"id":60654930,"name":"Health sciences/Health care"},{"id":60654931,"name":"Health sciences/Medical research"}],"tags":[],"updatedAt":"2026-04-25T09:09:52+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-08 14:16:41","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8157860","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8157860","identity":"rs-8157860","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}