Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation

Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation Salim Memon, Adam Brand, Bianca Taylor, Adelaide Michael, Rachael Smithson This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7033639/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 08 Jan, 2026 Read the published version in BMC Health Services Research → Version 1 posted 11 You are reading this latest preprint version Abstract Background: In 2024, Gold Coast Hospital and Health Service outpatient division initiated a 16-week trial of artificial intelligence (AI)-enabled ambient listening scribe technology. The objective of this pilot study was to evaluate the acceptability and impact of scribe technology in producing clinical notes and discharge summaries among outpatient clinicians and patients receiving care. Methods: A mixed method research design combined analysis of data from patient and staff surveys, staff interviews, scribe outputs and electronic medical records across a breadth of outpatient specialties. Results: By and large, ambient listening technology was associated with positive patient and staff experience. On average, 58% of scribe outputs were accepted without modification into the electronic outpatient note. There was limited evidence of bias in outputs, however there was some evidence of hallucination or incorrect outputs. Conclusions: Qualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can 1) produce an accurate summary of outpatient appointments, 2) enhance clinical note quality and 3) improve both clinician and patient experience. Figures Figure 1 What is already known on this topic Ambient listening scribe technology is being adopted at pace by health services in the hope that it will alleviate clinical documentation burden. What this study adds – This study contributes evidence of ambient listening scribe technology performance in a real-world outpatient department application. How this study might affect research, practice, or policy – This study aims to inform practice in the implementation and use of ambient listening scribe technology in other health services considering adoption. INTRODUCTION AND BACKGROUND Medical records are a foundational communication platform for healthcare professionals, facilitating the coordination of patient care across clinical settings [ 1 , 2 ]. These records enable information exchange among a patient’s care team, supporting the management of diagnostic procedures, treatment protocols, and ongoing care activities in both hospital and community environments [ 1 , 2 ]. Despite the importance of accurate and timely recording of medical notes, the burden of documentation experienced by clinicians represents a significant and well-recognised challenge facing the healthcare system [ 3 , 4 ]. To alleviate this burden, ambient listening medical scribes (‘scribes’) are being adopted at pace [ 5 , 6 ]. Scribe technology collects real-time voice data during clinical encounters, instantaneously transforming this content into a summary in the clinician’s chosen format [ 7 , 8 ]. The basis of this technology is machine learning and deep learning algorithms that transform live transcripts into continuously higher quality and more meaningful clinical documentation outputs [ 8 ]. Evidence to date associates this technology with increased efficiency and productivity, improved note quality and note accuracy, and a better experience for both patients and clinicians during encounters [ 7 , 9 – 11 ]. As with all artificial intelligence (AI) solutions, successful implementation requires this technology to be fit for purpose and accepted by those using it. To date, use of ambient listening scribes has not been evaluated in an Australian hospital outpatient department context, representing a gap in literature. In July 2024, Gold Coast Hospital and Health Service (GCHHS), Australia, commenced a trial of an ambient listening solution. Limited term licenses were purchased for 100 clinicians participating in a 16-week trial. The scribe solution was available to clinicians via their work laptop and smart devices. Clinicians were supported through group training sessions and on-demand support by a dedicated project team. Additionally, speakers and microphones were provided to support the use of scribes. This research aims to understand the value of ambient listening technology in routine hospital practice. Using a mixed-methods approach, it aims to produce an early evaluation of AI-enabled ambient listening scribe accuracy, acceptability and impact. METHODS Study design This research adopted a mixed-methods approach, which is noted for its suitability in application to complex effectiveness studies [ 12 ]. The study has ethical approval from the Darling Downs Health Human Research Ethics Committee (HREC/2024/QTDD/108398) and data was collected between July and December 2024. Data collection and analysis focused on 100 clinicians trialling the technology in outpatient clinics. Table 1 summarises the outpatient specialties involved in the pilot. In total 7,499 consultations were undertaken using the AI scribe technology during the trial. Table 1 Summary of outpatient specialty trial participant numbers and average total consults per trial participant Outpatient specialty Count of trial participants Average of total consults per trial participant Paediatrics 17 132 Orthopaedics 10 7 Rheumatology 9 126 Respiratory 8 103 Neurology 8 143 Renal 6 27 General Medicine 6 32 Haematology 6 35 Endocrinology 5 162 Infectious Diseases 4 31 Geriatrics 3 18 Pain medicine 3 46 Palliative Care 3 20 Digestive Health 2 8 Obstetrics 2 27 General Surgery 2 30 Cardiology 2 30 Gynaecology 2 13 Mental Health 2 60 Grand Total 100 75 Quality In the pre-implementation phase, the ambient listening technology was set up to run silently in five outpatient clinics of consenting medical officers (‘super users’) and patients. Eighteen raw outputs from individual patient consultations were then compared to the clinician-generated medical records of the same consultations to evaluate note quality by SM, a senior clinician on the project team. This approach was chosen to isolate the quality of the initial ambient listening output, prior to clinician amendment. The modified Physician Documentation Quality Instrument (PDQI-9) is a validated tool that uses a 5-point Likert scale to score documentation quality against eight domains, with one representing poorest quality, and a score of five representing high quality. For each document, the maximum value of 40 represents a perfectly completed document [ 6 , 13 ]. Utility Saved and uploaded (final) electronic medical records initiated by scribe technology were compared to the original (or raw) scribe output in order to measure the average number of clinician amendments required to reach a satisfactory standard of appointment summarisation. The ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score is a widely used metric for evaluating the similarity between an automatic text summarisation output to a human-created reference [ 14 ]. ROUGE scores are expressed as percentages, representing the degree of similarity between the generated text and the reference summaries; an indication of required clinician modification efforts to the raw outputs [ 14 ]. Pairs of ambient listening generated notes and final medical records were compared using the Microsoft Word ‘compare’ and ‘track changes’ functionality. Differences were counted manually, and values inputted into the ROUGE-1 formula. Acceptance and impact : Semi-structured staff interviews : A convenience sampling method was employed to recruit six medical officers engaged as ‘super users’ in the planning of the trial, providing early feedback on solution functionality. In total, 13 semi-structured interviews (seven prior to the scribe introduction and six post) took place and were included in this study. Semi-structured interviews were conducted by AM, either an in-person or a virtual interview format. Interviews explored themes of medical record updating practice, frustrations and opportunities – prior and post the use of ambient listening technology. With participant consent, interviews were recorded and transcribed. Staff survey : Surveys were distributed to all 100 clinicians (medical officers, clinical nurses who ran nurse-led clinics and registrars) who participated in the trial, with 43 responses (43%). Surveys were distributed through a Microsoft Forms link and comprised six questions (Likert scale and open ended) to elicit participant reflections on their experience on the ease of using ambient listening technology, observations of the technology’s performance and its impact on their outpatient clinical practice. Responses were exported into Excel for analysis. Patient surveys : Patient surveys were distributed by medical officers who participated in the trial via a QR code link to an anonymous Microsoft Forms survey. This survey was based on previous work by Tierney et al and Mishra et al [ 6 , 15 ]. Responses from 22 surveys were included. Surveys comprised four questions (Likert scale and open ended) to elicit patient reflections on the clinician-patient interaction – amount of time the doctor spent speaking directly with them, time the doctor looked at the computer screen and the overall effect of the ambient listening on their visit compared to previous appointments. Responses were exported into Excel for analysis. Analysis Analysis followed a ‘complementarity’ approach to synthesising mixed-methods data, where distinct insights from qualitative and quantitative data sources were combined to achieve a greater breadth and depth of understanding than could be achieved using one in isolation. An ‘embedded’ approach was taken to integrate qualitative and quantitative data whereby each source of information provides answers to related questions simultaneously [ 12 ]. The data underwent a content analysis of deductively derived themes, whilst remaining open to inductive themes [ 16 ]. RESULTS Quality On average, the scribe technology produced equivalent, or slightly higher, quality patient notes compared to current clinician practice. Eighteen pairs of notes were assessed using the PDQI-9 tool (one clinician-created and one AI-generated). Clinician notes averaged a quality score 34.6/40 versus AI-generated notes which scored 37.06/40. These results were particularly influenced by ambient listening’s stronger performance on metrics related to “creating a more thorough and well-organised note”. Table 2 PDSQI analysis for AOPD and Ambient Listening notes generated for the same appointment Attribute Description of Ideal Note AOPD Note Average (0–5, 5 = best) Ambient Listening Average (0–5, 5 = best) Difference Accurate The note is true. It is free of incorrect information. 4.83 4.67 -0.17 Thorough The note is complete and free from omission and documents all of the issues of importance to the patient. 3.94 4.56 0.61 Useful The note is extremely relevant, providing valuable information and/or analysis. 4.33 4.61 0.28 Organized The note is well-formed and structured in a way that helps the reader understand the patient’s clinical course. 3.89 4.67 0.78 Comprehensible The note is clear, without ambiguity or sections that are difficult to understand. 4.33 4.56 0.22 Succinct The note is brief, to the point, and without redundancy. 4.17 4.61 0.44 Synthesized The note reflects the AI scribe’s understanding of the patient’s status and ability to develop a plan of care. 4.28 4.50 0.22 Internally Consistent No part of the note ignores or contradicts any other part. 4.78 4.89 0.11 Free from Hallucination The note is free of hallucination and only contains information verifiable by the transcript. n/a 4.83 4.83 Free from Bias The note is free of bias and contains only information verifiable by the transcript and not derived from characteristics of the patient or visit. n/a 5.00 5.00 Average 34.56 37.06 2.50 This finding is consistent with staff experience. Of staff survey respondents, 88% felt that ambient listening produced a good quality clinic note. Respondents corroborated that ambient listening improved the amount of relevant detail available for patients and clinicians involved in their care. “My note is more extensive than I would have without [ambient listening] ... more complete with it.” – Medical officer; interview 6 “Traditionally, I'd done very minimalist letters for NDIS because they take so much extra time, whereas now I'm providing a really comprehensive summary.” – Medical officer; interview 2 Utility ROUGE score analysis of 21 pairs of ambient-listening generated and final patient notes showed that, on average, 58% of AI-generated notes were used verbatim by clinicians in the final medical records. While ROUGE performance can be subjective and context specific, a median ROUGE F1 score of 0.58 indicates strong performance in capturing the essential content of outpatient conversations relative to other extractive summarisation research [ 14 ]. Table 2 displays an example comparative text produced by ambient listening technology and the corresponding text within a patient’s note, to illustrate how notes were amended and elaborated by clinicians throughout the trial. Table 3 Example excerpt of a comparative text with ROUGE score 0.624 Ambient listening excerpt OPD note excerpt Plan: • Repeat stool test to confirm negative C. diff status before ileostomy reversal. • Colonoscopy scheduled for xx/xx/202x. • Liaise with colorectal surgeon regarding timing of ileostomy reversal. • Stop Nexium (esomeprazole) permanently. • Avoid antibiotics unless absolutely necessary. • Discussed risk factors for C. diff recurrence and measures to reduce infection risk. Plan: Given slight ambiguity about the symptoms, need for surgery and patient anxiety, repeat stool test to confirm negative C. diff status before ileostomy reversal. Liaise with colorectal surgeon regarding timing of ileostomy reversal to discuss peri-op abs (and plan for amoxicillin) Stop Nexium (esomeprazole) indefintiely. Avoid antibiotics unless absolutely necessary. Continue amoxicillin 250mg PO daily as prophylaxis until surgery, stop whist on other antibiotics peripop then restart. If patient continues to not have symptoms of infection and does not have more than prophylactic antibiotics peri-op, and has two negative stool tests prior, I would not give prophylactic for C.diff. Discussed risk factors for C. diff recurrence and measures to reduce infection risk. Across clinical specialties, those with a greater emphasis on patient history – particularly chronic disease management, such as endocrinology – had lower ROUGE score performance. To explain this, these clinicians reported that they routinely refer to historical inputs from a patient’s medical record history to inform their consultation and complete their notes. As this content is not necessarily discussed during the recorded consult, it is not captured by the AI scribe. Before a consultation, if I've seen [the patient] I'll summarise any previous diagnosis, any previous assessments they've had, and any updates in the last year that I've seen them …for a new patient I will scout around for any other information that I could collect by the patient before I bring them in. So there's a little bit of a preparation work. – Medical officer; interview 2 Clinicians further reported that not all clinical information relevant to a patient’s medical record will be discussed - or discussed with the level of detail that care partners require to be informed of - requiring this detail to be added to the ambient-listening generated text. There are certain parts I still have to add in because I don't necessarily say everything. – Medical officer; interview 5 When we share CT reports and imaging reports … we say to the patient and ‘we found this and that’, but we give a sort of general outlining. The GP would want to know more precise terms. - Consultant 1 Hallucination and bias When trialling the technology, 47% of staff survey respondents reported hallucinations in ambient listening scribe outputs. In an AI context, hallucination refers to when an AI solution generates text that is factually incorrect, misleading, or fabricated, while presenting false information as if it were true. Of those who reported hallucinations, 20% reported that they occurred frequently. These hallucinations pertained to undiscussed and irrelevant clinical information, i.e. medications and investigations, drug allergies, clinical measurements (weight, blood pressure), living arrangements, smoking status, and information about the care of an unrelated disease. Further, 16% of staff survey respondents observed bias in outputs. At times the technology made incorrect determinations on the relevance and meaning of clinical discussion with a tendency towards particular diagnoses and their typical features. “[The technology] jumped to conclusions that were not my own” – Medical officer; survey 26 There are times in the clinic when …we talked about multiple things and [the Ambient Listening tool] emphasised the more minor problems. - Medical officer; interview 1 For some clinicians this deviation was within an accepted scope and not a major limitation. Nonetheless, they emphasised a need to continuously review the AI-generated outputs before accepting them as the final medical record: “I think we need to [adopt AI] with caution, simply because it's never going to be accurate 100% of the time. I know that busy doctors don't check things, so my concern is that people won't be checking what [the technology] is doing once [they] get used to it. … I think we should move forward with it, but with caution. And very clear ground rules.” – Medical officer; interview 6 Other staff indicated that this unreliability was a significant barrier to their adoption of the technology. “One time … it came out with this whole spiel. I thought, ‘what's all this stuff?’.… I don't know whether it actually happened or not happened, and I'm just trusting what it wrote down if I can't recall the whole consultation. …I thought “if I have to go through all this again, [then this] doesn't really benefit me” because this was [meant to] save my time… [double checking] could take just as long as if you were writing it yourself.” – Medical officer; interview 5 Staff experience On average, clinicians used ambient listening during four consultations per week (Median = 2, range = 0 to 38). Graph 1 shows the variation in ambient listening trial participation and usage across outpatient specialities. Paediatrics, orthopaedics, and rheumatology had the largest number of participants. However, usage per participant was highest among endocrinology, neurology and paediatric clinicians. Notably orthopaedics had a high participation representation, but low usage. Graph 1: Outpatient specialty participants and usage Prior to the commencement of the ambient listening trial, senior medical officers participating in semi-structured interviews reported that existing clinical documentation protocols carried a significant administrative burden and resulted in 1) a need for clinicians to work overtime, 2) hampered patient access to timely letters and information and 3) sub-optimal patient engagement during consultations. Together, these effects culminated in poorer work satisfaction. “I don't like the word burnout. But from that point of view in terms of satisfaction, [it’s] just that lack of it. It's not the same stress that people would have, say, if I'm going to run to do a CPR or something, but that stress level is much more of a low-grade constant kind of thing.” - Medical officer; interview 7 Survey and interview feedback following the conclusion of the ambient listening trial indicate that, for most clinicians, these administrative pressures were alleviated by ambient listening technology. Of survey respondents, 84% reported that ambient listening technology had a positive impact on their efficiency, alleviating administrative burden and releasing time for other high-value tasks. 79% of staff survey respondents also reported that the technology improved the quality of consultations through increased focus on patients. You no longer feel awful when you get the request for extra information between clinics. ... That used to be like, ‘Oh my God, that's so much work’. Whereas now I'm like, OK, look, ... it's much quicker. There's no longer the dread. – Medical officer; interview 2 However, staff identified template customisation as an up-front investment required for the technology to meet the needs of their clinical specialty, which sometimes presented a barrier to use. I had a go at making a template for consultant clinic and found it was a) time consuming and b) didn't produce as good of a note at the end of my time. … I would have loved to sort this myself but realistically I didn't have time to sit and tinker while patients are waiting in clinic. – Medical officer, survey. Layout, conciseness, and tone were frequently cited as requiring refinement. Subtleties across these dimensions reportedly had implications for clinical meaning, i.e. ambiguous assignment of responsibility for aspects of a patient’s care plan. Additionally, some summarisations removed important clinical detail, i.e. dates of events. Patient experience Patients reported a better appointment experience when ambient listening technology was in use compared to their experiences of usual care. Of patients surveyed, 68% reported that their clinician spent more time speaking directly with them, and less time than usual looking at the computer screen, compared to previous visits. 59% of patients reported that the ambient listening scribe had a positive effect on their visit. Concordantly, staff reported that this sentiment was reflected back to them by patients. “[Patients] are delighted for me to use [ambient listening]. I've had really good responses from people … [they know] that I'm going to use an approach that means I'm focused towards them and their needs.” - Medical officer; interview 3 Medical officers expressed satisfaction that ambient listening technology gave them greater opportunity to act in more wholistic ways to improve patient care, including active listening and promotion of health literacy. “When I'm having the more sensitive conversations around domestic violence or some of the kids are really in some unsafe environments, I know that I can give better eye contact and capture the detail I want.” – Medical officer; interview 2 I think it's nice being able to interact with the patient more. I like that. – Medical officer; interview 6 “[Ambient listening] gives me far more time to have a more complete consultation with the patient and form a better connection with them. ... I can use more complex terminology at times with the patients [and] then I back it up with the lay person language as well. I think that actually helps the quality of the discussions that I have.” – Medical officer; interview 3 DISCUSSION The findings from this evaluation indicate that ambient listening technology has the potential to address some of the reported challenges facing clinicians working in outpatient clinics. Burnout and job satisfaction Addressing clinician burnout – characterised by emotional exhaustion, depersonalisation, and a diminished sense of accomplishment – has become a significant healthcare challenge [ 17 ]. A sizable contributing factor is the growing burden of documentation, which, in turn, has been exacerbated by the advent of electronic medical records [ 18 ]. Documentation requirements can place persistently high demands upon clinician time, evidenced by “pyjama time” whereby clinicians work beyond their scheduled hours to complete patient documentation [ 9 ]. A further consequence of high documentation demands is that clinicians perceive a diminished or compromised opportunity to provide therapeutic care and attention to their patient during consultation. This study corroborates existing evidence that AI-powered medical scribes can reduce documentation burden on clinicians and reduce pyjama time that contributes to burnout [ 19 , 20 ]. Moreover, scribes can enhance the quality of the clinical encounter and facilitate higher levels of staff satisfaction with the quality of care they can provide [ 9 , 11 , 21 , 22 ]. Surveys and interviews reveal that, for most staff, ambient listening technology alleviated burnout through two primary mechanisms 1) reducing the need for clinicians to work overtime and 2) enhancing clinician satisfaction by enabling more meaningful patient interactions. Ambient listening technologies represent a potential intervention to address some of the drivers of burnout and foster a more sustainable clinical environment. Patient experience and health literacy Patient experience is significantly influenced by clinician engagement and health literacy levels, two interconnected factors [ 23 , 24 ]. The outpatient setting offers a unique opportunity to enhance health literacy [ 25 , 26 ]. Previous research has identified clinician time constraints as a primary barrier in accommodating variations in patients' health literacy during consultations, thereby limiting opportunities for tailored communication and assessment of patient comprehension [ 27 ]. The results from this study find that ambient listening technology can support a more therapeutic patient-physician relationship in outpatient appointments by mitigating the burden of clinical documentation. This finding aligns with other studies investigating the impact of ambient listening technology on patient experience [ 9 , 11 , 21 , 22 ]. Quality of documentation for continuity of care The importance of accurate and reliable current clinical documentation for patient outcomes is well-reported in the literature [ 28 ]. This study highlights the potential of AI-powered medical scribes to enhance documentation quality through improved accuracy and comprehensiveness, in agreement with previous research [ 6 , 7 , 29 ]. Results of this study show that ambient listening technology can generate high-quality summaries of patient appointments with a reasonable degree of accuracy. Enablers and challenges To realise the opportunities of ambient listening technology, this study corroborates two previously noted enablers for clinician adoption of ambient listening scribes, 1) technology usability, and 2) clinician interest [ 19 ]. Consistent with prior research, this study found that template customisation and familiarisation were central to usability, and, at times, were perceived as a barrier to use among time-poor clinicians [ 29 , 30 ]. This study found clinician willingness to overcome this barrier was influenced by their medical note requirements and their perceptions of ambient listening scribe value. For instance, the high rates of diminished orthopaedic participation in the trial is explainable by specialty-specific clinical documentation workflows and requirements. Within the orthopaedics specialty, junior doctors typically completed the consult notes, with a preference towards brevity. While more comprehensive clinical note detail is frequently considered a positive improvement created by ambient listening technology, for some clinicians the duplication of content from previous notes could be seen as redundant and reducing clarity. Furthermore, clinician interest was influenced by trust in ambient listening scribe outputs based on their observations of the frequency and potential consequences of hallucinations and bias. Hallucinations pose a significant risk to ongoing use and expanded adoption, particularly if clinicians neglect to thoroughly review and edit the generated notes and associated documentation [ 31 ]. Electronic medical record data showed that most staff identified the need to amend moderate amounts of the ambient listening-generated text before clinical use. There is the risk that trust and reliance on the tool – and subsequently complacency – will increase over time, which may perpetuate and amplify inaccuracies [ 28 ]. These findings underscore the critical importance of maintaining human oversight and frequently quality assurance of AI-assisted medical documentation processes [ 19 , 20 , 31 ]. Study data indicate that hallucinations may arise from suboptimal prompt engineering used during note customisation. Staff identified that customisation was a challenging aspect of ambient listening adoption, and it is important to note that customisation comes with increased risk of hallucination. In this study, bias was only observed in relation to the identification and emphasis of more common diseases and their symptoms. However, larger studies, with more data, are required to explore biases that may stem from social and structural determinants of health and inequitable dataset representation across different patient identities [ 31 , 32 ]. Limitations While data during this pilot was sufficient to gather a reliable indication of ambient listening scribe accuracy, acceptability, and impact in an outpatient context, further research is required to explore its impact in other hospital settings (i.e., inpatient ward rounds, emergency department interactions). Furthermore, medical officers were the focus of this study, and further research is required to understand if the findings translate to other staff groups (i.e., nurses and allied health professionals). The internal consistency of multiple data sources lends weight to the validity of these findings. Nevertheless, there is an opportunity to employ larger sample sizes to increase the reliability of accuracy and quality analysis. Moreover, the value proposition and health economic components were not evaluated, which is warranted by the implementation costs of this technology [ 33 , 34 ]. CONCLUSIONS This research studied ambient listening technology in real-world hospital outpatient practice, based on a 16-week trial among 100 staff and their patient’s consultations. Qualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can 1) produce an accurate summary of outpatient appointments, 2) enhance clinical note quality and 3) improve both clinician and patient experience. This technology has the potential to offer a solution to address some of the most significant challenges facing healthcare systems, such as burnout, low patient health literacy and delayed or suboptimal patient documentation. However, this result is tempered by evidence of hallucination, and some potential bias, that requires attention prior to any further adoption in routine clinical practice. List of acronyms and abbreviations Abbreviations DRG Diagnosis Related Group AI Artificial Intelligence GCHHS Gold Coast Hospital and Health Service (GCHHS) Declarations Ethics approval and consent to participate: Darling Downs Hospital and Health Service Human Research Ethics Committee provided ethical approval for this study (HREC/2024/QTDD/108398). A waiver of consent was approved by this committee for the use of health service data. Informed consent was received for all staff interview and survey and patient survey participants. This study adhered to the study adhered to the Declaration of Helsinki. Consent for publication: Not Applicable. Clinical trial number: Not applicable. Consent for publication: Not applicable. Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Competing interests: None to declare. Funding: Resources to complete this study, including the time of the researchers, were funded by the Gold Coast Health Hospital and Health Service. Authors’ contributions: R.S., A.B., and S.M. conceived this research. B.T. led the trial this study was based on. B.T., A.M where primarily responsible for data collection. R.S., S.M, A.B, B.T., and A.M contributed to analysis and manuscript finalisation. AM is responsible for the overall content as guarantor. AI use: Perplexity AI technology was used to search for relevant literature and improve the clarity of sentences. Acknowledgements: We thank the doctors that participated in this study for their time and valuable insights, particularly Dr. Angela Owens for her efforts in refining the ambient listening solution for her local context in consultation with the vendor. We also acknowledge the dedicated efforts of the project team that delivered the initiative evaluated and supported the distribution of surveys. References Majeed A, Car J, Sheikh A. Accuracy and completeness of electronic patient records in primary care. Fam Pract. 2008;25(4):213–4. 10.1093/fampra/cmn047 . Mathioudakis A, Rousalova I, Gagnat AA, et al. How to keep good clinical records. Breathe. 2016;12:369–73. 10.1183/20734735.018016 . Gesner E, Gazarian P, Dykes P. The Burden and Burnout in Documenting Patient Care: An Integrative Literature Review. Stud Health Technol Inform. 2019;264:1194–8. 10.3233/SHTI190415 . Gesner E, Dykes P, Zhang L, et al. Documentation Burden in Nursing and Its Role in Clinician Burnout Syndrome. Appl Clin Inf. 2022;13 5:983–90. 10.1055/s-0042-1757157 . Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol. 2021;11(4):803–9. 10.1007/s12553-021-00568-0 . Tierney AA, Gayre G, Hoberman B, et al. Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation. NEJM Catalyst. 2024;5(3):CAT. 23.0404. Balloch J, Sridharan S, Oldham G, et al. Use of an ambient artificial intelligence tool to improve quality of clinical documentation. Future Healthc J. 2024;11(3):100157. 10.1016/j.fhj.2024.100157 . Jayakumar P, Oude Nijhuis KD, Oosterhoff JHF, et al. Value-based Healthcare: Can Generative Artificial Intelligence and Large Language Models be a Catalyst for Value-based Healthcare? Clin Orthop Relat Res. 2023;481(10):1890–4. 10.1097/CORR.0000000000002854 . Tierney AA, Gayre G, Hoberman B, et al. Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation. NEJM Catalyst. 2024;5(3). 10.1056/cat.23.0404 . Duggan MJ, Gervase J, Schoenbaum A et al. Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency. JAMA Open. 2025. Galloway JL, Munroe D, Vohra-Khullar PD, et al. Impact of an Artificial Intelligence-Based Solution on Clinicians’ Clinical Documentation Experience: Initial Findings Using Ambient Listening Technology. J Gen Intern Med. 2024;39(13):2625–7. 10.1007/s11606-024-08924-2 . Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40:423–42. Stetson PD, Bakken S, Wrenn JO, et al. Assessing Electronic Note Quality Using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inf. 2012;3(2):164–74. 10.4338/aci-2011-11-ra-0070 . Akhmetov I, Mussabayev R, Gelbukh A. Reaching for upper bound ROUGE score of extractive summarization methods. PeerJ Comput Sci. 2022;8. 10.7717/peerj-cs.1103 . Mishra P, Kiang JC, Grant RW. Association of Medical Scribes in Primary Care With Physician Workflow and Patient Experience. JAMA Intern Med. 2018;178(11):1467–72. 10.1001/jamainternmed.2018.3956 . Miles M, Huberman A, Saldana J. Qualitative Data Analysis. USA: Sage; 2019. Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of Burnout Among Physicians: A Systematic Review. JAMA. 2018;320(11):1131–50. 10.1001/jama.2018.12777 . Moy AJ, Schwartz JM, Chen R, et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021;28(5):998–1008. 10.1093/jamia/ocaa325 . Shah SJ, Devon-Sand A, Ma SP, et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J Am Med Inf Assoc. 2025;32(2):375–80. 10.1093/jamia/ocae295 . Misurac J, Knake LA, Blum JM. Impact of Ambient Artificial Intelligence Notes on Provider Burnout. 2024. 10.1101/2024.07.18.24310656 Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribes in dermatology: Implications for practice. J Am Acad Dermatol. 2022;86(4):968–9. 10.1016/j.jaad.2021.11.011 . Ayer M. Relieving administrative burden on clinical staff with streamlined workflows and speech-recognition software. Br J Nurs. 2023;32(Sup16b):S4–9. 10.12968/bjon.2023.32.sup16b.s4 . Parker R. Health literacy: a challenge for American patients and their health care providers. Health Promot Int. 2000;15:277–83. 10.1093/HEAPRO/15.4.277 . Wynia M, Osborn C. Health Literacy and Communication Quality in Health Care Organizations. J Health Communication. 2010;15:102–15. 10.1080/10810730.2010.499981 . Schillinger D, Piette J, Grumbach K, et al. Closing the Loop: Physician Communication With Diabetic Patients Who Have Low Health Literacy. Arch Intern Med. 2003;163(1):83–90. 10.1001/archinte.163.1.83 . Rohwer E, Mojtahedzadeh N, Neumann FA, et al. The Role of Health Literacy among Outpatient Caregivers during the COVID-19 Pandemic. Int J Environ Res Public Health. 2021;18(22):11743. Murugesu L, Heijmans M, Rademakers J, et al. Challenges and solutions in communication with patients with low health literacy: Perspectives of healthcare providers. PLoS ONE. 2022;17(5):e0267782. 10.1371/journal.pone.0267782 . Weis JM, Levy PC. Copy, paste, and cloned notes in electronic health records: prevalence, benefits, risks, and best practice recommendations. Chest. 2014;145(3):632–8. 10.1378/chest.13 . Joseph J, Moore ZEH, Patton D, et al. The impact of implementing speech recognition technology on the accuracy and efficiency (time to complete) clinical documentation by nurses: A systematic review. J Clin Nurs. 2020;29(13–14):2125–37. 10.1111/jocn.15261 . Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol (Berl). 2021;11(4):803–9. 10.1007/s12553-021-00568-0 . Mess SA, Mackey AJ, Yarowsky DE. Artificial Intelligence Scribe and Large Language Model Technology in Healthcare Documentation: Advantages, Limitations, and Recommendations. Plast Reconstr Surg Glob Open. 2025;13(1):e6450. 10.1097/GOX.0000000000006450 . Pfohl SR, Cole-Lewis H, Sayres R, et al. A toolbox for surfacing health equity harms and biases in large language models. Nat Med. 2024;30(12):3590–600. 10.1038/s41591-024-03258-2 . Shah NH, Entwistle D, Pfeffer MA. Creation and Adoption of Large Language Models in Medicine. JAMA. 2023;330(9):866–9. 10.1001/jama.2023.14217 . Kotek H, Dockum R, Sun DQ. Gender bias and stereotypes in Large Language Models. arXivorg. 2023. 10.48550/arxiv.2308.14921 . Additional Declarations No competing interests reported. Supplementary Files SupplementaryFile1AmbientListeningQuestionnairesCombined.pdf Cite Share Download PDF Status: Published Journal Publication published 08 Jan, 2026 Read the published version in BMC Health Services Research → Version 1 posted Editorial decision: Revision requested 17 Oct, 2025 Reviews received at journal 17 Oct, 2025 Reviews received at journal 30 Sep, 2025 Reviewers agreed at journal 21 Sep, 2025 Reviewers agreed at journal 17 Sep, 2025 Reviewers agreed at journal 18 Aug, 2025 Reviewers invited by journal 24 Jul, 2025 Editor assigned by journal 21 Jul, 2025 Editor invited by journal 15 Jul, 2025 Submission checks completed at journal 14 Jul, 2025 First submitted to journal 14 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7033639","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":491363826,"identity":"ab25b4aa-c59e-4c2c-8a23-98f8088b404c","order_by":0,"name":"Salim Memon","email":"","orcid":"","institution":"Gold Coast Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"Salim","middleName":"","lastName":"Memon","suffix":""},{"id":491363827,"identity":"712b3b4b-4ab0-4e22-bda1-406ef14e5eac","order_by":1,"name":"Adam Brand","email":"","orcid":"","institution":"Gold Coast Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"Adam","middleName":"","lastName":"Brand","suffix":""},{"id":491363828,"identity":"9d83c557-0db2-457b-9461-b1e1d61f853c","order_by":2,"name":"Bianca Taylor","email":"","orcid":"","institution":"Gold Coast Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"Bianca","middleName":"","lastName":"Taylor","suffix":""},{"id":491363829,"identity":"1901550b-456f-4a48-b2ee-84f0ca4bab58","order_by":3,"name":"Adelaide Michael","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABT0lEQVRIie2Rz0rDQBCHJyzEy7a5yZaU9BW2FIrQ4LNsCKSXQnPsQbBSiBfxnMfQN0gZaC95gAV7aBB68lCpSA5F3a0oXcF/N5F8h2H47XwMwwJUVPxJhJUBJ6qxxro2HabqEjKAA+CfKGAotJHqUCiFfKUA2bU7hctvFOc8zLIy7nlwiGeb+MSnnZvJdB2UC6gjubIeRghOKvYVlq/E9IL3O+AGEzedRbS7mIVMiBU00I5JM0dg0lBADnhGOQZjN0gItZF2VaIUBI6Uk0YSqRlDacnherrlePqqPD3TTjrclIbSMhUuB6CeUIBWaonayAb2+xbrPvHVjKG08xXHJu+3E31L7TKkTEbdIxEh1bcg5D5t58t9xZuHRXG37bUct19s6OOx56ThrVz76NXnk+uiHDHPm5vnv2F/DKj+L6S6+TlKscpfzFdUVFT8V14ApC1+BOgSW6IAAAAASUVORK5CYII=","orcid":"","institution":"Gold Coast Hospital and Health Service","correspondingAuthor":true,"prefix":"","firstName":"Adelaide","middleName":"","lastName":"Michael","suffix":""},{"id":491363830,"identity":"5be24811-4d19-4777-b00e-5e51b434667f","order_by":4,"name":"Rachael Smithson","email":"","orcid":"","institution":"Gold Coast Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"Rachael","middleName":"","lastName":"Smithson","suffix":""}],"badges":[],"createdAt":"2025-07-03 03:08:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7033639/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7033639/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12913-025-13954-5","type":"published","date":"2026-01-08T15:58:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":87804391,"identity":"4f3fb86b-09e3-4796-b119-fdbfe8b64517","added_by":"auto","created_at":"2025-07-29 08:16:23","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":39110,"visible":true,"origin":"","legend":"\u003cp\u003eGraph 1: Outpatient specialty participants and usage\u003c/p\u003e","description":"","filename":"G1.png","url":"https://assets-eu.researchsquare.com/files/rs-7033639/v1/830ca927630dd799c7cb6f8d.png"},{"id":100069475,"identity":"f96c035d-bba3-48b8-9aab-b13d42b283ec","added_by":"auto","created_at":"2026-01-12 16:14:34","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":920181,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7033639/v1/8d14aa24-e250-4de0-831a-ab0715c1068e.pdf"},{"id":87805432,"identity":"b3db3ccb-23c6-420a-b941-c8a5d1a5c0c8","added_by":"auto","created_at":"2025-07-29 08:24:24","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":223430,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFile1AmbientListeningQuestionnairesCombined.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7033639/v1/c23c329a8201fe73fca0feeb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation","fulltext":[{"header":"What is already known on this topic ","content":"\u003cp\u003eAmbient listening scribe technology is being adopted at pace by health services in the hope that it will alleviate clinical documentation burden.\u003c/p\u003e\u003cp\u003e\u003cb\u003eWhat this study adds\u003c/b\u003e \u0026ndash; This study contributes evidence of ambient listening scribe technology performance in a real-world outpatient department application.\u003c/p\u003e\u003cp\u003e\u003cb\u003eHow this study might affect research, practice, or policy\u003c/b\u003e \u0026ndash; This study aims to inform practice in the implementation and use of ambient listening scribe technology in other health services considering adoption.\u003c/p\u003e"},{"header":"INTRODUCTION AND BACKGROUND","content":"\u003cp\u003eMedical records are a foundational communication platform for healthcare professionals, facilitating the coordination of patient care across clinical settings [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. These records enable information exchange among a patient\u0026rsquo;s care team, supporting the management of diagnostic procedures, treatment protocols, and ongoing care activities in both hospital and community environments [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Despite the importance of accurate and timely recording of medical notes, the burden of documentation experienced by clinicians represents a significant and well-recognised challenge facing the healthcare system [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eTo alleviate this burden, ambient listening medical scribes (\u0026lsquo;scribes\u0026rsquo;) are being adopted at pace [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Scribe technology collects real-time voice data during clinical encounters, instantaneously transforming this content into a summary in the clinician\u0026rsquo;s chosen format [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The basis of this technology is machine learning and deep learning algorithms that transform live transcripts into continuously higher quality and more meaningful clinical documentation outputs [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Evidence to date associates this technology with increased efficiency and productivity, improved note quality and note accuracy, and a better experience for both patients and clinicians during encounters [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan additionalcitationids=\"CR10\" citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. As with all artificial intelligence (AI) solutions, successful implementation requires this technology to be fit for purpose and accepted by those using it.\u003c/p\u003e\u003cp\u003eTo date, use of ambient listening scribes has not been evaluated in an Australian hospital outpatient department context, representing a gap in literature. In July 2024, Gold Coast Hospital and Health Service (GCHHS), Australia, commenced a trial of an ambient listening solution. Limited term licenses were purchased for 100 clinicians participating in a 16-week trial. The scribe solution was available to clinicians via their work laptop and smart devices. Clinicians were supported through group training sessions and on-demand support by a dedicated project team. Additionally, speakers and microphones were provided to support the use of scribes. This research aims to understand the value of ambient listening technology in routine hospital practice. Using a mixed-methods approach, it aims to produce an early evaluation of AI-enabled ambient listening scribe accuracy, acceptability and impact.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003e\u003cstrong\u003eStudy design\u003c/strong\u003e\u003cp\u003eThis research adopted a mixed-methods approach, which is noted for its suitability in application to complex effectiveness studies [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. The study has ethical approval from the Darling Downs Health Human Research Ethics Committee (HREC/2024/QTDD/108398) and data was collected between July and December 2024. Data collection and analysis focused on 100 clinicians trialling the technology in outpatient clinics. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e summarises the outpatient specialties involved in the pilot. In total 7,499 consultations were undertaken using the AI scribe technology during the trial.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eSummary of outpatient specialty trial participant numbers and average total consults per trial participant\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"3\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOutpatient specialty\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eCount of trial participants\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAverage of total consults per trial participant\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePaediatrics\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e17\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e132\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOrthopaedics\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e10\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e7\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eRheumatology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e9\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e126\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eRespiratory\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e8\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e103\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eNeurology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e8\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e143\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eRenal\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e6\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e27\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGeneral Medicine\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e6\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e32\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eHaematology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e6\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e35\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eEndocrinology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e5\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e162\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInfectious Diseases\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e4\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e31\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGeriatrics\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e18\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePain medicine\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e46\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePalliative Care\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e20\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eDigestive Health\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e8\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eObstetrics\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e27\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGeneral Surgery\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e30\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCardiology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e30\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGynaecology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e13\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eMental Health\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e60\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003eGrand Total\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e\u003cb\u003e100\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e\u003cb\u003e75\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eQuality\u003c/strong\u003e\u003c/p\u003e\u003cp\u003eIn the pre-implementation phase, the ambient listening technology was set up to run silently in five outpatient clinics of consenting medical officers (\u0026lsquo;super users\u0026rsquo;) and patients. Eighteen raw outputs from individual patient consultations were then compared to the clinician-generated medical records of the same consultations to evaluate note quality by SM, a senior clinician on the project team. This approach was chosen to isolate the quality of the initial ambient listening output, prior to clinician amendment. The modified Physician Documentation Quality Instrument (PDQI-9) is a validated tool that uses a 5-point Likert scale to score documentation quality against eight domains, with one representing poorest quality, and a score of five representing high quality. For each document, the maximum value of 40 represents a perfectly completed document [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eUtility\u003c/strong\u003e\u003cp\u003eSaved and uploaded (final) electronic medical records initiated by scribe technology were compared to the original (or raw) scribe output in order to measure the average number of clinician amendments required to reach a satisfactory standard of appointment summarisation. The ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score is a widely used metric for evaluating the similarity between an automatic text summarisation output to a human-created reference [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. ROUGE scores are expressed as percentages, representing the degree of similarity between the generated text and the reference summaries; an indication of required clinician modification efforts to the raw outputs [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Pairs of ambient listening generated notes and final medical records were compared using the Microsoft Word \u0026lsquo;compare\u0026rsquo; and \u0026lsquo;track changes\u0026rsquo; functionality. Differences were counted manually, and values inputted into the ROUGE-1 formula.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eAcceptance and impact\u003c/b\u003e:\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eSemi-structured staff interviews\u003c/span\u003e: A convenience sampling method was employed to recruit six medical officers engaged as \u0026lsquo;super users\u0026rsquo; in the planning of the trial, providing early feedback on solution functionality. In total, 13 semi-structured interviews (seven prior to the scribe introduction and six post) took place and were included in this study. Semi-structured interviews were conducted by AM, either an in-person or a virtual interview format. Interviews explored themes of medical record updating practice, frustrations and opportunities \u0026ndash; prior and post the use of ambient listening technology. With participant consent, interviews were recorded and transcribed.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eStaff survey\u003c/span\u003e: Surveys were distributed to all 100 clinicians (medical officers, clinical nurses who ran nurse-led clinics and registrars) who participated in the trial, with 43 responses (43%). Surveys were distributed through a Microsoft Forms link and comprised six questions (Likert scale and open ended) to elicit participant reflections on their experience on the ease of using ambient listening technology, observations of the technology\u0026rsquo;s performance and its impact on their outpatient clinical practice. Responses were exported into Excel for analysis.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003ePatient surveys\u003c/span\u003e: Patient surveys were distributed by medical officers who participated in the trial via a QR code link to an anonymous Microsoft Forms survey. This survey was based on previous work by Tierney et al and Mishra et al [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. Responses from 22 surveys were included. Surveys comprised four questions (Likert scale and open ended) to elicit patient reflections on the clinician-patient interaction \u0026ndash; amount of time the doctor spent speaking directly with them, time the doctor looked at the computer screen and the overall effect of the ambient listening on their visit compared to previous appointments. Responses were exported into Excel for analysis.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eAnalysis\u003c/strong\u003e\u003cp\u003eAnalysis followed a \u0026lsquo;complementarity\u0026rsquo; approach to synthesising mixed-methods data, where distinct insights from qualitative and quantitative data sources were combined to achieve a greater breadth and depth of understanding than could be achieved using one in isolation. An \u0026lsquo;embedded\u0026rsquo; approach was taken to integrate qualitative and quantitative data whereby each source of information provides answers to related questions simultaneously [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. The data underwent a content analysis of deductively derived themes, whilst remaining open to inductive themes [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e].\u003c/p\u003e\u003c/p\u003e"},{"header":"RESULTS","content":"\u003cp\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cb\u003eQuality\u003c/b\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eOn average, the scribe technology produced equivalent, or slightly higher, quality patient notes compared to current clinician practice. Eighteen pairs of notes were assessed using the PDQI-9 tool (one clinician-created and one AI-generated). Clinician notes averaged a quality score 34.6/40 versus AI-generated notes which scored 37.06/40. These results were particularly influenced by ambient listening\u0026rsquo;s stronger performance on metrics related to \u0026ldquo;creating a more thorough and well-organised note\u0026rdquo;.\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003ePDSQI analysis for AOPD and Ambient Listening notes generated for the same appointment\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"5\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAttribute\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDescription of Ideal Note\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAOPD Note Average (0\u0026ndash;5, 5\u0026thinsp;=\u0026thinsp;best)\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eAmbient Listening Average\u003c/p\u003e\u003cp\u003e(0\u0026ndash;5, 5\u0026thinsp;=\u0026thinsp;best)\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e\u003cp\u003eDifference\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAccurate\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is true. It is free of incorrect information.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.83\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.67\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e-0.17\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eThorough\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is complete and free from omission and documents all of the issues of importance to the patient.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e3.94\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.56\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.61\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eUseful\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is extremely relevant, providing valuable information and/or analysis.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.33\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.61\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.28\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOrganized\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is well-formed and structured in a way that helps the reader understand the patient\u0026rsquo;s clinical course.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e3.89\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.67\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.78\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eComprehensible\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is clear, without ambiguity or sections that are difficult to understand.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.33\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.56\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.22\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eSuccinct\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is brief, to the point, and without redundancy.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.17\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.61\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.44\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eSynthesized\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note reflects the AI scribe\u0026rsquo;s understanding of the patient\u0026rsquo;s status and ability to develop a plan of care.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.28\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.50\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.22\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInternally Consistent\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eNo part of the note ignores or contradicts any other part.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4.78\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.89\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e0.11\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFree from Hallucination\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is free of hallucination and only contains information verifiable by the transcript.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003en/a\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e4.83\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e4.83\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFree from Bias\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe note is free of bias and contains only information verifiable by the transcript and not derived from characteristics of the patient or visit.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003en/a\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e5.00\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e5.00\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003eAverage\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e34.56\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e37.06\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e\u003cp\u003e2.50\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eThis finding is consistent with staff experience. Of staff survey respondents, 88% felt that ambient listening produced a good quality clinic note. Respondents corroborated that ambient listening improved the amount of relevant detail available for patients and clinicians involved in their care.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;My note is more extensive than I would have without [ambient listening] ... more complete with it.\u0026rdquo; \u0026ndash; Medical officer; interview 6\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;Traditionally, I'd done very minimalist letters for NDIS because they take so much extra time, whereas now I'm providing a really comprehensive summary.\u0026rdquo; \u0026ndash; Medical officer; interview 2\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eUtility\u003c/b\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eROUGE score analysis of 21 pairs of ambient-listening generated and final patient notes showed that, on average, 58% of AI-generated notes were used verbatim by clinicians in the final medical records. While ROUGE performance can be subjective and context specific, a median ROUGE F1 score of 0.58 indicates strong performance in capturing the essential content of outpatient conversations relative to other extractive summarisation research [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e displays an example comparative text produced by ambient listening technology and the corresponding text within a patient\u0026rsquo;s note, to illustrate how notes were amended and elaborated by clinicians throughout the trial.\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eExample excerpt of a comparative text with ROUGE score 0.624\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"2\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAmbient listening excerpt\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eOPD note excerpt\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePlan:\u003c/p\u003e\u003cp\u003e\u0026bull; Repeat stool test to confirm negative C. diff status before ileostomy reversal.\u003c/p\u003e\u003cp\u003e\u0026bull; Colonoscopy scheduled for xx/xx/202x.\u003c/p\u003e\u003cp\u003e\u0026bull; Liaise with colorectal surgeon regarding timing of ileostomy reversal.\u003c/p\u003e\u003cp\u003e\u0026bull; Stop Nexium (esomeprazole) permanently.\u003c/p\u003e\u003cp\u003e\u0026bull; Avoid antibiotics unless absolutely necessary.\u003c/p\u003e\u003cp\u003e\u0026bull; Discussed risk factors for C. diff recurrence and measures to reduce infection risk.\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003ePlan:\u003c/p\u003e\u003cp\u003eGiven slight ambiguity about the symptoms, need for surgery and patient anxiety, repeat stool test to confirm negative C. diff status before ileostomy reversal.\u003c/p\u003e\u003cp\u003eLiaise with colorectal surgeon regarding timing of ileostomy reversal to discuss peri-op abs (and plan for amoxicillin)\u003c/p\u003e\u003cp\u003eStop Nexium (esomeprazole) indefintiely.\u003c/p\u003e\u003cp\u003eAvoid antibiotics unless absolutely necessary.\u003c/p\u003e\u003cp\u003eContinue amoxicillin 250mg PO daily as prophylaxis until surgery, stop whist on other antibiotics peripop then restart.\u003c/p\u003e\u003cp\u003eIf patient continues to not have symptoms of infection and does not have more than prophylactic antibiotics peri-op, and has two negative stool tests prior, I would not give prophylactic for C.diff.\u003c/p\u003e\u003cp\u003eDiscussed risk factors for C. diff recurrence and measures to reduce infection risk.\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eAcross clinical specialties, those with a greater emphasis on patient history \u0026ndash; particularly chronic disease management, such as endocrinology \u0026ndash; had lower ROUGE score performance. To explain this, these clinicians reported that they routinely refer to historical inputs from a patient\u0026rsquo;s medical record history to inform their consultation and complete their notes. As this content is not necessarily discussed during the recorded consult, it is not captured by the AI scribe.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eBefore a consultation, if I've seen [the patient] I'll summarise any previous diagnosis, any previous assessments they've had, and any updates in the last year that I've seen them \u0026hellip;for a new patient I will scout around for any other information that I could collect by the patient before I bring them in. So there's a little bit of a preparation work. \u0026ndash; Medical officer; interview 2\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eClinicians further reported that not all clinical information relevant to a patient\u0026rsquo;s medical record will be discussed - or discussed with the level of detail that care partners require to be informed of - requiring this detail to be added to the ambient-listening generated text.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eThere are certain parts I still have to add in because I don't necessarily say everything. \u0026ndash; Medical officer; interview 5\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eWhen we share CT reports and imaging reports \u0026hellip; we say to the patient and \u0026lsquo;we found this and that\u0026rsquo;, but we give a sort of general outlining. The GP would want to know more precise terms. - Consultant 1\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eHallucination and bias\u003c/b\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eWhen trialling the technology, 47% of staff survey respondents reported hallucinations in ambient listening scribe outputs. In an AI context, hallucination refers to when an AI solution generates text that is factually incorrect, misleading, or fabricated, while presenting false information as if it were true. Of those who reported hallucinations, 20% reported that they occurred frequently. These hallucinations pertained to undiscussed and irrelevant clinical information, i.e. medications and investigations, drug allergies, clinical measurements (weight, blood pressure), living arrangements, smoking status, and information about the care of an unrelated disease. Further, 16% of staff survey respondents observed bias in outputs. At times the technology made incorrect determinations on the relevance and meaning of clinical discussion with a tendency towards particular diagnoses and their typical features.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;[The technology] jumped to conclusions that were not my own\u0026rdquo; \u0026ndash; Medical officer; survey 26\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eThere are times in the clinic when \u0026hellip;we talked about multiple things and [the Ambient Listening tool] emphasised the more minor problems. - Medical officer; interview 1\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eFor some clinicians this deviation was within an accepted scope and not a major limitation. Nonetheless, they emphasised a need to continuously review the AI-generated outputs before accepting them as the final medical record:\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;I think we need to [adopt AI] with caution, simply because it's never going to be accurate 100% of the time. I know that busy doctors don't check things, so my concern is that people won't be checking what [the technology] is doing once [they] get used to it. \u0026hellip; I think we should move forward with it, but with caution. And very clear ground rules.\u0026rdquo; \u0026ndash; Medical officer; interview 6\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eOther staff indicated that this unreliability was a significant barrier to their adoption of the technology.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;One time \u0026hellip; it came out with this whole spiel. I thought, \u0026lsquo;what's all this stuff?\u0026rsquo;.\u0026hellip; I don't know whether it actually happened or not happened, and I'm just trusting what it wrote down if I can't recall the whole consultation. \u0026hellip;I thought \u0026ldquo;if I have to go through all this again, [then this] doesn't really benefit me\u0026rdquo; because this was [meant to] save my time\u0026hellip; [double checking] could take just as long as if you were writing it yourself.\u0026rdquo; \u0026ndash; Medical officer; interview 5\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eStaff experience\u003c/b\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eOn average, clinicians used ambient listening during four consultations per week (Median\u0026thinsp;=\u0026thinsp;2, range\u0026thinsp;=\u0026thinsp;0 to 38). Graph 1 shows the variation in ambient listening trial participation and usage across outpatient specialities. Paediatrics, orthopaedics, and rheumatology had the largest number of participants. However, usage per participant was highest among endocrinology, neurology and paediatric clinicians. Notably orthopaedics had a high participation representation, but low usage.\u003c/p\u003e\u003cp\u003eGraph 1: Outpatient specialty participants and usage\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e Prior to the commencement of the ambient listening trial, senior medical officers participating in semi-structured interviews reported that existing clinical documentation protocols carried a significant administrative burden and resulted in 1) a need for clinicians to work overtime, 2) hampered patient access to timely letters and information and 3) sub-optimal patient engagement during consultations. Together, these effects culminated in poorer work satisfaction.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;I don't like the word burnout. But from that point of view in terms of satisfaction, [it\u0026rsquo;s] just that lack of it. It's not the same stress that people would have, say, if I'm going to run to do a CPR or something, but that stress level is much more of a low-grade constant kind of thing.\u0026rdquo; - Medical officer; interview 7\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eSurvey and interview feedback following the conclusion of the ambient listening trial indicate that, for most clinicians, these administrative pressures were alleviated by ambient listening technology. Of survey respondents, 84% reported that ambient listening technology had a positive impact on their efficiency, alleviating administrative burden and releasing time for other high-value tasks. 79% of staff survey respondents also reported that the technology improved the quality of consultations through increased focus on patients.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eYou no longer feel awful when you get the request for extra information between clinics. ... That used to be like, \u0026lsquo;Oh my God, that's so much work\u0026rsquo;. Whereas now I'm like, OK, look, ... it's much quicker. There's no longer the dread. \u0026ndash; Medical officer; interview 2\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eHowever, staff identified template customisation as an up-front investment required for the technology to meet the needs of their clinical specialty, which sometimes presented a barrier to use.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI had a go at making a template for consultant clinic and found it was a) time consuming and b) didn't produce as good of a note at the end of my time. \u0026hellip; I would have loved to sort this myself but realistically I didn't have time to sit and tinker while patients are waiting in clinic. \u0026ndash; Medical officer, survey.\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eLayout, conciseness, and tone were frequently cited as requiring refinement. Subtleties across these dimensions reportedly had implications for clinical meaning, i.e. ambiguous assignment of responsibility for aspects of a patient\u0026rsquo;s care plan. Additionally, some summarisations removed important clinical detail, i.e. dates of events.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cb\u003ePatient experience\u003c/b\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003ePatients reported a better appointment experience when ambient listening technology was in use compared to their experiences of usual care. Of patients surveyed, 68% reported that their clinician spent more time speaking directly with them, and less time than usual looking at the computer screen, compared to previous visits. 59% of patients reported that the ambient listening scribe had a positive effect on their visit. Concordantly, staff reported that this sentiment was reflected back to them by patients.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;[Patients] are delighted for me to use [ambient listening]. I've had really good responses from people \u0026hellip; [they know] that I'm going to use an approach that means I'm focused towards them and their needs.\u0026rdquo; - Medical officer; interview 3\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e Medical officers expressed satisfaction that ambient listening technology gave them greater opportunity to act in more wholistic ways to improve patient care, including active listening and promotion of health literacy.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;When I'm having the more sensitive conversations around domestic violence or some of the kids are really in some unsafe environments, I know that I can give better eye contact and capture the detail I want.\u0026rdquo; \u0026ndash; Medical officer; interview 2\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eI think it's nice being able to interact with the patient more. I like that. \u0026ndash; Medical officer; interview 6\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;[Ambient listening] gives me far more time to have a more complete consultation with the patient and form a better connection with them. ... I can use more complex terminology at times with the patients [and] then I back it up with the lay person language as well. I think that actually helps the quality of the discussions that I have.\u0026rdquo; \u0026ndash; Medical officer; interview 3\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eThe findings from this evaluation indicate that ambient listening technology has the potential to address some of the reported challenges facing clinicians working in outpatient clinics.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eBurnout and job satisfaction\u003c/strong\u003e\u003cp\u003eAddressing clinician burnout \u0026ndash; characterised by emotional exhaustion, depersonalisation, and a diminished sense of accomplishment \u0026ndash; has become a significant healthcare challenge [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. A sizable contributing factor is the growing burden of documentation, which, in turn, has been exacerbated by the advent of electronic medical records [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. Documentation requirements can place persistently high demands upon clinician time, evidenced by \u0026ldquo;pyjama time\u0026rdquo; whereby clinicians work beyond their scheduled hours to complete patient documentation [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. A further consequence of high documentation demands is that clinicians perceive a diminished or compromised opportunity to provide therapeutic care and attention to their patient during consultation. This study corroborates existing evidence that AI-powered medical scribes can reduce documentation burden on clinicians and reduce pyjama time that contributes to burnout [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Moreover, scribes can enhance the quality of the clinical encounter and facilitate higher levels of staff satisfaction with the quality of care they can provide [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. Surveys and interviews reveal that, for most staff, ambient listening technology alleviated burnout through two primary mechanisms 1) reducing the need for clinicians to work overtime and 2) enhancing clinician satisfaction by enabling more meaningful patient interactions. Ambient listening technologies represent a potential intervention to address some of the drivers of burnout and foster a more sustainable clinical environment.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003ePatient experience and health literacy\u003c/strong\u003e\u003cp\u003ePatient experience is significantly influenced by clinician engagement and health literacy levels, two interconnected factors [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. The outpatient setting offers a unique opportunity to enhance health literacy [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]. Previous research has identified clinician time constraints as a primary barrier in accommodating variations in patients' health literacy during consultations, thereby limiting opportunities for tailored communication and assessment of patient comprehension [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. The results from this study find that ambient listening technology can support a more therapeutic patient-physician relationship in outpatient appointments by mitigating the burden of clinical documentation. This finding aligns with other studies investigating the impact of ambient listening technology on patient experience [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e].\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eQuality of documentation for continuity of care\u003c/strong\u003e\u003cp\u003eThe importance of accurate and reliable current clinical documentation for patient outcomes is well-reported in the literature [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. This study highlights the potential of AI-powered medical scribes to enhance documentation quality through improved accuracy and comprehensiveness, in agreement with previous research [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Results of this study show that ambient listening technology can generate high-quality summaries of patient appointments with a reasonable degree of accuracy.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eEnablers and challenges\u003c/strong\u003e\u003cp\u003eTo realise the opportunities of ambient listening technology, this study corroborates two previously noted enablers for clinician adoption of ambient listening scribes, 1) technology usability, and 2) clinician interest [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. Consistent with prior research, this study found that template customisation and familiarisation were central to usability, and, at times, were perceived as a barrier to use among time-poor clinicians [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e]. This study found clinician willingness to overcome this barrier was influenced by their medical note requirements and their perceptions of ambient listening scribe value. For instance, the high rates of diminished orthopaedic participation in the trial is explainable by specialty-specific clinical documentation workflows and requirements. Within the orthopaedics specialty, junior doctors typically completed the consult notes, with a preference towards brevity. While more comprehensive clinical note detail is frequently considered a positive improvement created by ambient listening technology, for some clinicians the duplication of content from previous notes could be seen as redundant and reducing clarity.\u003c/p\u003e\u003c/p\u003e\u003cp\u003eFurthermore, clinician interest was influenced by trust in ambient listening scribe outputs based on their observations of the frequency and potential consequences of hallucinations and bias. Hallucinations pose a significant risk to ongoing use and expanded adoption, particularly if clinicians neglect to thoroughly review and edit the generated notes and associated documentation [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e]. Electronic medical record data showed that most staff identified the need to amend moderate amounts of the ambient listening-generated text before clinical use. There is the risk that trust and reliance on the tool \u0026ndash; and subsequently complacency \u0026ndash; will increase over time, which may perpetuate and amplify inaccuracies [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. These findings underscore the critical importance of maintaining human oversight and frequently quality assurance of AI-assisted medical documentation processes [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eStudy data indicate that hallucinations may arise from suboptimal prompt engineering used during note customisation. Staff identified that customisation was a challenging aspect of ambient listening adoption, and it is important to note that customisation comes with increased risk of hallucination.\u003c/p\u003e\u003cp\u003eIn this study, bias was only observed in relation to the identification and emphasis of more common diseases and their symptoms. However, larger studies, with more data, are required to explore biases that may stem from social and structural determinants of health and inequitable dataset representation across different patient identities [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e].\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003cp\u003eWhile data during this pilot was sufficient to gather a reliable indication of ambient listening scribe accuracy, acceptability, and impact in an outpatient context, further research is required to explore its impact in other hospital settings (i.e., inpatient ward rounds, emergency department interactions). Furthermore, medical officers were the focus of this study, and further research is required to understand if the findings translate to other staff groups (i.e., nurses and allied health professionals). The internal consistency of multiple data sources lends weight to the validity of these findings. Nevertheless, there is an opportunity to employ larger sample sizes to increase the reliability of accuracy and quality analysis. Moreover, the value proposition and health economic components were not evaluated, which is warranted by the implementation costs of this technology [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e].\u003c/p\u003e\u003c/p\u003e"},{"header":"CONCLUSIONS","content":"\u003cp\u003eThis research studied ambient listening technology in real-world hospital outpatient practice, based on a 16-week trial among 100 staff and their patient\u0026rsquo;s consultations. Qualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can 1) produce an accurate summary of outpatient appointments, 2) enhance clinical note quality and 3) improve both clinician and patient experience. This technology has the potential to offer a solution to address some of the most significant challenges facing healthcare systems, such as burnout, low patient health literacy and delayed or suboptimal patient documentation. However, this result is tempered by evidence of hallucination, and some potential bias, that requires attention prior to any further adoption in routine clinical practice.\u003c/p\u003e\u003cp\u003e\u003cb\u003eList of acronyms and abbreviations\u003c/b\u003e\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 29.7837%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDRG\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 70.2163%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDiagnosis Related Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 29.7837%;\"\u003e\n \u003cp\u003eAI\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 70.2163%;\"\u003e\n \u003cp\u003eArtificial Intelligence\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 29.7837%;\"\u003e\n \u003cp\u003eGCHHS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 70.2163%;\"\u003e\n \u003cp\u003eGold Coast Hospital and Health Service (GCHHS)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate:\u0026nbsp;\u003c/strong\u003eDarling Downs Hospital and Health Service Human Research Ethics Committee provided ethical approval for this study (HREC/2024/QTDD/108398). A waiver of consent was approved by this committee for the use of health service data. Informed consent was received for all staff interview and survey and patient survey participants. This study adhered to the study adhered to the Declaration of Helsinki.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003e Not Applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical trial number:\u003c/strong\u003e Not applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003e Not applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u0026nbsp;\u003c/strong\u003eThe datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u0026nbsp;\u003c/strong\u003eNone to declare.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003eResources to complete this study, including the time of the researchers, were funded by the Gold Coast Health Hospital and Health Service.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026rsquo; contributions:\u0026nbsp;\u003c/strong\u003eR.S., A.B., and S.M. conceived this research. B.T. led the trial this study was based on. B.T., A.M where primarily responsible for data collection. R.S., S.M, A.B, B.T., and A.M contributed to analysis and manuscript finalisation. AM is responsible for the overall content as guarantor.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAI use:\u003c/strong\u003e Perplexity AI technology was used to search for relevant literature and improve the clarity of sentences.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements:\u003c/strong\u003e We thank the doctors that participated in this study for their time and valuable insights, particularly Dr. Angela Owens for her efforts in refining the ambient listening solution for her local context in consultation with the vendor. We also acknowledge the dedicated efforts of the project team that delivered the initiative evaluated and supported the distribution of surveys.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eMajeed A, Car J, Sheikh A. Accuracy and completeness of electronic patient records in primary care. Fam Pract. 2008;25(4):213\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/fampra/cmn047\u003c/span\u003e\u003cspan address=\"10.1093/fampra/cmn047\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMathioudakis A, Rousalova I, Gagnat AA, et al. How to keep good clinical records. Breathe. 2016;12:369\u0026ndash;73. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1183/20734735.018016\u003c/span\u003e\u003cspan address=\"10.1183/20734735.018016\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGesner E, Gazarian P, Dykes P. The Burden and Burnout in Documenting Patient Care: An Integrative Literature Review. Stud Health Technol Inform. 2019;264:1194\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3233/SHTI190415\u003c/span\u003e\u003cspan address=\"10.3233/SHTI190415\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGesner E, Dykes P, Zhang L, et al. Documentation Burden in Nursing and Its Role in Clinician Burnout Syndrome. Appl Clin Inf. 2022;13 5:983\u0026ndash;90. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1055/s-0042-1757157\u003c/span\u003e\u003cspan address=\"10.1055/s-0042-1757157\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGhatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol. 2021;11(4):803\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s12553-021-00568-0\u003c/span\u003e\u003cspan address=\"10.1007/s12553-021-00568-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTierney AA, Gayre G, Hoberman B, et al. Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation. NEJM Catalyst. 2024;5(3):CAT. 23.0404.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBalloch J, Sridharan S, Oldham G, et al. Use of an ambient artificial intelligence tool to improve quality of clinical documentation. Future Healthc J. 2024;11(3):100157. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.fhj.2024.100157\u003c/span\u003e\u003cspan address=\"10.1016/j.fhj.2024.100157\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJayakumar P, Oude Nijhuis KD, Oosterhoff JHF, et al. Value-based Healthcare: Can Generative Artificial Intelligence and Large Language Models be a Catalyst for Value-based Healthcare? Clin Orthop Relat Res. 2023;481(10):1890\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/CORR.0000000000002854\u003c/span\u003e\u003cspan address=\"10.1097/CORR.0000000000002854\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTierney AA, Gayre G, Hoberman B, et al. Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation. NEJM Catalyst. 2024;5(3). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1056/cat.23.0404\u003c/span\u003e\u003cspan address=\"10.1056/cat.23.0404\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDuggan MJ, Gervase J, Schoenbaum A et al. Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency. JAMA Open. 2025.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGalloway JL, Munroe D, Vohra-Khullar PD, et al. Impact of an Artificial Intelligence-Based Solution on Clinicians\u0026rsquo; Clinical Documentation Experience: Initial Findings Using Ambient Listening Technology. J Gen Intern Med. 2024;39(13):2625\u0026ndash;7. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11606-024-08924-2\u003c/span\u003e\u003cspan address=\"10.1007/s11606-024-08924-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePalinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40:423\u0026ndash;42.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eStetson PD, Bakken S, Wrenn JO, et al. Assessing Electronic Note Quality Using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inf. 2012;3(2):164\u0026ndash;74. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.4338/aci-2011-11-ra-0070\u003c/span\u003e\u003cspan address=\"10.4338/aci-2011-11-ra-0070\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAkhmetov I, Mussabayev R, Gelbukh A. Reaching for upper bound ROUGE score of extractive summarization methods. PeerJ Comput Sci. 2022;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.7717/peerj-cs.1103\u003c/span\u003e\u003cspan address=\"10.7717/peerj-cs.1103\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMishra P, Kiang JC, Grant RW. Association of Medical Scribes in Primary Care With Physician Workflow and Patient Experience. JAMA Intern Med. 2018;178(11):1467\u0026ndash;72. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jamainternmed.2018.3956\u003c/span\u003e\u003cspan address=\"10.1001/jamainternmed.2018.3956\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMiles M, Huberman A, Saldana J. Qualitative Data Analysis. USA: Sage; 2019.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRotenstein LS, Torre M, Ramos MA, et al. Prevalence of Burnout Among Physicians: A Systematic Review. JAMA. 2018;320(11):1131\u0026ndash;50. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jama.2018.12777\u003c/span\u003e\u003cspan address=\"10.1001/jama.2018.12777\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMoy AJ, Schwartz JM, Chen R, et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021;28(5):998\u0026ndash;1008. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/jamia/ocaa325\u003c/span\u003e\u003cspan address=\"10.1093/jamia/ocaa325\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShah SJ, Devon-Sand A, Ma SP, et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J Am Med Inf Assoc. 2025;32(2):375\u0026ndash;80. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/jamia/ocae295\u003c/span\u003e\u003cspan address=\"10.1093/jamia/ocae295\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMisurac J, Knake LA, Blum JM. Impact of Ambient Artificial Intelligence Notes on Provider Burnout. 2024. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1101/2024.07.18.24310656\u003c/span\u003e\u003cspan address=\"10.1101/2024.07.18.24310656\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGhatnekar S, Faletsky A, Nambudiri VE. Digital scribes in dermatology: Implications for practice. J Am Acad Dermatol. 2022;86(4):968\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.jaad.2021.11.011\u003c/span\u003e\u003cspan address=\"10.1016/j.jaad.2021.11.011\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAyer M. Relieving administrative burden on clinical staff with streamlined workflows and speech-recognition software. Br J Nurs. 2023;32(Sup16b):S4\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.12968/bjon.2023.32.sup16b.s4\u003c/span\u003e\u003cspan address=\"10.12968/bjon.2023.32.sup16b.s4\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eParker R. Health literacy: a challenge for American patients and their health care providers. Health Promot Int. 2000;15:277\u0026ndash;83. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/HEAPRO/15.4.277\u003c/span\u003e\u003cspan address=\"10.1093/HEAPRO/15.4.277\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWynia M, Osborn C. Health Literacy and Communication Quality in Health Care Organizations. J Health Communication. 2010;15:102\u0026ndash;15. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1080/10810730.2010.499981\u003c/span\u003e\u003cspan address=\"10.1080/10810730.2010.499981\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSchillinger D, Piette J, Grumbach K, et al. Closing the Loop: Physician Communication With Diabetic Patients Who Have Low Health Literacy. Arch Intern Med. 2003;163(1):83\u0026ndash;90. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/archinte.163.1.83\u003c/span\u003e\u003cspan address=\"10.1001/archinte.163.1.83\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRohwer E, Mojtahedzadeh N, Neumann FA, et al. The Role of Health Literacy among Outpatient Caregivers during the COVID-19 Pandemic. Int J Environ Res Public Health. 2021;18(22):11743.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMurugesu L, Heijmans M, Rademakers J, et al. Challenges and solutions in communication with patients with low health literacy: Perspectives of healthcare providers. PLoS ONE. 2022;17(5):e0267782. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1371/journal.pone.0267782\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0267782\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWeis JM, Levy PC. Copy, paste, and cloned notes in electronic health records: prevalence, benefits, risks, and best practice recommendations. Chest. 2014;145(3):632\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1378/chest.13\u003c/span\u003e\u003cspan address=\"10.1378/chest.13\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJoseph J, Moore ZEH, Patton D, et al. The impact of implementing speech recognition technology on the accuracy and efficiency (time to complete) clinical documentation by nurses: A systematic review. J Clin Nurs. 2020;29(13\u0026ndash;14):2125\u0026ndash;37. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/jocn.15261\u003c/span\u003e\u003cspan address=\"10.1111/jocn.15261\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGhatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol (Berl). 2021;11(4):803\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s12553-021-00568-0\u003c/span\u003e\u003cspan address=\"10.1007/s12553-021-00568-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMess SA, Mackey AJ, Yarowsky DE. Artificial Intelligence Scribe and Large Language Model Technology in Healthcare Documentation: Advantages, Limitations, and Recommendations. Plast Reconstr Surg Glob Open. 2025;13(1):e6450. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/GOX.0000000000006450\u003c/span\u003e\u003cspan address=\"10.1097/GOX.0000000000006450\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePfohl SR, Cole-Lewis H, Sayres R, et al. A toolbox for surfacing health equity harms and biases in large language models. Nat Med. 2024;30(12):3590\u0026ndash;600. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41591-024-03258-2\u003c/span\u003e\u003cspan address=\"10.1038/s41591-024-03258-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShah NH, Entwistle D, Pfeffer MA. Creation and Adoption of Large Language Models in Medicine. JAMA. 2023;330(9):866\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jama.2023.14217\u003c/span\u003e\u003cspan address=\"10.1001/jama.2023.14217\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKotek H, Dockum R, Sun DQ. Gender bias and stereotypes in Large Language Models. arXivorg. 2023. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arxiv.2308.14921\u003c/span\u003e\u003cspan address=\"10.48550/arxiv.2308.14921\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-health-services-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bhsr","sideBox":"Learn more about [BMC Health Services Research](http://bmchealthservres.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/BHSR/default.aspx","title":"BMC Health Services Research","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7033639/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7033639/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground: \u003c/strong\u003eIn 2024, Gold Coast Hospital and Health Service outpatient division initiated a 16-week trial of artificial intelligence (AI)-enabled ambient listening scribe technology. The objective of this pilot study was to evaluate the acceptability and impact of scribe technology in producing clinical notes and discharge summaries among outpatient clinicians and patients receiving care.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e A mixed method research design combined analysis of data from patient and staff surveys, staff interviews, scribe outputs and electronic medical records across a breadth of outpatient specialties.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults: \u003c/strong\u003eBy and large, ambient listening technology was associated with positive patient and staff experience. On average, 58% of scribe outputs were accepted without modification into the electronic outpatient note. There was limited evidence of bias in outputs, however there was some evidence of hallucination or incorrect outputs.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions: \u003c/strong\u003eQualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can 1) produce an accurate summary of outpatient appointments, 2) enhance clinical note quality and 3) improve both clinician and patient experience.\u003c/p\u003e","manuscriptTitle":"Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-29 08:16:19","doi":"10.21203/rs.3.rs-7033639/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-10-17T23:54:58+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-17T06:47:07+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-30T04:41:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"217830294525871728199857275699390144071","date":"2025-09-21T09:38:00+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"191503610546875324850699840865075240720","date":"2025-09-17T16:43:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"21804114795783465568306385581819247208","date":"2025-08-18T23:20:48+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-24T15:59:44+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-21T14:35:18+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-07-15T20:13:04+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-14T22:57:26+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Health Services Research","date":"2025-07-14T22:54:05+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-health-services-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bhsr","sideBox":"Learn more about [BMC Health Services Research](http://bmchealthservres.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/BHSR/default.aspx","title":"BMC Health Services Research","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e0e3df87-8456-40f4-bb98-dc23af494213","owner":[],"postedDate":"July 29th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-01-12T16:07:51+00:00","versionOfRecord":{"articleIdentity":"rs-7033639","link":"https://doi.org/10.1186/s12913-025-13954-5","journal":{"identity":"bmc-health-services-research","isVorOnly":false,"title":"BMC Health Services Research"},"publishedOn":"2026-01-08 15:58:00","publishedOnDateReadable":"January 8th, 2026"},"versionCreatedAt":"2025-07-29 08:16:19","video":"","vorDoi":"10.1186/s12913-025-13954-5","vorDoiUrl":"https://doi.org/10.1186/s12913-025-13954-5","workflowStages":[]},"version":"v1","identity":"rs-7033639","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7033639","identity":"rs-7033639","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}