Enhancing Patient Participation in Emergency Department through Patient-Friendly Clinical Notes Generated by Large Language Models

Enhancing Patient Participation in Emergency Department through Patient-Friendly Clinical Notes Generated by Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Enhancing Patient Participation in Emergency Department through Patient-Friendly Clinical Notes Generated by Large Language Models Sung-In Kim, Joonyoung Park, Taewan Kim, Woosuk Seo, Taerim Kim, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5958826/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background To achieve patient-centered care (PCC), it is essential to provide patients with sufficient information about their medical conditions and treatment processes to actively engage in their care. However, the busy and crowded nature of the emergency department (ED) setting, combined with differences in communication styles, varying levels of health literacy, and other barriers between clinicians and patients, makes effective communication and participation challenging. Recent studies have focused on applying large language model (LLM) technologies to create patient-friendly documents that present patients’ medical information in plain language with the goal of enhancing patient understanding and participation. Nevertheless, further research is needed to evaluate the impact of LLM-generated patient-friendly documents on improving patient participation, particularly in high-pressure settings like the ED. Objective This study aimed to develop patient-friendly clinical notes (PFCNs) generated by LLM, which transform clinicians’ clinical notes into patient-friendly documents for use in ED consultations, and to evaluate whether PFCNs could enhance patient participation. Methods A preliminary study was conducted to identify the contents, format, and scenarios of PFCNs in the ED setting by conducting individual interviews with patients (n = 9) and clinicians (n = 7). Based on these findings, we developed a system which generates PFCNs using the GPT-4o model, transforming clinicians’ clinical notes from ED consultations into patient-friendly documents through zero-shot learning. To evaluate the efficacy of PFCNs, we conducted a simulated ED consultation role-play with patients (n = 20) and clinicians (n = 10), who were provided with PFCNs. After the simulations, the participants completed the scales to assess the quality of the documents and patient participation questionnaires (PPQ) during pre- and post- simulations, followed by post-individual interviews to explore how PFCNs supported patient participation. Results We confirmed that the LLM successfully generated PFCNs (n = 120) suitable for patient comprehension, as indicated by a Patient Education Materials Assessment Tool (PEMAT) understandability score of 87.2 (> 70%) validated by both clinicians and patients. Compared to previous ED visits without PFCNs, patients who received the documents demonstrated a significantly higher PPQ scale ( P < 0.05). In line with the quantitative results in patients’ participation, post-interview results revealed that PFCNs supported communication with clinicians, improved understanding of clinical information, provided emotional reassurance, and strengthened patient-clinician relationships. However, two key challenges were identified regarding the content and utility of the documents: (1) potential risks from inaccuracies in PFCNs and (2) differing perspectives between patients and clinicians on the documents. Based on these findings, we suggested practical strategies for implementing PFCNs in ED settings. Conclusions This study demonstrates that PFCNs enhanced patient participation in ED consultations, supporting patient communication, decision-making, and relationship-building with healthcare providers. These findings suggest that PFCNs not only promote patient-centered care by supporting active patient participation but also have the potential to improve ED consultation and workflow effectively. Health sciences/Health care/Health services Health sciences/Medical research Biological sciences/Computational biology and bioinformatics/Machine learning patient-centered care patient participation large language model emergency department clinical note patient-friendly clinical note Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Background Patient-centered care (PCC) emphasizes the partnership between clinicians and patients, focusing on integrating patients' health needs and personal preferences into the medical decision-making process 1 . Central to PCC is the promotion of patient autonomy, which requires active patient participation in various aspects of care, such as shared decision-making and communication with healthcare providers 2–4 . Active patient participation has been shown to enhance patients' understanding and management of their symptoms 5 , increase patient satisfaction 6 , and alleviate emotional burdens such as anxiety and depression 7,8 . Moreover, patient participation has demonstrated clinical benefits, including improved health outcomes (e.g., better blood pressure and glucose control) 9 , reduced hospital visit durations 10 , and enhanced cost-effectiveness 11 . For patients to participate effectively in their care, it is crucial to provide them with sufficient and accessible clinical information (e.g. expected diagnosis, treatment options, examination and test results) 12 . Several research studies have focused on strategies for effectively sharing information during medical processes, such as providing interactive Chatbot-based informed consent 13 , identifying doctors’ communication strategies 14 and providing discharge education 15 . For instance, Hess et al. (2016) demonstrated that providing low-risk acute coronary syndrome (ACS) patients with risk information increased their understanding, encouraged engagement, and safely reduced unnecessary admissions for cardiac testing 14,16 . However, in the emergency department (ED) setting, where the environment is fast-paced and highly stressful for both patients and healthcare providers, there are challenges to facilitating patient participation 1,17,18 . First, the information gap between clinicians and patients often leads to communication breakdowns, making it difficult for patients to receive adequate information. Clinicians tend to focus on obtaining patient data and delivering only essential information, which can decrease patient satisfaction 15,19 . Second, differences in communication styles between clinicians and patients can create further barriers. According to Park et al., clinicians often use a data-driven communication style aimed at information gathering, while patients prefer a narrative-driven approach to discuss their concerns 20 . Although data-focused communication may efficiently identify medical issues, it often leads to decreased patient empathy and increased psychological distress. To address these challenges, sharing clinical notes with patients could offer a valuable opportunity to bridge the information gap between patients and clinicians 21 . Clinical notes are comprehensive records that capture essential patient data, diagnostic reasoning, and treatment plans of the clinicians during patients’ consultation . However, complex medical concepts and jargon within clinical notes hinder patient comprehension and may cause emotional burden 22 . By making clinical notes more accessible, patients can better understand their medical conditions, engage in informed discussions with their providers, and actively participate in their care 23,24 . For example, Bala et al. reported that patients viewed patient-friendly format of clinical notes as useful for understanding their medical care, improving patient-physician relationships, and supporting self-management 25 . Recently, there is growing interest in leveraging Large language models (LLMs) to make patient-friendly documents to bridge the information gap between clinicians and patients. Large language models (LLMs) are machine learning-based AI models designed to understand and analyze text 26 . These models have advanced significantly, enabling them to generate highly coherent and contextually appropriate responses based on user prompts 27 . As research on the application of LLM technology in clinical practice (e.g. generating electronic medical record (EMR), patient case adjudication) continues to expand 28–30 , recent studies have explored the potential of LLMs to generate patient-friendly documents, such as discharge summaries and simplified clinical notes 25,31,32 . Previous related studies showed that LLM-generated documents can be used to bridge the information gap between patients and clinicians by providing clear, tailored explanations that enhance patient understanding and foster better communication between patients and clinicians. However, most of these studies have not specifically addressed the unique challenges of high-pressure environments like the ED, where time constraints and communication barriers are more pronounced. Therefore, this research aims to fill this gap by evaluating the impact of LLM-generated documents on improving patient participation in the ED setting, where effective communication and participation is particularly critical yet difficult to achieve. Objective This study aims to evaluate whether LLM-generated PFCNs can enhance patient participation in ED consultations. As a preliminary study, we conducted individual interviews with clinicians and patients to determine the appropriate content, format, and applicable scenarios for LLM-generated PFCNs in the ED setting. Based on the findings from these interviews, we developed LLM-generated PFCNs based on clinicians’ clinical notes during ED consultations. Finally, we conducted a simulated ED role-play study involving both patients and clinicians, during which the LLM-generated PFCNs were provided. Participants completed quality assessment scales for the documents and patient participation questionnaires, followed by post-interviews to assess the documents' efficacy on patient engagement in the consultation process. Methods Ethical Considerations & Recruitments This study protocol and participant recruitment plan were approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB No. 2024-11-023-001). The inclusion criteria for clinician participants were: 1) adults aged 18 or older, 2) those with experience in ED care, 3) individuals capable of effective communication, and 4) those who agreed to the purpose and content of the study. The inclusion criteria for patient participants were: 1) adults aged 18 or older, 2) individuals with personal experience receiving ED care, 3) those capable of effective communication, and 4) those who agreed to the purpose and content of the study. We aimed to recruit 20 physician participants and 30 patient participants. Recruitment was conducted by posting announcements on the offline bulletin boards of the researchers' affiliated institution (Samsung Medical Center) and on an online community platform Everytime (Vinu Labs Inc. https://everytime.kr/) dedicated to Seoul National University students. Recruited participants were informed of the study’s purpose and methods and were asked to provide written consent. Participants were also informed that personal information (e.g., age, gender) and the data (e.g. interviews, scales, LLM-generated documents) collected during the study would be anonymized and destroyed after the study concluded. Each participant who completed the study received a 20 USD compensation. Preliminary Study: Interview for developing PFCNs using LLM In this study, we conducted a preliminary study to explore the content, delivery context and concerns of PFCNs provided during ED care. We conducted individual interviews with nine patients who had previous ED experiences and seven clinicians who had provided care in the ED within one hour period. In the interview, we confirmed the experiences and challenges in ED care and communication of the participants. In addition, we asked them about the type of information provided during ED care and how it was delivered. Lastly, we identified the participants’ expectations and challenges on the LLM-generated PFCNs in the ED setting. Based on the interview results, we identified: 1) the concept proof of LLM-generated PFCNs, 2) the content of the documents, 3) the appropriate scenarios and target patient groups for their use, and 4) concerns regarding the accuracy of the documents. Concept Proof of LLM-Generated PFCNs Through the interview results, we identified the need for PFCNs to bridge the information gap between clinicians and patients in the ED setting. Many patients reported difficulty understanding medical information due to the fast-paced environment and the complex medical terminology often used by clinicians. On the other hand, clinicians found it challenging to provide detailed explanations to fulfill patients’ information needs due to their heavy workload. Given these challenges, both patients and clinicians expressed support for incorporating PFCNs that include the patient’s condition, treatment process, and clinical information (e.g. diagnosis, test results) into the ED consultation process. Additionally, we confirmed that LLM can serve as effective technology to generate PFCNs that met the expectations of both patients and clinicians. Interview results revealed that patients desired documents with empathetic and reassuring language tailored to their individual symptoms and circumstances (e.g. Tommy, you came to the emergency room today with abdominal pain—how difficult has it been for you?). Clinicians, in turn, expressed interest in using LLMs to create patient-friendly documents supplemented with additional information to support their explanations (e.g., explaining the purpose of laboratory or X-ray tests or providing information about appendicitis). Given that LLMs can generate documents tailored to users’ needs through prompt engineering, we incorporated the expectations of both clinicians and patients into the LLM prompting process. Contents of the PFCNs Interview results revealed that patients expected the content of LLM-generated PFCNs to closely align with their clinicians’ verbal explanations. Clinicians also stressed the importance of generating document content based on their verbal consultations and medical records to address potential legal concerns. Both patients and clinicians expressed that any discrepancies between the PFCNs and the clinicians’ explanations during ED consultations could lead to patient confusion and increased communication burdens. To resolve these concerns, we designed the PFCNs to be directly based on the clinician’s clinical notes, which detail the patient’s condition, diagnosis, test results, and future treatment plans recorded in the clinical notes after consultations. Additionally, both patients and clinicians emphasized that the documents should be patient-friendly to ensure they are accessible even to individuals with low health literacy. To achieve thi, the documents should be written in plain language, replacing complex medical terms with simpler, easily understood explanations. Supplemental information should also be included to aid comprehension of medical terms when necessary. Additionally, to help patients easily identify key information (e.g., diagnoses, test names), the documents should incorporate headings, emphasis marks, and underlining. For instance, headings such as Diagnosis and Future Treatment Plan can organize the content, while critical information like diagnoses and test names can be highlighted in bold to draw attention. Appropriate Scenarios and Patient groups for Applying the PFCNs in ED Interviews revealed that long waiting times between consultations often left patients feeling uninformed and disconnected from their care process in the ER. Recognizing this issue, we identified three key post-consultation situations for delivering PFCNs: after the initial consultation, during follow-up consultations, and at discharge. Although the LLM-generated documents were designed to be patient-friendly and easy to read, both patients and clinicians expressed concerns about their applicability to patients with highly acute symptoms (e.g., dyspnea, acute heart attack), critical conditions (e.g., myocardial infarction, stroke) and low health literacy. Such patients might not be physically or mentally capable of reading and understanding the documents. Therefore, we selected patients with non-severe conditions who were better able to understand and benefit from the PFCNs. Concerns in Accuracy of the PFCNs Clinicians expressed concerns about the possibility of incorrect information being provided to patients through LLM-generated content. In particular, clinicians with prior experience using LLMs highlighted the risk of hallucinations , where inaccurate or fabricated information could lead to confusion in subsequent communication with patients. To address these concerns, we integrated specific strategies into prompt engineering to improve the accuracy of the generated content. This included context-based prompts that provide background information to enhance the AI's understanding of the task (e.g., " This document is written by a clinician and provided to patients in the emergency department. ") and error-handling prompts designed to anticipate and minimize potential inaccuracies (e.g., " Avoid including any information not explicitly documented in the clinical notes ") 33 . Additionally, to monitor how patients interpret and understand the information in the PFCNs, we implemented a feature allowing patients to submit questions to their clinicians. This feature not only helps clinicians address patient concerns but also serves as a safeguard to identify and rectify potential misunderstandings caused by inaccurate information. Development of the LLM-based PFCNs Prompt Engineering for Generating the PFCNs Based on the results of the preliminary study, we developed prompts to generate PFCNs using clinicians’ clinical notes after patient consultations. First, we selected GPT-4o (OpenAI, https://openai.com/) as the LLM, as it was the most up-to-date model available in July 2024 and capable of supporting the Korean language. Second, we added prompts instructing the model to transform clinicians’ clinical notes into PFCNs that patients could easily understand. To address the variability in how individual clinicians document clinical notes, we employed a zero-shot learning technique. Third, we incorporated three key considerations identified from the preliminary study into the prompts: (1) ensuring accuracy, (2) enhancing patient-friendliness, and (3) improving readability (Textbox 1). Finally, we ensured that the prompts reflected the specific content needed for the three main scenarios in which PFCNs would be provided in ER: initial consultation, follow-up consultation, and discharge consultation. For this, we added specific rules to the prompts for generating three types of documents. For example, prompts for initial and follow-up consultations included future treatment plans, while discharge consultation prompts emphasized post-discharge care instructions. The full prompts and detailed considerations are provided in Supplement Data 1. Textbox 1. Considerations for prompt development for the PFCNs 1. Ensuring accuracy: To minimize hallucinations, the generated content was strictly based on the clinician's notes, avoiding any information that was not explicitly documented. 2. Enhancing patient-friendliness: The PFCNs were designed to include greeting messages, convert medical jargon into plain language, and provide guidance on the document’s question function. 3. Improving readability: Key points were highlighted using bold or underlined text, and the content was structured under subheadings for better organization. LLM-based system for Generating and Delivering the PFCNs To facilitate the generation and delivery of PFCNs during the ED consultations process, we designed a LLM-based system embedded with the prompts. This system enables clinicians to input clinical notes, generate PFCNs, and send them to patients. Patients can be provided the documents and submit questions which clinicians can address during subsequent consultations (Figure 1). The system was developed using Next.js (Vercel, https://nextjs.org/) along with TypeScript, HTML, and CSS. On the clinician interface, clinicians can write a clinical note, generate and modify PFCNs, and send finalized documents to patients (Figure 2). On the patient interface, patients can read the PFCNs sent by their clinicians and send questions, which are displayed on the clinician interface. (Figure 3). Evaluation of LLM-generated PFCNs Phase 1: Clinician’s Evaluation of LLM-generated PFCNs We conducted an evaluation with 10 clinicians to assess the applicability of LLM-generated PFCNs for ED patients (Figure 3). First, an orientation session was held to explain the study’s purpose, experimental methods, and compensation plan. Next, the clinician participants conducted two role-play sessions with a simulated patient (researcher SIK) presenting with abdominal pain and low back pain, respectively (Table 1). In each scenario, clinicians performed three consultations (initial visit, follow-up, and discharge) and typed their consultation records in clinical notes for each consultation. Based on the clinical notes, PFCNs were generated and provided to the clinicians for evaluation. To observe changes in doctor-patient communication following document delivery, the simulated patient asked questions about the document’s contents during the consultation. Table 1. Simulated role-play scenarios for clinicians in ED consultations Scenario Chief complaint Descriptions #1 Abdomen pain Chief complaint: Right Lower abdomen pain Onset: before 6 hours ago Subjective symptoms: Right Lower abdomen pain, abdomen discomfort, dyspepsia Physical findings: Tenderness(+), Rebound tenderness (-) Previous medical history: n/a #2 Low back pain Chief complaint: low back pain Onset: 3 hours ago Subjective symptoms: Low back pain, Radiating leg pain, numbness Physical findings: straight leg raise test. right (+) Previous medical history: herniated intervertebral disk in L5-6, L6-7 Clinicians evaluated six PFCNs generated from their medical note data on three criteria: Perceived Accuracy, Completeness, and Understandability [Supplement Data 2]. Perceived Accuracy was assessed on a 6-point Likert scale (ranging from 1 to 6) to assess how accurately the document reflected the contents of the medical note (e.g. history taking, physical examination, diagnosis, treatment) 31 . Participants also explained the reasons for their accuracy ratings and commented on the document content. Completeness was evaluated by determining how comprehensively the medical note contents were reflected in the three types of documents (initial assessment, progress explanation, and discharge explanation). Participants were asked to note any missing information in the completeness evaluation form. Lastly, Understandability was measured using the Patient Education Materials Assessment Tool (PEMAT) 34 [Supplement Data 2]. PEMAT consists of two components: understandability and actionability. PEMAT can be used by both healthcare providers (e.g., physicians, nurses) and patients to evaluate the understandability and actionability of patient education materials. Since the PFCNs did not include actionable instructions for patients, the actionability component of PEMAT was excluded from this study. The PEMAT understandability section consists of 16 items, evaluated on a 2-point scale (Agree or Disagree). For items 6 and 13–16, if the criteria are not applicable to the document being evaluated, "N/A" can be selected. The evaluation score is calculated as the percentage of items marked as "Agree" out of the total applicable items (excluding those marked as "N/A"). According to PEMAT guidelines, a score of 70% or higher is generally considered indicative of adequate understandability. After completing the evaluation forms, we conducted individual interviews with clinicians to gather their feedback on LLM-generated PFCNs and their potential applications. The interviews covered four main areas: (1) the process of reviewing the documents during consultations, (2) the communication process based on the documents, (3) suggestions for improving the documents, and (4) potential strategies for implementing the documents in clinical practice. Participants were informed that the interviews would be recorded, anonymized, and analyzed. They were encouraged to freely share their thoughts on the use of the PFCNs and were assured that they could withdraw from the interview at any time. Phase 2: Patients’ Evaluation of LLM-generated PFCNs We conducted an evaluation with 20 patients to assess their perspectives on LLM-generated PFCNs (Figure 4). Patients with prior ED experience participated in simulated clinical scenarios. Each role-play session lasted approximately 30 minutes, during which patients engaged in three consultations (initial assessment, progress explanation, and discharge explanation) with the clinician (SIK). Patients role-played based on their recent ED experiences, presenting symptoms similar to their past cases. After each consultation, patients received PFCNs on a tablet PC and were instructed to review the document and ask questions about any unclear or confusing content during the subsequent consultation. Participants were informed in advance that the sessions would be recorded and consent was obtained. They were also advised to inform the researcher of any discomfort or concerns during the study and assured that they could withdraw at any time if discomfort persisted. Before and after the role-play sessions, patients were asked to evaluate their experience of participation in care. To assess patient participation, we adapted the Patient Participation Questionnaire (PPQ) 2 [Supplement Data 2]. The PPQ is a validated tool designed to assess the extent of patient participation in healthcare settings. The PPQ evaluates various aspects of patient engagement, focusing on how patients perceive their involvement in medical decision-making, communication with healthcare providers, and the overall care process. The PPQ consists of 17 items. Items 1 to 16 are evaluated on a 4-point scale, while the 17th item (Overall Participation) is assessed using a 2-point scale (Yes or No). In the study, patients completed the PPQ to evaluate their participation in previous ED experiences before the role-play and then evaluated their participation during the simulated consultations using the PFCNs. After reviewing the documents, participants evaluated their usefulness and understandability. To measure the perceived usefulness of the LLM-generated PFCNs, patients completed the Usefulness Scale for Patient Information Material (USE) 35 [Supplement Data 2]. The USE is designed to evaluate how well educational materials support patient education and informed decision-making, thereby empowering patients. The USE consists of 9 items, each rated on a scale from 0 to 10. The items are divided into three key domains: 1) Cognition (Items 1–3, Measures how patient education materials improve patients’ knowledge), 2) Emotion (Items 4–6, Assesses how the materials help patients cope emotionally with their condition), 3) Behavior (Items 7–9, Evaluates how the materials support patients in managing their illness and adopting appropriate health behaviors). In the patient group, the understandability of the PFCNs was assessed using only the understandability section of the PEMAT tool 34 . To further explore patients’ experiences with the PFCNs, individual interviews lasting approximately 30 minutes were conducted after the role-play sessions. The interviews focused on (1) the experience of reviewing and using the documents in the simulated consultation, (2) their experience of participation in ED care, and (3) suggestions for improving and implementing the documents. Participants were informed that the interviews would be recorded, anonymized, and analyzed. They were encouraged to share their thoughts freely and were assured that they could withdraw from the interview at any time. Analysis For quantitative analysis, the PPQ data collected during the study were analyzed using Jamovi, an open-source data analysis tool (ver. 2.5, The Jamovi Project, www.jamovi.org). Changes in PPQ scores before and after the role-play sessions were examined. Since the Shapiro-Wilks test indicated a non-normal distribution (p < 0.05), we used the Wilcoxon Signed-Rank Test to compare pre- and post-study PPQ scores. The results of the PEMAT and USE scales were summarized in Excel (Microsoft) as total scores and separately for each document type (initial visit, follow-up, discharge). For completeness, researchers manually reviewed the documents to identify included and omitted items. Perceived accuracy was analyzed by calculating the average scores assigned by clinicians for each document in Excel. We also categorized factors that lowered accuracy based on clinicians’ written feedback. To identify these factors, researchers coded the feedback through open coding and conducted a thematic analysis 36 . For qualitative analysis, interviews with both patients and clinicians were transcribed and coded by the researchers (SIK, JYP). Open coding and thematic analysis were conducted to identify themes related to the creation and delivery of the PFCNs, how they facilitated patient participation, and challenges encountered during their use 36 . The coding process was iterative and continued until the researcher’s reached consensus on the themes. Interviews and surveys were conducted and analyzed in Korean to ensure precise interpretation of participants’ original responses. Results Participants A total of 30 participants (10 clinicians and 20 patients) participated in the LLM-generated PFCNs evaluation study. The demographic information of the participants was included in Multimedia Appendix 3. Among the clinician participants, 9 (90%) were junior doctors currently in their residency training. The average age of the clinicians was 29.4 years, with an average work experience of 4.7 years. Five clinicians (50%) were from the Department of Emergency Medicine. The average age of the patient participants was 29.5 years, consisting of 5 men and 15 women. Patients had an average of 4.65 previous ED visits. The primary reasons for their most recent ED visits included trauma (7 cases, e.g., ankle pain from hiking), infections (6 cases, e.g., EBV infection, fever), gastrointestinal problems (5 cases, e.g., abdominal pain, diarrhea), and psychosomatic issues (e.g., dyspnea in stressful situations). Quality Evaluation of PFCNs General Information In this study, a total of 120 PFCNs (60 from clinicians and 60 from patients) were generated. The examples of the clinical notes written by clinicians and the PFCNs generated by LLM were included in Supplement Data 4. The average word count for the 120 clinical notes written by clinicians was 40.52 words, while the average word count for the 120 PFCNs was 231.31 words (Table 2 ). Table 2 Quality assessment results of the PFCNs Phase 1: clinician’s evaluation Phase 2: patient’s evaluation Total Category Initial visit (n = 20) Follow-up (n = 20) Discharge (n = 20) Total (n = 60) Initial visit (n = 20) Follow-up (n = 20) Discharge (n = 20) Total (n = 60) Total (n = 120) Word count Clinical note [Avg. (SD)] 43.8 (13.98) 24.8 (10.77) 29.6 (7.62) 32.73 (13.72) 73.85 (17.02) 31.65 (17.02) 39.4 (7.89) 48.3 (19.83) 40.52 (18.69) PFCN [Avg. (SD)] 261.5 (10.87) 225 (25.08) 210.5 (22.61) 232.33 (29.66) 264.75 (30.67) 217.6 (24.31) 208.5 (16,91) 230.28 (34.84) 231.31 (32.22) Understandability PEMAT Understandability score [%, Avg. (SD)] 86.7 (8.21) 86.3 (5.87) 88.8 (5.87) 87.2 (6.59) 86.7 (9.07) 86.3 (8.76) 88.8 (9.92) 87.2 (9.13) 87.2 (7.93) Completeness Number of omitted information 0 0 5 5 N/A N/A N/A N/A N/A Perceived accuracy Total [avg.(SD)] 5.35 (0.75) 5.55 (0.83) 5.45 (0.60) 5.45 (0.72) N/A N/A N/A N/A N/A Usefulness USE- Cognition [Avg.(SD)] N/A N/A N/A N/A 25.2 (3.53) 25.6 (7.62) 24.2 (7.13) 25.0 (5.14) N/A USE - Emotion [Avg.(SD)] N/A N/A N/A N/A 20.8 (7.81) 20.8 (7.63) 21.1 (7.35) 20.9 (7.47) N/A USE - Behavior [Avg.(SD)] N/A N/A N/A N/A 24.1 (5.20) 24.1 (6.79) 24. (7.42)1 24.1 (6.42) N/A USE - Total [Avg.(SD)] N/A N/A N/A N/A 70.0 (14.6) 70.4 (16.9) 69.3 (20.6) 69.9 (17.3) N/A Understandability The PEMAT understandability score evaluated by clinicians was 87.2% (> 70%), while the score assessed by patients was also 87.2% (> 70%) (Table 1 ). The PEMAT understandability score of > 70% observed in this study suggests that the PFCNs generated by the LLM-based system are understandable to patients. Both clinicians and patients rated the discharge PFCNs highest (clinicians: 88.8%, patients: 87.9%), but the follow-up PFCNs were rated lowest (clinicians: 86.3%, patients: 86.3%). The statement, “Medical terms in this document are used only to familiarize the patient with the diagnostic and treatment processes. When used, the meanings of the terms are defined,” received the lowest ratings from patient participants (applicable to 51 out of 60 documents) and the second-lowest ratings from clinicians (applicable to 50 out of 60 documents). The statement, “This document does not include information beyond its intended purpose,” received the lowest score from clinicians (applicable to 49 out of 60 documents) but was rated highly by most patients (applicable to 59 out of 60 documents). Completeness Among the 60 PFCNs evaluated by clinicians, 55 included all the required elements (Table 2 ). However, five discharge documents were missing specific content: three instances of missing “History and physical examination details” and two instances of missing “Expected diagnoses.” Perceived Accuracy The average perceived accuracy of the PFCNs was 5.45 out of 6 (6 points: 44 documents, 5 points: 20 documents, 4 points: 5 documents, 3 points: 1 document) (Table 2 ). The accuracy scores for the three types of documents were: initial consultation document 5.35, follow-up explanation document 5.55, and discharge document 5.45. Since the perceived accuracy scores for the three types of documents did not meet normality assumptions, a Friedman non-parametric repeated measures ANOVA was conducted, which showed no statistically significant difference between the scores ( p = 0.066, > 0.05). We analyzed the factors contributing to reduced accuracy in the PFCNs to identify issues affecting the quality and usability of the documents. Based on the perceived accuracy evaluation results and clinicians’ interview findings, we identified two primary factors: 1) Hallucinations and 2) Misalignments. Hallucinations refer to systematic errors in the content generation process, such as missing, inaccurate, or fabricated information does not present in the clinician’s original clinical notes 31 . These errors often arise from the LLMs’ tendency to infer or fabricate details beyond the provided input. On the other hand, we defined misalignments which are characterized by discrepancies between the clinician’s intent or reasoning and the content of the generated document, reflecting a failure to capture the subtleties of clinical judgment or context. Therefore, the key distinction lies in their origins: hallucinations stem from the LLMs’ content creation process, while misalignments result from the LLMs’ inability to accurately interpret and convey the clinician’s intent. Addressing both issues is essential for improving the accuracy, usability, and trustworthiness of LLM-generated documents in clinical settings 37 , 38 . To better understand the underlying challenges of LLM-generated PFCNs, we further categorized these two factors into detailed subtypes (Table 3 ). 1) Hallucinations included missing information (e.g., outpatient follow-up instructions documented in the clinical note but omitted from the PFCNs), inaccurate information (e.g., marking the location of the appendix on the left side), and the addition of the unmentioned information (e.g., abdominal ultrasound details not recorded by the clinician were included). 2) Misalignments involved discrepancies between the clinician’s own reasoning process and the document content. For example, while the clinician did not document the reasoning behind empirical antibiotic use in the clinical note, the PFCNs incorrectly stated that it was “to identify the source of infection,” whereas the clinician’s primary intent was “symptom relief and bacterial eradication.” Other cases included unintended disclosure of information (e.g., revealing the reason for a patient’s refusal of an MRI, which was documented for legal protection) and awkward phrasing in the document (e.g., using unusual patient addresses like “Mr. Tommy,” which the clinician would not typically use). In addition, clinicians anticipated more supplemental information in the PFCNs, covering topics they frequently and repeatedly discussed with patients during consultations, such as waiting times for tests and test preparation requirements. Table 3 Factors contributing to lower perceived accuracy of the LLM-generated PFCNs Factors Category Examples Hallucination Omission of information - Outpatient follow-up instruction recorded in the clinical note was not reflected in the PFCN - Expected diagnosis missing from the discharge PFCN. Generation of inaccurate information - The location of the appendix was incorrectly recorded as being on the left side. - The PFCN informed the patient that the medication for controlling abdominal pain intended for discharge was already administered in the ED. Addition of the unmentioned information - An abdominal ultrasound test was generated despite not being recorded by the clinician. - The follow-up explanatory note includes information indicating that the patient is eligible for discharge. Misalignment Lack of reflection of the clinician’s clinical reasoning process and routine practices - Although the reason for empirical antibiotic prescription was 'infection source identification' in the document, the clinician’s intent was symptom relief and bacterial eradication - The PFCN includes factors that may contribute to herniated intervertebral disc (HIVD) (e.g., severe impact from an accident); however, these seem unrelated to the information confirmed by the clinician during the patient’s consultation (e.g., a previous history of HIVD, desk job). Unwanted disclosure of information - The PFCN disclosed the reason why the patient refused an MRI, which was recorded by the clinician for legal protection. Awkward phrasing in the explanation - Inconsistent or unusual patient address forms (e.g., 'Mr. Kim Young-ho,' 'Patient Kim Young-ho') Anticipation to the supplement information on clician’s explanation - Waiting times for blood tests and imaging (CT, MRI) examinations. - Preparation requirements for imaging tests (e.g., removing metallic objects from clothing, remaining still during the procedure). Usefulness The results of the patients’ usefulness evaluation showed USE scores of 70.0 for the initial visit document, 70.4 for the follow-up document, and 69.3 for the discharge document (Table 1 ). Since the usefulness scores for the three document types did not meet normality assumptions, a Friedman non-parametric repeated measures ANOVA was conducted, which confirmed that the score differences among the initial, progress, and discharge documents were not statistically significant ( p > 0.05). The average scores for each subcategory of the usefulness evaluation were as follows: Cognition scored the highest at 8.32, followed by Behavior at 8.02, and Emotion at 6.96. ​​These results indicate that patients found the PFCNs particularly useful for acquiring information, followed by their usefulness in supporting communication during the consultation process. To investigate the relationship between usefulness and patient participation, we examined the correlation between USE scores and PPQ scores. Using the PPQ scores measured after the study, rather than the changes in PPQ scores before and after the study, is more appropriate because the post-study PPQ scores directly reflect the influence of the PFCNs on patient participation. The analysis revealed a Pearson correlation coefficient of 0.86 between the PPQ scores after the study and the usefulness scores. These results suggest that patients who perceived the PFCNs as more useful were more likely to actively participate in the consultation process. Patient Participation with the Use of PFCNs Changes in PPQ score before and after ED Consultations Role-Play The results showed that the patient participation score in the simulated consultation with PFCNs was 56.5, a statistically significant increase compared to the score of 40 from previous ED consultations ( p < 0.001) (Table 4 ). Since the subcategories Involvement, Information, and Communication did not meet normality assumptions, the Wilcoxon test was used, and all three showed significant score increases after the simulated consultations ( p < 0.001). The subcategory Relationship to Staff met the normality assumption and was analyzed with a student’s paired samples t-test, which also indicated a significant score increase ( p < 0.001). For the Overall Assessment of Involvement, a Wilcoxon test was conducted, and the score increase was again statistically significant ( p < 0.001). Table 4 PPQ scores of the patients before and after simulated role-play in ED consultations Category Before Role-play [Avg. (SD)] (n = 20) After Role-Play [Avg. (SD)] (n = 20) p -value PPQ score Involvement 9.3 (2.18) 13.35 (2.64) p < 0.001 Information 10.3 (1.89) 14.1(1.86) p < 0.001 Communication 7.35 (1.31) 10.3 (1.78) p < 0.001 Relationship to staff 11.9 (1.94) 16.8 (1.90) p < 0.001 Overall assessment of involvement 1.15 (0.37) 1.9 (0.31) p < 0.001 Total 40 (6.13) 56.6 (8.56) p < 0.001 Patient Perception of the LLM-generated PFCNs Through the patient interview results, we found that patients perceived PFCNs as supportive for: 1) preparing for the consultation, 2) supplementing the clinicians’ explanation, 3) providing emotional assurance, and 4) enhancing relationships with clinicians. First, the LLM-generated documents encouraged patients to participate more actively in their ED consultations by helping them prepare questions in advance based on the information of the documents (e.g. "Could you let me know if hospitalization will be necessary? Why is the pain medication administered only after the physical examination and imaging?" ) Most patients noted that the hectic ED environment often made it difficult to think of questions during consultations. However, the PFCNs provided them with time to reflect and formulate more specific questions, with some even revising their questions based on the information in the documents. “Now I understand why certain tests were done and what the results mean. This really helps reduce the information gap. I feel like I can prepare more refined questions for the doctor. For example, instead of asking, ‘Is this just a cold?’—which wastes time and might frustrate the doctor—I could ask, ‘Is it influenza, rhinovirus, or Epstein-Barr virus?’ The doctor would likely see that I’m engaged and want to participate in decision-making, which could lead to better communication.” (Patient 4) Second, the PFCNs supplemented the clinician's verbal explanations, helping patients better understand their condition and treatment progress. Especially in Korea’s busy tertiary hospitals 39 , these documents could be useful for acquiring information about expected diagnosis, test types, test results and discharge education. Patients also reported that during waiting times, they could read the document and look up unfamiliar terms (e.g. appendicitis, migraine) using online search engines to better understand their situation. I really liked how everything—my treatment plan and procedures—was clearly outlined. From what I remember, the ED is such a chaotic place where I’m not sure what to do next. Sometimes I’d even have to go ask the staff, ‘When is this going to be done?’ And being in pain makes it even harder to keep track of everything. But with everything organized in the document, I feel like I could’ve said, ‘Hey, I haven’t done this yet,’ more easily. (Patient 3) Third, patients reported that reviewing the PFCNs provided emotional reassurance. Revisiting the clinician’s explanations helped alleviate the anxiety they felt during the consultation. Additionally, the inclusion of empathetic comments, such as “Wishing you a speedy recovery,” was comforting. Unless it’s an extremely critical situation, most ED patients are conscious and aware. Seeing the explanations laid out in the document helps me feel reassured. I can clearly see my condition and the steps ahead, which makes me feel more at ease. (Patient 9) Fourth, patients reported that the PFCNs could improve their relationships with medical staff. By reducing unnecessary questions during consultations, patients perceived that the clinician could focus more on their specific needs and concerns. Clinicians also reported that receiving pre-submitted patient questions allowed them to better understand the patient’s specific concerns, level of comprehension, and health literacy, enabling them to adjust their communication style and explanations to better meet the patient’s needs. As a result, clinicians anticipated that the PFCNs could ease the burden of challenging conversations and enhance the overall quality of patient interactions. “I feel like this would make the doctor-patient relationship more positive because I wouldn’t have that unnecessary guilt for asking too many questions. From the doctor’s perspective, having pre-prepared questions would probably be much easier than answering impromptu ones. It could even improve workflow efficiency.” (Patient 2) Challenges in the Applying PFCNs to ED consultations Clinicians’ Challenges from Decreased Information Accuracy Clinicians highlighted that inaccurate or missing information in the PFCNs (e.g., test types, waiting times, or outpatient appointments) could lead to increased communication burdens and additional work. For instance, five clinicians pointed out that while an ultrasound test was not documented in the clinical notes for a patient with abdominal pain, the PFCNs included ultrasound results, which they deemed medically valid. However, they noted that this discrepancy could lead to patient complaints and increased workload. The ultrasound inclusion seemed like a system error. If there’s a mismatch between tests conducted and those mentioned, patients often file complaints, and it can become a big issue. (Clinician 9) Additionally, clinicians expressed concerns about potential legal liability when PFCNs included diagnoses or tests not explicitly mentioned in their clinical notes. The differential diagnoses I list might be used as evidence, and that could become problematic if communicated through the document... For example, in cases of back pain, I initially considered a fracture but didn’t record it. If it turns out to be a fracture, patients might ask why I didn’t document or address it earlier. (Clinician 6) Moreover, clinicians emphasized that providing inaccurate or incomplete information could lead to conflicts with patients and undermine trust in the doctor-patient relationship. If there’s a discrepancy between the clinician’s explanation and the text received, the patient might distrust one side. This could challenge the trust between the patient and the clinician, and rebuilding that trust could cause significant stress for the clinician. (Clinician 3) Differences in Perspective on the PFCNs Between Patients and Clinicians Although the LLM-generated documents aimed to bridge the information gap between patients and clinicians by supplementing clinician’s clinical notes, patients often found the information insufficient for understanding their symptoms and care process. While patients desired more comprehensive information, clinicians preferred to limit the content to minimize legal risks. “To me, the document felt like a simple summary of what was mentioned during the consultation. I think more detailed information could have been included. For example, during the long waiting times, I could’ve read about what a CT scan involves and how to prepare for it. Or there could’ve been more detailed post-discharge instructions, like dietary recommendations. That kind of additional guidance would’ve been really helpful, but the document only included what the doctor said, which felt like a missed opportunity. (Patient 8) ” What I’m most worried about is that this AI generates the document based on the clinical notes I’ve written. If something gets left out, that’s not a big deal. But if something is misrepresented or, worse, if even one piece of incorrect information gets included, it means the patient is getting wrong information. As a doctor, I’d have to take on the ethical responsibility and legal risks, and that feels way too heavy to handle. (Clinician 10) Additionally, clinicians evaluated the PFCNs based on the accuracy and alignment with their clinical intent, focusing on whether objective facts were accurately reflected. In contrast, patients primarily trusted the documents if they believed the content mirrored the clinician’s verbal explanation. This difference sometimes prevented patients from identifying errors in the documents (e.g., misdiagnoses in the initial assessment document). “Since the content matched exactly what the doctor told me, I just accepted it as is. I didn’t feel there was anything untrustworthy about it.” (Patient 1) In the document, it might mention general risk factors or the severity of symptoms on its own, but sometimes these things don’t align with the doctor’s original intent. Like, when you order a test, the reason for the test, or when you prescribe a treatment, the purpose of the treatment—things like what condition the patient actually has or what they need to be cautious about—might end up being reflected differently from what I intended. (Clinician 4) Discussion Promoting Patient Participation Through LLM-Generated PFCNs This study aimed to verify how LLM-generated documents could enhance patient participation in the ER setting. The key findings are as follows: (1) LLMs successfully generated PFCNs suitable for patient comprehension, as demonstrated by a PEMAT understandability score of 87.2% (> 70%); (2) providing LLM-generated PFCNs significantly increased patients’ PPQ scores (P < 0.05); and (3) the documents facilitated patient participation during ER consultations by expanding opportunities for communication through document review and question submission, improving understanding of clinical information, alleviating anxiety through clear explanations, and strengthening relationships with clinicians. Our findings demonstrate that LLM technology can be effectively utilized to generate PFCNs based on clinicians’ clinical notes for ED patients. Since providing PFCNs during ER consultations is a new concept, unlike previous studies 31 , 40 , direct comparisons between clinician-written and LLM-generated documents were not considered in the study. However, when comparing the PEMAT understandability score of 81% reported by Zaretsky et al. (2024) for inpatient discharge summaries generated using GPT-4, the 87.2% score from our study demonstrates that the understandability of the PFCNs was well ensured 31 . In terms of accuracy, Zaretsky et al. reported that 54% (54 out of 100) of their documents received a top rating, whereas our study achieved 73.3% (44 out of 60) 31 . Therefore, although we did not use other LLMs (e.g. Bard, Gemini), our results supported the previous study results that LLMs is a promising tool to generate clinical documents with a quality level comparable to clinicians. Despite addressing concerns about hallucinations in the preliminary study by incorporating relevant adjustments into the prompts, issues with the accuracy of LLM-generated documents due to hallucinations were still observed in the study results. Furthermore, our study identified that misalignments, where clinicians’ intentions were not accurately reflected in the PFCNs, were a significant factor lowering perceived accuracy. This finding highlights the importance of not only ensuring the inclusion of accurate and well-documented clinical information but also reflecting the clinician’s intent to improve the accuracy and reliability of PFCNs provided to patients. Our study results demonstrate that LLM-generated PFCNs can facilitate patient participation in the ED consultation process. Through the USE scores, we observed that users found the documents particularly helpful for acquiring information and supporting communication. Moreover, the findings indicated that clinicians who were more inclined to utilize the documents in their practice were also more likely to engage patients in the care process. These results align with prior studies demonstrating the benefits of using LLMs to deliver patient-friendly information, improving patients’ understanding of medical content and their participation in decision-making 11 , 25 , 41 . Expanding on this, our study is the first to confirm that AI-generated documents in the ED setting can enhance patient communication and strengthen the patient-physician relationship, providing patients with opportunities to actively participate in their care. Implications of LLM Development for ED Settings The study highlights the potential of LLM-generated explanations to support ED consultation and workflow, particularly in busy and fast-paced environments such as tertiary hospital ED. By supplementing verbal explanations with detailed written summaries, clinicians can save time and ensure consistency in patient education. However, the need for clinicians to review and correct inaccuracies in LLM outputs may initially increase workload. Therefore, considering the challenges identified in this study and the heavy workload on emergency department workflows, we proposed implications for development of LLMs for generating and implementing PFCNs in ED settings. Collaboration Between Clinicians and LLM-based Systems to Enhance Accuracy and Alignment We identified critical challenges related to the accuracy and alignment of LLM-generated content. Hallucinations (e.g., generation of incorrect or unmentioned information) and misalignments (e.g., failure to reflect the physician's intent or clinical reasoning) emerged as potential critical barriers. These issues not only risk patient confusion but may also strain patient-clinician relationships and increase the clinician's workload. For instance, discrepancies between clinical notes and generated explanations led to concerns about medico-legal accountability and the potential for patient complaints. To address these challenges, collaboration between clinician and AI systems during the document generation process could minimize errors and ensure explanations accurately reflect clinicians’ intentions. However, considering the heavy workload of clinicians, it is essential to ensure that a robust validation process for LLM outputs does not become an excessive burden. For instance, in MedKnowts, the system assists ED clinicians to document patients’ clinical information in Electronic Health Records (EHRs) rapidly by providing autocomplete suggestions for typed medical terms (e.g., WBC, WBCCAST) or displaying value trends for recently recorded items (e.g., CK, CRP) 42 . While this prior work introduced features to support clinicians in the documentation process, we further propose incorporating editing features, such as highlighting or deletion, to better capture clinicians' intention and reflect them in LLM-generated documents. To achieve this, future research should explore how various factors—such as the context of documentation (e.g., initial visit, follow-up, discharge), the clinician’s level of experience, and the patient’s perceived health literacy—are integrated into the editing process. For example, during initial visits, a less experienced junior clinician might deliberately remove differential diagnoses from the generated document to minimize potential patient complaints about their medical note. Balancing Information Sensitivity and Comprehensiveness Our findings also underscore the importance of balancing information sensitivity with comprehensiveness. We identified differing perspectives on the desired depth of information between patients and clinicians, which align with findings from previous studies 18 , 43 . To meet a balance between these preferences, it is required to consider both practical and ethical factors carefully in prompting. One approach could be offering layered information, where a core explanation provides essential details (e.g. reason for inferring the diagnosis, necessity of the test), and additional resources (e.g. costs of the test, general information about the disease) are available for patients seeking deeper understanding using external links such as hyperlink connected to the hospitals’ clinical education page. This modular design could accommodate varying levels of patient health literacy and interest. In addition, it is crucial to ensure that patients are fully aware of the potential inaccuracies and errors in AI-generated information. To address this, the document should clearly indicate the possibility of errors, and healthcare providers should explicitly notify patients that the content was generated by AI. Understanding Patient Interaction with PFCNs The findings of this study reveal that patients actively engaged with the PFCNs to enhance their understanding of medical information and participate more effectively in their care. By gaining deeper insights into patient interaction with PFCNs, healthcare providers can bridge communication gaps, enhance patient understanding, and foster more effective and collaborative care delivery. Therefore, it is essential not only to provide these documents but also to implement strategies that maximize their utility for patients. Our findings highlight the diverse ways in which patients interact with PFCNs, providing valuable insights for future research. For example, in this study, patients utilized the questioning feature to seek additional information (e.g., explanation of waiting times or the necessity of certain tests), clarify difficult terms (e.g., "What is an autoimmune disease?"), or correct inaccuracies in the documents (e.g., amending the number of emergency department visits). Likewise, incorporating features like question-and-answer functionalities or highlighting unclear sections in the documents could help capture how patients engage with and interpret the content. Additionally, consideration should be given to how this feedback can be effectively delivered to clinicians to improve the accuracy and relevance of the explanations provided. Limitations Although this study aimed to assess whether LLM-generated PFCNs could be applied in clinical settings to support patient participation, the study was conducted in a lab setting due to safety and ethical concerns regarding the application of AI technology to patients. To address these limitations, we not only recruited patients with prior ED experience but also employed simulated role-play consultations to reflect patients’ recent ED visits. However, our in-lab experiments may not fully capture the dynamics and atmosphere of an actual emergency department. Thus, future research is required to validate the efficacy of the LLM-generated PFCNs in real-world ED settings. Another limitation is the lack of generalizability due to the study results. We only generated 120 documents using LLM during the evaluation study with 20 patients and 10 clinicians in this study. In addition, most participants in this study were in their 20s and 30s, had high levels of digital literacy, and were familiar with using LLMs, which minimized any discomfort in receiving LLM-generated documents. In contrast, real ED environments involve patients with a wide range of symptoms, low health literacy or older ages. Therefore, further research is needed to involve a more diverse range of patients. In addition, while prominent LLMs such as Bard (Google AI) and Gemini (Microsoft) exist, we did not explore how different LLMs generate PFCNs. Given the rapid advancement of LLM technology, future research should consider selecting a variety of LLMs to identify those most suitable for generating PFCNs in ED settings. Lastly, validation of survey instruments is another limitation. Given the novelty of using generative AI with medical records, there is a lack of validated tools to measure the accuracy and completeness of these documents. Furthermore, due to the absence of appropriate readability assessment tools for Korean documents, we could not conduct readability tests, such as the Flesch–Kincaid readability tests [28,29]. Although tools like the PPQ (Patient Participation Questionnaire) are being developed to evaluate patient engagement, there remains no consistently used definition of patient-centered care (PCC) or participation in the ED [1]. Therefore, future research should aim to develop and validate instruments that are tailored to assessing LLM-generated documents and patient participation, considering the specific research settings and hospital environments. Conclusions This study aimed to determine whether PFCNs based on clinicians’ clinical notes could facilitate patient participation during emergency department care. To achieve this, we conducted interviews with patients and clinicians with ED experience to explore their perspectives on patient participation and the provision of PFCNs during care. Based on the interview findings, we designed LLM prompts and developed the content and application strategies for the PFCNs. Subsequently, patients and clinicians participated in simulated ED scenarios to evaluate the quality of LLM-generated documents and their impact on patient participation. Follow-up interviews were conducted to gather feedback on the experience of using the documents and suggestions for improvement and implementation. The results of this study demonstrate that LLM-generated PFCNs can enhance patient communication, decision-making, and relationship-building with healthcare providers, thereby promoting patient participation in emergency care. Additionally, this study identified potential challenges in applying LLM-generated documents in real ED settings and proposed considerations for their implementation. In the future, the findings of this study may not only contribute to improving the stressful workflow of emergency departments but also support the realization of patient-centered care by leveraging LLM-generated PFCNs to foster patient participation. Although this study aimed to assess whether LLM-generated PFCNs could be applied in clinical settings to support patient participation, the study was conducted in a lab setting due to safety and ethical concerns regarding the application of AI technology to patients. To address these limitations, we not only recruited patients with prior ED experience but also employed simulated role-play consultations to reflect patients’ recent ED visits. However, our in-lab experiments may not fully capture the dynamics and atmosphere of an actual emergency department. Thus, future research is required to validate the efficacy of the LLM-generated PFCNs in real-world ER settings. Another limitation is the lack of generalizability due to the study results. We only generated 120 documents using LLM during the evaluation study with 20 patients and 10 clinicians in this study. In addition, most participants in this study were in their 20s and 30s, had high levels of digital literacy, and were familiar with using LLMs, which minimized any discomfort in receiving LLM-generated documents. In contrast, real ED environments involve patients with a wide range of symptoms, low health literacy or older ages. Therefore, further research is needed to involve a more diverse range of patients. In addition, while prominent LLMs such as Copilot(Microsoft, copilot.microsoft.com) and Gemini (Google, gemini.google.com) exist, we did not explore how different LLMs generate PFCNs. Given the rapid advancement of LLM technology, future research should consider selecting a variety of LLMs to identify those most suitable for generating PFCNs in ED settings. Lastly, validation of survey instruments is another limitation. Given the novelty of using generative AI with medical records, there is a lack of validated tools to measure the accuracy and completeness of these documents. Furthermore, due to the absence of appropriate readability assessment tools for Korean documents, we could not conduct readability tests, such as the Flesch–Kincaid readability tests. Although tools like the PPQ are being developed to evaluate patient engagement, there remains no consistently used definition of patient-centered care (PCC) or participation in the ED 1 . Therefore, future research should aim to develop and validate instruments that are tailored to assessing LLM-generated documents and patient participation, taking into account the specific research settings and hospital environments. Abbreviations PCC Patient-Centered Care ED Emergency Department LLM Large Language Model EMR Electronic Medical Record AI Artificial Intelligence PFCNs patient-friendly clinical notes IRB Institutional Review Board PEMAT Patient Education Materials Assessment Tool PPQ Patient Participation Questionnaire USE Usefulness Scale for Patient Information Material HIVD Herniated intervertebral disc EHRs Electronic Health Records Declarations Conflicts of Interest None declared Author Contribution SIK and JP conducted the studies, analyzed the data, and contributed to writing the manuscript. TWK and WS designed the interviews and user studies, as well as provided feedback on the manuscript. TRK and WCC reviewed the study goal, advised how to properly implement LLMs to generate patient friendly clinical note in ED settings and supported participant recruitment at Samsung Medical Center. HH, as the corresponding author, supervised the overall study implementation and analysis processes. All authors reviewed and approved the final manuscript. Acknowledgements This study received no funding. Data Availability The data sets generated during this study are available from the corresponding author upon reasonable request References Walsh, A. et al. Patient-centered care in the emergency department: a systematic review and meta-ethnographic synthesis. Int. J. Emerg. Med. 15, 36 (2022). Berg et al. Questionnaire measuring patient participation in health care: Scale development and psychometric evaluation. European journal of cardiovascular nursing 19, 600–608 (2020). Wang, L. et al. The emergency patient’s participation in medical decision-making. J. Clin. Nurs. 25, 2550–2558 (2016). Frank, C., Fridlund, B., Baigi, A. & Asp, M. Patient participation in the emergency department: an evaluation using a specific instrument to measure patient participation (PPED). J. Adv. Nurs. 67, 728–735 (2011). Aung, E., Donald, M., Williams, G. M., Coll, J. R. & Doi, S. A. R. Joint influence of patient-assessed chronic illness care and patient activation on glycaemic control in type 2 diabetes. Int. J. Qual. Heal. Care 27, 117–124 (2015). Barello, S., Graffigna, G. & Vegni, E. Patient Engagement as an Emerging Challenge for Healthcare Services: Mapping the Literature. Nurs. Res. Pr. 2012, 905934 (2012). Sharma, A. E. et al. Patient Engagement In Health Care Safety: An Overview Of Mixed-Quality Evidence. Heal. Aff. 37, 1813–1820 (2018). Gillani, S. M. R., Nevill, A. M. & Singh, B. M. A randomised controlled trail in diabetes demonstrating the positive impact of a patient activation strategy on diabetes processes and HbA1c: The WICKED project. Br. J. Diabetes 17, 58–63 (2017). Sacristán, J. A. Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients. BMC Méd. Inform. Decis. Mak. 13, 6 (2013). Rogers, S. C. et al. CARES: Improving the Care and Disposition of Psychiatric Patients in the Pediatric Emergency Department. Pediatr. Emerg. Care 31, 173–177 (2015). Marzban, S., Najafi, M., Agolli, A. & Ashrafi, E. Impact of Patient Engagement on Healthcare Quality: A Scoping Review. J. Patient Exp. 9, 23743735221125440 (2022). Esmaeilzadeh, P., Dharanikota, S. & Mirzaei, T. The role of patient engagement in patient-centric health information exchange (HIE) initiatives: an empirical study in the United States. Inf. Technol. People 37, 521–552 (2024). Xiao, Z., Li, T. W., Karahalios, K. & Sundaram, H. Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot. arXiv (2023) doi: 10.48550/arxiv.2302.00832 . Yu, Q. J. & Pun, J. Promoting Patient Engagement in Medical Informed Consent – A Qualitative Study of Chinese Doctors’ Communication Strategies. Heal. Commun. 38, 71–79 (2023). Marty, H., Bogenstätter, Y., Franc, G., Tschan, F. & Zimmermann, H. How well informed are patients when leaving the emergency department? comparing information provided and information retained. Emerg. Med. J. 30, 53 (2013). Hess, E. P. et al. Shared decision making in patients with low risk chest pain: prospective randomized pragmatic trial. BMJ 355, i6165 (2016). Lateef, F. Patient expectations and the paradigm shift of care in emergency medicine. J. Emergencies, Trauma Shock 4, 163–167 (2011). Park, S. Y. & Chen, Y. Design Opportunities for Supporting Patient Information Needs during an Emergency Visit. Proc. 10th EAI Int. Conf. Pervasive Comput. Technol. Healthc. (2016) doi: 10.4108/eai.16-5-2016.2263339 . Tan, D. et al. Characterizing patient-friendly “micro-explanations”of medical events. Proc. SIGCHI Conf. Hum. Factors Comput. Syst. 2011, 29–32 (2011). Park, S. Y., Chen, Y. & Raj, S. Beyond Health Literacy. Proc. 2017 ACM Conf. Comput. Support. Cooperative Work Soc. Comput. 2179–2192 (2017) doi: 10.1145/2998181.2998357 . Johnson, C., Richwine, C. & Patel, V. Individuals’ Access and Use of Patient Portals and Smartphone Health Apps, 2020. ASTP Health IT Data Brief [Internet] (2020). Mannhardt, N. et al. Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study. arXiv (2024) doi: 10.48550/arxiv.2401.09637 . Walker, J. et al. OpenNotes After 7 Years: Patient Experiences With Ongoing Access to Their Clinicians’ Outpatient Visit Notes. J. Méd. Internet Res. 21, e13876 (2019). Moramarco, F. et al. Towards more patient friendly clinical notes through language models and ontologies. AMIA Annu. Symp. Proc. AMIA Symp. 2021, 881–890 (2021). Bala, S., Keniston, A. & Burden, M. Patient Perception of Plain-Language Medical Notes Generated Using Artificial Intelligence Software: Pilot Mixed-Methods Study. JMIR Form. Res. 4, e16670 (2020). Seo, W., Yang, C. & Kim, Y.-H. ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Events. Proc. CHI Conf. Hum. Factors Comput. Syst. 1–20 (2024) doi: 10.1145/3613904.3642152 . Kim, T., Shin, D., Kim, Y.-H. & Hong, H. DiaryMate: Understanding User Perceptions and Experience in Human-AI Collaboration for Personal Journaling. Proc. CHI Conf. Hum. Factors Comput. Syst. 1–15 (2024) doi: 10.1145/3613904.3642693 . Ding, H. et al. Evaluation and practical application of prompt-driven ChatGPTs for EMR generation. npj Digit. Med. 8, 77 (2025). Schuemie, M. J. et al. Standardized patient profile review using large language models for case adjudication in observational research. npj Digit. Med. 8, 18 (2025). Anibal, J. T., Huth, H. B., Gunkel, J., Gregurick, S. K. & Wood, B. J. Simulated misuse of large language models and clinical credit systems. npj Digit. Med. 7, 317 (2024). Zaretsky, J. et al. Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format. JAMA Netw. Open 7, e240357 (2024). Decker, H. et al. Large Language Model – Based Chatbot vs Surgeon-Generated Informed Consent Documentation for Common Procedures. JAMA Netw. Open 6, e2336997 (2023). Kulkarni, N. D. Crafting Effective Prompts: Enhancing AI Performance through Structured Input Design. J. RECENT TRENDS Comput. Sci. Eng. 12, 1–10 (2024). Shoemaker, S. J., Wolf, M. S. & Brach, C. Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information. Patient Educ. Couns. 96, 395–403 (2014). Hölzel, L. P. et al. Usefulness scale for patient information material (USE) - development and psychometric properties. BMC Méd. Inform. Decis. Mak. 15, 34 (2015). Braun, V. & Clarke, V. Using thematic analysis in psychology. Qualitative research in psychology 3, 77–101 (2006). Seo, J. et al. Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study. J. Méd. Internet Res. 26, e58329 (2024). Jung, H. et al. Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients. arXiv (2024) doi: 10.48550/arxiv.2404.05144 . Chang, H. et al. Emergency department crowding: a national data report. Clin. Exp. Emerg. Med. 11, 331–334 (2024). Tung, J. Y. M. et al. Comparison of the Quality of Discharge Letters Written by Large Language Models and Junior Clinicians: Single-Blinded Study. J. Méd. Internet Res. 26, e57721 (2024). Probst, M. A., Tschatscher, C. F., Lohse, C. M., Bellolio, M. F. & Hess, E. P. Factors Associated With Patient Involvement in Emergency Care Decisions: A Secondary Analysis of the Chest Pain Choice Multicenter Randomized Trial. Acad. Emerg. Med. 25, 1107–1117 (2018). Murray, L. et al. MedKnowts: Unified Documentation and Information Retrieval for Electronic Health Records. 34th Annu. ACM Symp. User Interface Softw. Technol. 1169–1183 (2021) doi: 10.1145/3472749.3474814 . Fritz, Z., Schlindwein, A. & Slowther, A.-M. Patient engagement or information overload: patient and physician views on sharing the medical record in the acute setting. Clin. Med. 19, 386–391 (2019). Additional Declarations No competing interests reported. Supplementary Files SupplementData.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5958826","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":411805856,"identity":"b1c3f9de-d51d-4240-b5b6-6827a88d44ac","order_by":0,"name":"Sung-In Kim","email":"","orcid":"","institution":"Seoul National University","correspondingAuthor":false,"prefix":"","firstName":"Sung-In","middleName":"","lastName":"Kim","suffix":""},{"id":411805857,"identity":"f8283761-b4e6-4eaa-a3b0-4ea1a07b93b8","order_by":1,"name":"Joonyoung Park","email":"","orcid":"","institution":"Korea Advanced Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Joonyoung","middleName":"","lastName":"Park","suffix":""},{"id":411805858,"identity":"a7eb1fd2-99db-4c2e-8295-19e9b4a1339c","order_by":2,"name":"Taewan Kim","email":"","orcid":"","institution":"Samsung (South Korea)","correspondingAuthor":false,"prefix":"","firstName":"Taewan","middleName":"","lastName":"Kim","suffix":""},{"id":411805859,"identity":"b88d110d-92fe-4315-9a4c-7d4dc2e11c23","order_by":3,"name":"Woosuk Seo","email":"","orcid":"","institution":"University of Michigan–Ann Arbor","correspondingAuthor":false,"prefix":"","firstName":"Woosuk","middleName":"","lastName":"Seo","suffix":""},{"id":411805860,"identity":"962aed72-8ee8-4744-9a03-02e9e5f766b1","order_by":4,"name":"Taerim Kim","email":"","orcid":"","institution":"Samsung Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Taerim","middleName":"","lastName":"Kim","suffix":""},{"id":411805861,"identity":"067fa4e8-500e-45e2-92c9-b026c9d50e48","order_by":5,"name":"Won Chul Cha","email":"","orcid":"","institution":"Samsung Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Won","middleName":"Chul","lastName":"Cha","suffix":""},{"id":411805862,"identity":"96da0a9e-d6f8-41ce-9e61-c7c7da526280","order_by":6,"name":"Hwajung Hong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIiWNgGAWjYBACAyiWk2BgbIAI8RCpxRis5QCxWkAgcQaIJEqLuUTyg2LeHbXpM9sPN3/+wGAnz8Bz9gFeLZYz0gyMec8cz53Nk9gmcYAh2bCBt90ArxaD2wlALW3HcucxJLYBHcacwMDPRsAvt9M/gLSky/E/bP5wgKGeGC05IFtqEqQlEhuADjucwMDbRkDL/TcFhnPbDhjOnPGwTeKMwXHDNp5jBLScOb7N4G1bnbzE+fTHHyoqquX5edLwawECNmAAHYaZAOQS1MDAwPyAgaGOCHWjYBSMglEwYgEA9v5D6kpWgpYAAAAASUVORK5CYII=","orcid":"","institution":"Korea Advanced Institute of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Hwajung","middleName":"","lastName":"Hong","suffix":""}],"badges":[],"createdAt":"2025-02-04 14:23:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5958826/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5958826/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":76083185,"identity":"36a9cb43-46f3-4da0-8641-6966b2d05b1a","added_by":"auto","created_at":"2025-02-12 07:03:09","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":119864,"visible":true,"origin":"","legend":"\u003cp\u003eThe process of generating and utilizing PFCNs: (1) The clinician writes clinical notes after the patient consultation. After that, the clinician adds these clinical notes to the LLM-based system. (2) The system processes the clinical notes into PFCNs, along with the system prompts developed by the researchers. (3) The PFCNs are delivered to the patient and continue to be used in subsequent ED experiences.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/0699eaf0d08987db1e022db0.png"},{"id":76083183,"identity":"6d92fea0-4014-40c1-a69e-89976438a7dc","added_by":"auto","created_at":"2025-02-12 07:03:08","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":637863,"visible":true,"origin":"","legend":"\u003cp\u003eClinician interface for generating PFCNs: (1) List of current patients in the ED. (2) Personal information of the selected patient from the list. (3) Text area for entering clinician's clinical note. (4) Buttons for selecting one of three consultation processes, which determines the type of PFCN to be generated. (5) Button to generate PFCNs from the clinical note. (6) Panel for displaying generated PFCNs. (7) Button to modify generated PFCNs. (8) Clicking the “Send to patient” button delivers the PFCNs to the patients’ interface. (9) Patients can view the same content in Figure 3-(2). (10) Panel for displaying questions received from the patients.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/7fdcaad4ba24ef10c0af178d.png"},{"id":76083188,"identity":"5f8c73b7-12d3-451d-989e-26b920848c82","added_by":"auto","created_at":"2025-02-12 07:03:11","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":545897,"visible":true,"origin":"","legend":"\u003cp\u003ePatient interface for reading PFCN and sending questions: (1) Panel displaying the patient’s personal information. (2) Panel showing the PFCN sent by the clinician. (3) Panel for sending questions to the clinician. After writing questions, the patient clicks the \"Send\" button, and the questions appear in the clinician’s interface (Figure 2-(10)).\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/62a1bb43b1c3db96782e4351.png"},{"id":76083178,"identity":"8fb46c22-c73b-4030-8861-0581ca62c570","added_by":"auto","created_at":"2025-02-12 07:03:07","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":318085,"visible":true,"origin":"","legend":"\u003cp\u003eEvaluation process of LLM-generated PFCNs of clinicians and patients\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/fe88f2b30f36b1e9ee7b67db.png"},{"id":79506290,"identity":"e1a17db3-fe32-4f98-b04f-c4b581a8ee4c","added_by":"auto","created_at":"2025-03-30 02:16:24","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2669117,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/00d7b4b8-1cc5-4a0e-b87e-1facbe695d72.pdf"},{"id":76083184,"identity":"8baa891b-b461-423b-b278-3637e921b06b","added_by":"auto","created_at":"2025-02-12 07:03:09","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1642475,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementData.docx","url":"https://assets-eu.researchsquare.com/files/rs-5958826/v1/a75d948860a62121c042a32a.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhancing Patient Participation in Emergency Department through Patient-Friendly Clinical Notes Generated by Large Language Models","fulltext":[{"header":"Introduction","content":"\u003ch3\u003eBackground\u003c/h3\u003e\n\u003cp\u003ePatient-centered care (PCC) emphasizes the partnership between clinicians and patients, focusing on integrating patients\u0026apos; health needs and personal preferences into the medical decision-making process \u003csup\u003e1\u003c/sup\u003e. Central to PCC is the promotion of patient autonomy, which requires active patient participation in various aspects of care, such as shared decision-making and communication with healthcare providers \u003csup\u003e2\u0026ndash;4\u003c/sup\u003e. Active patient participation has been shown to enhance patients\u0026apos; understanding and management of their symptoms \u003csup\u003e5\u003c/sup\u003e, increase patient satisfaction\u003csup\u003e6\u003c/sup\u003e, and alleviate emotional burdens such as anxiety and depression \u003csup\u003e7,8\u003c/sup\u003e. Moreover, patient participation has demonstrated clinical benefits, including improved health outcomes (e.g., better blood pressure and glucose control)\u003csup\u003e9\u003c/sup\u003e, reduced hospital visit durations \u003csup\u003e10\u003c/sup\u003e, and enhanced cost-effectiveness \u003csup\u003e11\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eFor patients to participate effectively in their care, it is crucial to provide them with sufficient and accessible clinical information (e.g. expected diagnosis, treatment options, examination and test results) \u003csup\u003e12\u003c/sup\u003e. Several research studies have focused on strategies for effectively sharing information during medical processes, such as providing interactive Chatbot-based informed consent\u003csup\u003e13\u003c/sup\u003e, identifying doctors\u0026rsquo; communication strategies \u003csup\u003e14\u003c/sup\u003e and providing discharge education\u003csup\u003e15\u003c/sup\u003e. For instance, Hess et al. (2016) demonstrated that providing low-risk acute coronary syndrome (ACS) patients with risk information increased their understanding, encouraged engagement, and safely reduced unnecessary admissions for cardiac testing \u003csup\u003e14,16\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eHowever, in the emergency department (ED) setting, where the environment is fast-paced and highly stressful for both patients and healthcare providers, there are challenges to facilitating patient participation \u0026nbsp;\u003csup\u003e1,17,18\u003c/sup\u003e. First, the information gap between clinicians and patients often leads to communication breakdowns, making it difficult for patients to receive adequate information. Clinicians tend to focus on obtaining patient data and delivering only essential information, which can decrease patient satisfaction \u003csup\u003e15,19\u003c/sup\u003e. Second, differences in communication styles between clinicians and patients can create further barriers. According to Park et al., clinicians often use a data-driven communication style aimed at information gathering, while patients prefer a narrative-driven approach to discuss their concerns\u0026nbsp;\u003csup\u003e20\u003c/sup\u003e. Although data-focused communication may efficiently identify medical issues, it often leads to decreased patient empathy and increased psychological distress.\u003cbr\u003e\u0026nbsp;\u003cbr\u003eTo address these challenges, sharing clinical notes with patients could offer a valuable opportunity to bridge the information gap between patients and clinicians \u003csup\u003e21\u003c/sup\u003e. Clinical notes are comprehensive records that capture essential patient data, diagnostic reasoning, and treatment plans of the clinicians during patients\u0026rsquo; consultation . However, complex medical concepts and jargon within clinical notes hinder patient comprehension and may cause emotional burden \u003csup\u003e22\u003c/sup\u003e. By making clinical notes more accessible, patients can better understand their medical conditions, engage in informed discussions with their providers, and actively participate in their care \u003csup\u003e23,24\u003c/sup\u003e. For example, Bala et al. reported that patients viewed patient-friendly format of clinical notes as useful for understanding their medical care, improving patient-physician relationships, and supporting self-management \u003csup\u003e25\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eRecently, there is growing interest in leveraging Large language models (LLMs) to make patient-friendly documents to bridge the information gap between clinicians and patients. Large language models (LLMs) are machine learning-based AI models designed to understand and analyze text \u003csup\u003e26\u003c/sup\u003e. These models have advanced significantly, enabling them to generate highly coherent and contextually appropriate responses based on user prompts \u003csup\u003e27\u003c/sup\u003e. \u0026nbsp;As research on the application of LLM technology in clinical practice (e.g. generating electronic medical record (EMR), patient case adjudication) continues to expand\u003csup\u003e28\u0026ndash;30\u003c/sup\u003e, recent studies have explored the potential of LLMs to generate patient-friendly documents, such as discharge summaries and simplified clinical notes \u003csup\u003e25,31,32\u003c/sup\u003e. \u0026nbsp;Previous related studies showed that LLM-generated documents can be used to bridge the information gap between patients and clinicians by providing clear, tailored explanations that enhance patient understanding and foster better communication between patients and clinicians. However, most of these studies have not specifically addressed the unique challenges of high-pressure environments like the ED, where time constraints and communication barriers are more pronounced. Therefore, this research aims to fill this gap by evaluating the impact of LLM-generated documents on improving patient participation in the ED setting, where effective communication and participation is particularly critical yet difficult to achieve.\u003c/p\u003e\n\u003ch3\u003eObjective\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eThis study aims to evaluate whether LLM-generated PFCNs can enhance patient participation in ED consultations. As a preliminary study, we conducted individual interviews with clinicians and patients to determine the appropriate content, format, and applicable scenarios for LLM-generated PFCNs in the ED setting. Based on the findings from these interviews, we developed LLM-generated PFCNs based on clinicians\u0026rsquo; clinical notes during ED consultations. Finally, we conducted a simulated ED role-play study involving both patients and clinicians, during which the LLM-generated PFCNs were provided. Participants completed quality assessment scales for the documents and patient participation questionnaires, followed by post-interviews to assess the documents\u0026apos; efficacy on patient engagement in the consultation process.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch3\u003eEthical Considerations \u0026amp; Recruitments\u003c/h3\u003e\n\u003cp\u003eThis study protocol and participant recruitment plan were approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB No. 2024-11-023-001). The inclusion criteria for clinician participants were: 1) adults aged 18 or older, 2) those with experience in ED care, 3) individuals capable of effective communication, and 4) those who agreed to the purpose and content of the study. The inclusion criteria for patient participants were: 1) adults aged 18 or older, 2) individuals with personal experience receiving ED care, 3) those capable of effective communication, and 4) those who agreed to the purpose and content of the study.\u003c/p\u003e\n\u003cp\u003eWe aimed to recruit 20 physician participants and 30 patient participants. Recruitment was conducted by posting announcements on the offline bulletin boards of the researchers\u0026apos; affiliated institution (Samsung Medical Center) and on an online community platform Everytime (Vinu Labs Inc. https://everytime.kr/) dedicated to Seoul National University students. Recruited participants were informed of the study\u0026rsquo;s purpose and methods and were asked to provide written consent. Participants were also informed that personal information (e.g., age, gender) and the data (e.g. interviews, scales, LLM-generated documents) collected during the study would be anonymized and destroyed after the study concluded. Each participant who completed the study received a 20 USD compensation.\u003c/p\u003e\n\u003ch3\u003ePreliminary Study: Interview for developing PFCNs using LLM\u003c/h3\u003e\n\u003cp\u003eIn this study, we conducted a preliminary study to explore the content, delivery context and concerns of PFCNs provided during ED care. We conducted individual interviews with nine patients who had previous ED experiences and seven clinicians who had provided care in the ED within one hour period. In the interview, we confirmed the experiences and challenges in ED care and communication of the participants. In addition, we asked them about the type of information provided during ED care and how it was delivered. Lastly, we identified the participants\u0026rsquo; expectations and challenges on the LLM-generated PFCNs in the ED setting. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eBased on the interview results, we identified: 1) the concept proof of LLM-generated PFCNs, 2) the content of the documents, 3) the appropriate scenarios and target patient groups for their use, and 4) concerns regarding the accuracy of the documents.\u003c/p\u003e\n\u003ch4\u003eConcept Proof of LLM-Generated PFCNs\u003c/h4\u003e\n\u003cp\u003eThrough the interview results, we identified the need for PFCNs to bridge the information gap between clinicians and patients in the ED setting. Many patients reported difficulty understanding medical information due to the fast-paced environment and the complex medical terminology often used by clinicians. On the other hand, clinicians found it challenging to provide detailed explanations to fulfill patients\u0026rsquo; information needs due to their heavy workload. Given these challenges, both patients and clinicians expressed support for incorporating PFCNs that include the patient\u0026rsquo;s condition, treatment process, and clinical information (e.g. diagnosis, test results) into the ED consultation process.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAdditionally, we confirmed that LLM can serve as effective technology to generate PFCNs that met the expectations of both patients and clinicians. Interview results revealed that patients desired documents with empathetic and reassuring language tailored to their individual symptoms and circumstances (e.g. Tommy, you came to the emergency room today with abdominal pain\u0026mdash;how difficult has it been for you?). Clinicians, in turn, expressed interest in using LLMs to create patient-friendly documents supplemented with additional information to support their explanations (e.g., explaining the purpose of laboratory or X-ray tests or providing information about appendicitis). Given that LLMs can generate documents tailored to users\u0026rsquo; needs through prompt engineering, we incorporated the expectations of both clinicians and patients into the LLM prompting process. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eContents of the PFCNs\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003cbr\u003e\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eInterview results revealed that patients expected the content of LLM-generated PFCNs to closely align with their clinicians\u0026rsquo; verbal explanations. Clinicians also stressed the importance of generating document content based on their verbal consultations and medical records to address potential legal concerns. Both patients and clinicians expressed that any discrepancies between the PFCNs and the clinicians\u0026rsquo; explanations during ED consultations could lead to patient confusion and increased communication burdens.\u003c/p\u003e\n\u003cp\u003eTo resolve these concerns, we designed the PFCNs to be directly based on the clinician\u0026rsquo;s clinical notes, which detail the patient\u0026rsquo;s condition, diagnosis, test results, and future treatment plans recorded in the clinical notes after consultations.\u003c/p\u003e\n\u003cp\u003eAdditionally, both patients and clinicians emphasized that the documents should be patient-friendly to ensure they are accessible even to individuals with low health literacy. To achieve thi, the documents should be written in plain language, replacing complex medical terms with simpler, easily understood explanations. Supplemental information should also be included to aid comprehension of medical terms when necessary. Additionally, to help patients easily identify key information (e.g., diagnoses, test names), the documents should incorporate headings, emphasis marks, and underlining. For instance, headings such as \u003cem\u003eDiagnosis\u003c/em\u003e and \u003cem\u003eFuture Treatment Plan\u003c/em\u003e can organize the content, while critical information like diagnoses and test names can be highlighted in bold to draw attention.\u003c/p\u003e\n\u003ch4\u003eAppropriate Scenarios and Patient groups for Applying the PFCNs in ED\u0026nbsp;\u003c/h4\u003e\n\u003cp\u003eInterviews revealed that long waiting times between consultations often left patients feeling uninformed and disconnected from their care process in the ER. Recognizing this issue, we identified three key post-consultation situations for delivering PFCNs: after the initial consultation, during follow-up consultations, and at discharge.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlthough the LLM-generated documents were designed to be patient-friendly and easy to read, both patients and clinicians expressed concerns about their applicability to patients with highly acute symptoms (e.g., dyspnea, acute heart attack), critical conditions (e.g., myocardial infarction, stroke) and low health literacy. Such patients might not be physically or mentally capable of reading and understanding the documents. Therefore, we selected patients with non-severe conditions who were better able to understand and benefit from the PFCNs. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConcerns in Accuracy of the PFCNs\u003cstrong\u003e\u003cem\u003e\u003cbr\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003eClinicians expressed concerns about the possibility of incorrect information being provided to patients through LLM-generated content. In particular, clinicians with prior experience using LLMs highlighted the risk of\u0026nbsp;\u003cem\u003ehallucinations\u003c/em\u003e, where inaccurate or fabricated information could lead to confusion in subsequent communication with patients.\u003cbr\u003eTo address these concerns, we integrated specific strategies into prompt engineering to improve the accuracy of the generated content. This included context-based prompts that provide background information to enhance the AI\u0026apos;s understanding of the task (e.g., \u0026quot;\u003cem\u003eThis document is written by a clinician and provided to patients in the emergency department.\u003c/em\u003e\u0026quot;) and error-handling prompts designed to anticipate and minimize potential inaccuracies (e.g., \u0026quot;\u003cem\u003eAvoid including any information not explicitly documented in the clinical notes\u003c/em\u003e\u0026quot;)\u0026nbsp;\u003csup\u003e33\u003c/sup\u003e.\u003cbr\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Additionally, to monitor how patients interpret and understand the information in the PFCNs, we implemented a feature allowing patients to submit questions to their clinicians. This feature not only helps clinicians address patient concerns but also serves as a safeguard to identify and rectify potential misunderstandings caused by inaccurate information.\u003c/p\u003e\n\u003ch3\u003eDevelopment of the LLM-based PFCNs\u003c/h3\u003e\n\u003cp\u003ePrompt Engineering for Generating the PFCNs\u003cbr\u003e\u0026nbsp;Based on the results of the preliminary study, we developed prompts to generate PFCNs using clinicians\u0026rsquo; clinical notes after patient consultations. First, we selected GPT-4o (OpenAI, https://openai.com/) as the LLM, as it was the most up-to-date model available in July 2024 and capable of supporting the Korean language.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSecond, we added prompts instructing the model to transform clinicians\u0026rsquo; clinical notes into PFCNs that patients could easily understand. To address the variability in how individual clinicians document clinical notes, we employed a zero-shot learning technique.\u003c/p\u003e\n\u003cp\u003eThird, we incorporated three key considerations identified from the preliminary study into the prompts: (1) ensuring accuracy, (2) enhancing patient-friendliness, and (3) improving readability (Textbox 1).\u003c/p\u003e\n\u003cp\u003eFinally, we ensured that the prompts reflected the specific content needed for the three main scenarios in which PFCNs would be provided in ER: initial consultation, follow-up consultation, and discharge consultation. For this, we added specific rules to the prompts for generating three types of documents. For example, prompts for initial and follow-up consultations included future treatment plans, while discharge consultation prompts emphasized post-discharge care instructions. The full prompts and detailed considerations are provided in Supplement Data 1.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u0026nbsp;Textbox 1. Considerations for prompt development for the PFCNs\u0026nbsp;\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1. \u0026nbsp;Ensuring accuracy: To minimize hallucinations, the generated content was strictly based on the clinician\u0026apos;s notes, avoiding any information that was not explicitly documented.\u003c/p\u003e\n \u003cp\u003e2. Enhancing patient-friendliness: The PFCNs were designed to include greeting messages, convert medical jargon into plain language, and provide guidance on the document\u0026rsquo;s question function.\u003c/p\u003e\n \u003cp\u003e3. Improving readability: Key points were highlighted using bold or underlined text, and the content was structured under subheadings for better organization.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cbr\u003eLLM-based system for Generating and Delivering the PFCNs\u003c/p\u003e\n\u003cp\u003eTo facilitate the generation and delivery of PFCNs during the ED consultations process, we designed a LLM-based system embedded with the prompts. This system enables clinicians to input clinical notes, generate PFCNs, and send them to patients. Patients can be provided the documents and submit questions which clinicians can address during subsequent consultations (Figure 1).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe system was developed using Next.js (Vercel, https://nextjs.org/) along with TypeScript, HTML, and CSS. On the clinician interface, clinicians can write a clinical note, generate and modify PFCNs, and send finalized documents to patients (Figure 2). On the patient interface, patients can read the PFCNs sent by their clinicians and send questions, which are displayed on the clinician interface. (Figure 3).\u003c/p\u003e\n\u003ch3\u003eEvaluation of LLM-generated PFCNs\u0026nbsp;\u003c/h3\u003e\n\u003ch4\u003ePhase 1: Clinician\u0026rsquo;s Evaluation of LLM-generated PFCNs\u0026nbsp;\u003c/h4\u003e\n\u003cp\u003eWe conducted an evaluation with 10 clinicians to assess the applicability of LLM-generated PFCNs for ED patients (Figure 3). First, an orientation session was held to explain the study\u0026rsquo;s purpose, experimental methods, and compensation plan. Next, the clinician participants conducted two role-play sessions with a simulated patient (researcher SIK) presenting with abdominal pain and low back pain, respectively (Table 1). In each scenario, clinicians performed three consultations (initial visit, follow-up, and discharge) and typed their consultation records in clinical notes for each consultation. Based on the clinical notes, PFCNs were generated and provided to the clinicians for evaluation. To observe changes in doctor-patient communication following document delivery, the simulated patient asked questions about the document\u0026rsquo;s contents during the consultation.\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u0026nbsp;Table 1. \u0026nbsp;Simulated role-play scenarios for clinicians in ED consultations\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eScenario\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eChief complaint\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eDescriptions\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e#1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eAbdomen pain\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eChief complaint: Right Lower abdomen pain\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eOnset: before 6 hours ago\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eSubjective symptoms: Right Lower abdomen pain, abdomen discomfort, dyspepsia \u0026nbsp;\u0026nbsp;\u003cbr\u003e\u0026nbsp;Physical findings: Tenderness(+), Rebound tenderness \u0026nbsp;(-)\u0026nbsp;\u003c/p\u003e\n \u003cp\u003ePrevious medical history: n/a\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e#2\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eLow back pain\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eChief complaint: low back pain\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eOnset: 3 hours ago\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eSubjective symptoms: Low back pain, Radiating leg pain, numbness\u003cbr\u003e\u0026nbsp;Physical findings: straight leg raise test. right (+)\u0026nbsp;\u003c/p\u003e\n \u003cp\u003ePrevious medical history: herniated intervertebral disk in L5-6, L6-7 \u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eClinicians evaluated six PFCNs generated from their medical note data on three criteria: Perceived Accuracy, Completeness, and Understandability [Supplement Data 2]. Perceived Accuracy was assessed on a 6-point Likert scale (ranging from 1 to 6) \u0026nbsp;to assess how accurately the document reflected the contents of the medical note (e.g. history taking, physical examination, diagnosis, treatment) \u003csup\u003e31\u003c/sup\u003e. Participants also explained the reasons for their accuracy ratings and commented on the document content. Completeness was evaluated by determining how comprehensively the medical note contents were reflected in the three types of documents (initial assessment, progress explanation, and discharge explanation). Participants were asked to note any missing information in the completeness evaluation form. Lastly, Understandability was measured using the Patient Education Materials Assessment Tool (PEMAT) \u003csup\u003e34\u003c/sup\u003e [Supplement Data 2]. PEMAT consists of two components: understandability and actionability. PEMAT can be used by both healthcare providers (e.g., physicians, nurses) and patients to evaluate the understandability and actionability of patient education materials. Since the PFCNs did not include actionable instructions for patients, the actionability component of PEMAT was excluded from this study. The PEMAT understandability section consists of 16 items, evaluated on a 2-point scale (Agree or Disagree). For items 6 and 13\u0026ndash;16, if the criteria are not applicable to the document being evaluated, \u0026quot;N/A\u0026quot; can be selected. The evaluation score is calculated as the percentage of items marked as \u0026quot;Agree\u0026quot; out of the total applicable items (excluding those marked as \u0026quot;N/A\u0026quot;). \u0026nbsp;According to PEMAT guidelines, a score of 70% or higher is generally considered indicative of adequate understandability. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAfter completing the evaluation forms, we conducted individual interviews with clinicians to gather their feedback on LLM-generated PFCNs and their potential applications. The interviews covered four main areas: (1) the process of reviewing the documents during consultations, (2) the communication process based on the documents, (3) suggestions for improving the documents, and (4) potential strategies for implementing the documents in clinical practice. Participants were informed that the interviews would be recorded, anonymized, and analyzed. They were encouraged to freely share their thoughts on the use of the PFCNs and were assured that they could withdraw from the interview at any time.\u003c/p\u003e\n\u003ch4\u003ePhase 2: Patients\u0026rsquo; Evaluation of LLM-generated PFCNs\u0026nbsp;\u003c/h4\u003e\n\u003cp\u003eWe conducted an evaluation with 20 patients to assess their perspectives on LLM-generated PFCNs (Figure 4). Patients with prior ED experience participated in simulated clinical scenarios. Each role-play session lasted approximately 30 minutes, during which patients engaged in three consultations (initial assessment, progress explanation, and discharge explanation) with the clinician (SIK). Patients role-played based on their recent ED experiences, presenting symptoms similar to their past cases. After each consultation, patients received PFCNs on a tablet PC and were instructed to review the document and ask questions about any unclear or confusing content during the subsequent consultation. Participants were informed in advance that the sessions would be recorded and consent was obtained. They were also advised to inform the researcher of any discomfort or concerns during the study and assured that they could withdraw at any time if discomfort persisted.\u003c/p\u003e\n\u003cp\u003eBefore and after the role-play sessions, patients were asked to evaluate their experience of participation in care. To assess patient participation, we adapted the Patient Participation Questionnaire (PPQ) \u003csup\u003e2\u003c/sup\u003e\u0026nbsp; [Supplement Data 2]. The PPQ is a validated tool designed to assess the extent of patient participation in healthcare settings. The PPQ evaluates various aspects of patient engagement, focusing on how patients perceive their involvement in medical decision-making, communication with healthcare providers, and the overall care process. The PPQ consists of 17 items. Items 1 to 16 are evaluated on a 4-point scale, while the 17th item (Overall Participation) is assessed using a 2-point scale (Yes or No). In the study, patients completed the PPQ to evaluate their participation in previous ED experiences before the role-play and then evaluated their participation during the simulated consultations using the PFCNs.\u003c/p\u003e\n\u003cp\u003eAfter reviewing the documents, participants evaluated their usefulness and understandability. To measure the perceived usefulness of the LLM-generated PFCNs, patients completed the Usefulness Scale for Patient Information Material (USE) \u003csup\u003e35\u003c/sup\u003e [Supplement Data 2]. The USE is designed to evaluate how well educational materials support patient education and informed decision-making, thereby empowering patients. The USE consists of 9 items, each rated on a scale from 0 to 10. The items are divided into three key domains: 1) Cognition (Items 1\u0026ndash;3, \u0026nbsp;Measures how patient education materials improve patients\u0026rsquo; knowledge), 2) Emotion (Items 4\u0026ndash;6, Assesses how the materials help patients cope emotionally with their condition), 3) Behavior (Items 7\u0026ndash;9, Evaluates how the materials support patients in managing their illness and adopting appropriate health behaviors). In the patient group, the understandability of the PFCNs was assessed using only the understandability section of the PEMAT tool \u003csup\u003e34\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eTo further explore patients\u0026rsquo; experiences with the PFCNs, individual interviews lasting approximately 30 minutes were conducted after the role-play sessions. The interviews focused on (1) the experience of reviewing and using the documents in the simulated consultation, (2) their experience of participation in ED care, and (3) suggestions for improving and implementing the documents. Participants were informed that the interviews would be recorded, anonymized, and analyzed. They were encouraged to share their thoughts freely and were assured that they could withdraw from the interview at any time.\u003cbr\u003e\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eAnalysis\u003c/h3\u003e\n\u003cp\u003eFor quantitative analysis, the PPQ data collected during the study were analyzed using Jamovi, an open-source data analysis tool (ver. 2.5, The Jamovi Project, www.jamovi.org). Changes in PPQ scores before and after the role-play sessions were examined. Since the Shapiro-Wilks test indicated a non-normal distribution (p \u0026lt; 0.05), we used the Wilcoxon Signed-Rank Test to compare pre- and post-study PPQ scores. The results of the PEMAT and USE scales were summarized in Excel (Microsoft) as total scores and separately for each document type (initial visit, follow-up, discharge). For completeness, researchers manually reviewed the documents to identify included and omitted items. Perceived accuracy was analyzed by calculating the average scores assigned by clinicians for each document in Excel. We also categorized factors that lowered accuracy based on clinicians\u0026rsquo; written feedback. To identify these factors, researchers coded the feedback through open coding and conducted a thematic analysis\u003csup\u003e36\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eFor qualitative analysis, interviews with both patients and clinicians were transcribed and coded by the researchers (SIK, JYP). Open coding and thematic analysis were conducted to identify themes related to the creation and delivery of the PFCNs, how they facilitated patient participation, and challenges encountered during their use \u003csup\u003e36\u003c/sup\u003e. The coding process was iterative and continued until the researcher\u0026rsquo;s reached consensus on the themes. Interviews and surveys were conducted and analyzed in Korean to ensure precise interpretation of participants\u0026rsquo; original responses.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e \u003ch2\u003eParticipants\u003c/h2\u003e \u003cp\u003eA total of 30 participants (10 clinicians and 20 patients) participated in the LLM-generated PFCNs evaluation study. The demographic information of the participants was included in Multimedia Appendix 3. Among the clinician participants, 9 (90%) were junior doctors currently in their residency training. The average age of the clinicians was 29.4 years, with an average work experience of 4.7 years. Five clinicians (50%) were from the Department of Emergency Medicine. The average age of the patient participants was 29.5 years, consisting of 5 men and 15 women. Patients had an average of 4.65 previous ED visits. The primary reasons for their most recent ED visits included trauma (7 cases, e.g., ankle pain from hiking), infections (6 cases, e.g., EBV infection, fever), gastrointestinal problems (5 cases, e.g., abdominal pain, diarrhea), and psychosomatic issues (e.g., dyspnea in stressful situations).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec19\" class=\"Section2\"\u003e \u003ch2\u003eQuality Evaluation of PFCNs\u003c/h2\u003e \u003cdiv id=\"Sec20\" class=\"Section3\"\u003e \u003ch2\u003eGeneral Information\u003c/h2\u003e \u003cp\u003eIn this study, a total of 120 PFCNs (60 from clinicians and 60 from patients) were generated. The examples of the clinical notes written by clinicians and the PFCNs generated by LLM were included in Supplement Data 4. The average word count for the 120 clinical notes written by clinicians was 40.52 words, while the average word count for the 120 PFCNs was 231.31 words (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eQuality assessment results of the PFCNs\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"10\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003ePhase 1: clinician\u0026rsquo;s evaluation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c9\" namest=\"c6\"\u003e \u003cp\u003ePhase 2: patient\u0026rsquo;s evaluation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCategory\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInitial visit (n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFollow-up\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDischarge\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTotal (n\u0026thinsp;=\u0026thinsp;60)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eInitial visit (n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eFollow-up\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eDischarge\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eTotal (n\u0026thinsp;=\u0026thinsp;60)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003cp\u003e(n\u0026thinsp;=\u0026thinsp;120)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eWord count\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"9\" nameend=\"c10\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eClinical note [Avg. (SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e43.8\u003c/p\u003e \u003cp\u003e(13.98)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e24.8\u003c/p\u003e \u003cp\u003e(10.77)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e29.6\u003c/p\u003e \u003cp\u003e(7.62)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e32.73\u003c/p\u003e \u003cp\u003e(13.72)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e73.85\u003c/p\u003e \u003cp\u003e(17.02)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e31.65\u003c/p\u003e \u003cp\u003e(17.02)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e39.4\u003c/p\u003e \u003cp\u003e(7.89)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e48.3\u003c/p\u003e \u003cp\u003e(19.83)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e40.52\u003c/p\u003e \u003cp\u003e(18.69)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePFCN [Avg. (SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e261.5\u003c/p\u003e \u003cp\u003e(10.87)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e225\u003c/p\u003e \u003cp\u003e(25.08)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e210.5\u003c/p\u003e \u003cp\u003e(22.61)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e232.33\u003c/p\u003e \u003cp\u003e(29.66)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e264.75\u003c/p\u003e \u003cp\u003e(30.67)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e217.6\u003c/p\u003e \u003cp\u003e(24.31)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e208.5\u003c/p\u003e \u003cp\u003e(16,91)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e230.28\u003c/p\u003e \u003cp\u003e(34.84)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e231.31\u003c/p\u003e \u003cp\u003e(32.22)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eUnderstandability\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"9\" nameend=\"c10\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePEMAT Understandability score \u003c/p\u003e \u003cp\u003e[%, Avg. (SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e86.7\u0026nbsp; (8.21)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e86.3 (5.87)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e88.8 (5.87)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e87.2 (6.59)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e86.7\u0026nbsp; (9.07)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e86.3 (8.76)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e88.8 (9.92)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e87.2 (9.13)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e87.2 (7.93)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCompleteness\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"9\" nameend=\"c10\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNumber of omitted information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePerceived accuracy\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"9\" nameend=\"c10\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal [avg.(SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5.35 (0.75)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5.55 (0.83)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5.45 (0.60)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e5.45 (0.72)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eUsefulness\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"9\" nameend=\"c10\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUSE- Cognition [Avg.(SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e25.2 (3.53)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e25.6 (7.62)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e24.2 (7.13)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e25.0 (5.14)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUSE - Emotion\u0026nbsp; [Avg.(SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e20.8 (7.81)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e20.8 (7.63)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e21.1 (7.35)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e20.9 (7.47)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUSE - Behavior\u0026nbsp; [Avg.(SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e24.1 (5.20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e24.1 (6.79)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e24. (7.42)1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e24.1 (6.42)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eUSE - Total [Avg.(SD)]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e70.0 (14.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e70.4 (16.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e69.3 (20.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e69.9 (17.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003eUnderstandability\u003c/h2\u003e \u003cp\u003eThe PEMAT understandability score evaluated by clinicians was 87.2% (\u0026gt;\u0026thinsp;70%), while the score assessed by patients was also 87.2% (\u0026gt;\u0026thinsp;70%) (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The PEMAT understandability score of \u0026gt;\u0026thinsp;70% observed in this study suggests that the PFCNs generated by the LLM-based system are understandable to patients.\u003c/p\u003e \u003cp\u003eBoth clinicians and patients rated the discharge PFCNs highest (clinicians: 88.8%, patients: 87.9%), but the follow-up PFCNs were rated lowest (clinicians: 86.3%, patients: 86.3%). The statement, \u0026ldquo;Medical terms in this document are used only to familiarize the patient with the diagnostic and treatment processes. When used, the meanings of the terms are defined,\u0026rdquo; received the lowest ratings from patient participants (applicable to 51 out of 60 documents) and the second-lowest ratings from clinicians (applicable to 50 out of 60 documents). The statement, \u0026ldquo;This document does not include information beyond its intended purpose,\u0026rdquo; received the lowest score from clinicians (applicable to 49 out of 60 documents) but was rated highly by most patients (applicable to 59 out of 60 documents).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003eCompleteness\u003c/h2\u003e \u003cp\u003eAmong the 60 PFCNs evaluated by clinicians, 55 included all the required elements (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). However, five discharge documents were missing specific content: three instances of missing \u0026ldquo;History and physical examination details\u0026rdquo; and two instances of missing \u0026ldquo;Expected diagnoses.\u0026rdquo;\u003c/p\u003e \u003cdiv id=\"Sec23\" class=\"Section3\"\u003e \u003ch2\u003ePerceived Accuracy\u003c/h2\u003e \u003cp\u003eThe average perceived accuracy of the PFCNs was 5.45 out of 6 (6 points: 44 documents, 5 points: 20 documents, 4 points: 5 documents, 3 points: 1 document) (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The accuracy scores for the three types of documents were: initial consultation document 5.35, follow-up explanation document 5.55, and discharge document 5.45. Since the perceived accuracy scores for the three types of documents did not meet normality assumptions, a Friedman non-parametric repeated measures ANOVA was conducted, which showed no statistically significant difference between the scores (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.066, \u0026gt;\u0026thinsp;0.05).\u003c/p\u003e \u003cp\u003eWe analyzed the factors contributing to reduced accuracy in the PFCNs to identify issues affecting the quality and usability of the documents. Based on the perceived accuracy evaluation results and clinicians\u0026rsquo; interview findings, we identified two primary factors: 1) Hallucinations and 2) Misalignments. Hallucinations refer to systematic errors in the content generation process, such as missing, inaccurate, or fabricated information does not present in the clinician\u0026rsquo;s original clinical notes \u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. These errors often arise from the LLMs\u0026rsquo; tendency to infer or fabricate details beyond the provided input. On the other hand, we defined misalignments which are characterized by discrepancies between the clinician\u0026rsquo;s intent or reasoning and the content of the generated document, reflecting a failure to capture the subtleties of clinical judgment or context. Therefore, the key distinction lies in their origins: hallucinations stem from the LLMs\u0026rsquo; content creation process, while misalignments result from the LLMs\u0026rsquo; inability to accurately interpret and convey the clinician\u0026rsquo;s intent.\u003c/p\u003e \u003cp\u003eAddressing both issues is essential for improving the accuracy, usability, and trustworthiness of LLM-generated documents in clinical settings \u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e,\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e. To better understand the underlying challenges of LLM-generated PFCNs, we further categorized these two factors into detailed subtypes (Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). 1) Hallucinations included missing information (e.g., outpatient follow-up instructions documented in the clinical note but omitted from the PFCNs), inaccurate information (e.g., marking the location of the appendix on the left side), and the addition of the unmentioned information (e.g., abdominal ultrasound details not recorded by the clinician were included). 2) Misalignments involved discrepancies between the clinician\u0026rsquo;s own reasoning process and the document content. For example, while the clinician did not document the reasoning behind empirical antibiotic use in the clinical note, the PFCNs incorrectly stated that it was \u0026ldquo;to identify the source of infection,\u0026rdquo; whereas the clinician\u0026rsquo;s primary intent was \u0026ldquo;symptom relief and bacterial eradication.\u0026rdquo; Other cases included unintended disclosure of information (e.g., revealing the reason for a patient\u0026rsquo;s refusal of an MRI, which was documented for legal protection) and awkward phrasing in the document (e.g., using unusual patient addresses like \u0026ldquo;Mr. Tommy,\u0026rdquo; which the clinician would not typically use). In addition, clinicians anticipated more supplemental information in the PFCNs, covering topics they frequently and repeatedly discussed with patients during consultations, such as waiting times for tests and test preparation requirements.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eFactors contributing to lower perceived accuracy of the LLM-generated PFCNs\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFactors\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCategory\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eExamples\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eHallucination\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOmission of information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- Outpatient follow-up instruction recorded in the clinical note was not reflected in the PFCN\u003c/p\u003e \u003cp\u003e- Expected diagnosis missing from the discharge PFCN.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGeneration of inaccurate information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- The location of the appendix was incorrectly recorded as being on the left side.\u003c/p\u003e \u003cp\u003e- The PFCN informed the patient that the medication for controlling abdominal pain intended for discharge was already administered in the ED.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAddition of the unmentioned information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- An abdominal ultrasound test was generated despite not being recorded by the clinician. \u003c/p\u003e \u003cp\u003e- The follow-up explanatory note includes information indicating that the patient is eligible for discharge.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMisalignment\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLack of reflection of the clinician\u0026rsquo;s clinical reasoning process and routine practices\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- Although the reason for empirical antibiotic prescription was 'infection source identification' in the document, the clinician\u0026rsquo;s intent was symptom relief and bacterial eradication\u003c/p\u003e \u003cp\u003e- The PFCN includes factors that may contribute to herniated intervertebral disc (HIVD) (e.g., severe impact from an accident); however, these seem unrelated to the information confirmed by the clinician during the patient\u0026rsquo;s consultation (e.g., a previous history of HIVD, desk job).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUnwanted disclosure of information\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- The PFCN disclosed the reason why the patient refused an MRI, which was recorded by the clinician for legal protection.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAwkward phrasing in the explanation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- Inconsistent or unusual patient address forms (e.g., 'Mr. Kim Young-ho,' 'Patient Kim Young-ho')\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnticipation to the supplement information on clician\u0026rsquo;s explanation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- Waiting times for blood tests and imaging (CT, MRI) examinations.\u003c/p\u003e \u003cp\u003e- Preparation requirements for imaging tests (e.g., removing metallic objects from clothing, remaining still during the procedure).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec24\" class=\"Section2\"\u003e \u003ch2\u003eUsefulness\u003c/h2\u003e \u003cp\u003eThe results of the patients\u0026rsquo; usefulness evaluation showed USE scores of 70.0 for the initial visit document, 70.4 for the follow-up document, and 69.3 for the discharge document (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Since the usefulness scores for the three document types did not meet normality assumptions, a Friedman non-parametric repeated measures ANOVA was conducted, which confirmed that the score differences among the initial, progress, and discharge documents were not statistically significant (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026gt;\u0026thinsp;0.05).\u003c/p\u003e \u003cp\u003eThe average scores for each subcategory of the usefulness evaluation were as follows: Cognition scored the highest at 8.32, followed by Behavior at 8.02, and Emotion at 6.96. ​​These results indicate that patients found the PFCNs particularly useful for acquiring information, followed by their usefulness in supporting communication during the consultation process.\u003c/p\u003e \u003cp\u003e To investigate the relationship between usefulness and patient participation, we examined the correlation between USE scores and PPQ scores. Using the PPQ scores measured after the study, rather than the changes in PPQ scores before and after the study, is more appropriate because the post-study PPQ scores directly reflect the influence of the PFCNs on patient participation. The analysis revealed a Pearson correlation coefficient of 0.86 between the PPQ scores after the study and the usefulness scores. These results suggest that patients who perceived the PFCNs as more useful were more likely to actively participate in the consultation process.\u003c/p\u003e \u003cdiv id=\"Sec25\" class=\"Section3\"\u003e \u003ch2\u003ePatient Participation with the Use of PFCNs\u003c/h2\u003e \u003cdiv id=\"Sec26\" class=\"Section4\"\u003e \u003ch2\u003eChanges in PPQ score before and after ED Consultations Role-Play\u003c/h2\u003e \u003cp\u003eThe results showed that the patient participation score in the simulated consultation with PFCNs was 56.5, a statistically significant increase compared to the score of 40 from previous ED consultations (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001) (Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). Since the subcategories Involvement, Information, and Communication did not meet normality assumptions, the Wilcoxon test was used, and all three showed significant score increases after the simulated consultations (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001). The subcategory Relationship to Staff met the normality assumption and was analyzed with a student\u0026rsquo;s paired samples t-test, which also indicated a significant score increase (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001). For the Overall Assessment of Involvement, a Wilcoxon test was conducted, and the score increase was again statistically significant (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePPQ scores of the patients before and after simulated role-play in ED consultations\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCategory\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBefore Role-play \u003c/p\u003e \u003cp\u003e[Avg. (SD)] (n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAfter Role-Play\u003c/p\u003e \u003cp\u003e[Avg. (SD)] (n\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e-value\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePPQ score\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInvolvement\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9.3 (2.18)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e13.35 (2.64)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInformation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10.3 (1.89)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e14.1(1.86)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCommunication\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7.35 (1.31)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e10.3 (1.78)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRelationship to staff\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e11.9 (1.94)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e16.8 (1.90)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOverall assessment of involvement\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1.15 (0.37)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.9 (0.31)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e40 (6.13)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e56.6 (8.56)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec27\" class=\"Section3\"\u003e \u003ch2\u003ePatient Perception of the LLM-generated PFCNs\u003c/h2\u003e \u003cp\u003eThrough the patient interview results, we found that patients perceived PFCNs as supportive for: 1) preparing for the consultation, 2) supplementing the clinicians\u0026rsquo; explanation, 3) providing emotional assurance, and 4) enhancing relationships with clinicians.\u003c/p\u003e \u003cp\u003eFirst, the LLM-generated documents encouraged patients to participate more actively in their ED consultations by helping them prepare questions in advance based on the information of the documents (e.g. \"Could you let me know if hospitalization will be necessary? Why is the pain medication administered only after the physical examination and imaging?\" ) Most patients noted that the hectic ED environment often made it difficult to think of questions during consultations. However, the PFCNs provided them with time to reflect and formulate more specific questions, with some even revising their questions based on the information in the documents.\u003c/p\u003e \u003cp\u003e \u003cem\u003e\u0026ldquo;Now I understand why certain tests were done and what the results mean. This really helps reduce the information gap. I feel like I can prepare more refined questions for the doctor. For example, instead of asking, \u0026lsquo;Is this just a cold?\u0026rsquo;\u0026mdash;which wastes time and might frustrate the doctor\u0026mdash;I could ask, \u0026lsquo;Is it influenza, rhinovirus, or Epstein-Barr virus?\u0026rsquo; The doctor would likely see that I\u0026rsquo;m engaged and want to participate in decision-making, which could lead to better communication.\u0026rdquo;\u003c/em\u003e (Patient 4)\u003c/p\u003e \u003cp\u003eSecond, the PFCNs supplemented the clinician's verbal explanations, helping patients better understand their condition and treatment progress. Especially in Korea\u0026rsquo;s busy tertiary hospitals\u003csup\u003e\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e, these documents could be useful for acquiring information about expected diagnosis, test types, test results and discharge education. Patients also reported that during waiting times, they could read the document and look up unfamiliar terms (e.g. appendicitis, migraine) using online search engines to better understand their situation.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eI really liked how everything\u0026mdash;my treatment plan and procedures\u0026mdash;was clearly outlined. From what I remember, the ED is such a chaotic place where I\u0026rsquo;m not sure what to do next. Sometimes I\u0026rsquo;d even have to go ask the staff, \u0026lsquo;When is this going to be done?\u0026rsquo; And being in pain makes it even harder to keep track of everything. But with everything organized in the document, I feel like I could\u0026rsquo;ve said, \u0026lsquo;Hey, I haven\u0026rsquo;t done this yet,\u0026rsquo; more easily.\u003c/em\u003e (Patient 3)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eThird, patients reported that reviewing the PFCNs provided emotional reassurance. Revisiting the clinician\u0026rsquo;s explanations helped alleviate the anxiety they felt during the consultation. Additionally, the inclusion of empathetic comments, such as \u003cem\u003e\u0026ldquo;Wishing you a speedy recovery,\u0026rdquo;\u003c/em\u003e was comforting.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eUnless it\u0026rsquo;s an extremely critical situation, most ED patients are conscious and aware. Seeing the explanations laid out in the document helps me feel reassured. I can clearly see my condition and the steps ahead, which makes me feel more at ease.\u003c/em\u003e (Patient 9)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eFourth, patients reported that the PFCNs could improve their relationships with medical staff. By reducing unnecessary questions during consultations, patients perceived that the clinician could focus more on their specific needs and concerns. Clinicians also reported that receiving pre-submitted patient questions allowed them to better understand the patient\u0026rsquo;s specific concerns, level of comprehension, and health literacy, enabling them to adjust their communication style and explanations to better meet the patient\u0026rsquo;s needs. As a result, clinicians anticipated that the PFCNs could ease the burden of challenging conversations and enhance the overall quality of patient interactions.\u003c/p\u003e \u003cp\u003e \u003cem\u003e\u0026ldquo;I feel like this would make the doctor-patient relationship more positive because I wouldn\u0026rsquo;t have that unnecessary guilt for asking too many questions. From the doctor\u0026rsquo;s perspective, having pre-prepared questions would probably be much easier than answering impromptu ones. It could even improve workflow efficiency.\u0026rdquo;\u003c/em\u003e (Patient 2)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec28\" class=\"Section2\"\u003e \u003ch2\u003eChallenges in the Applying PFCNs to ED consultations\u003c/h2\u003e \u003cdiv id=\"Sec29\" class=\"Section3\"\u003e \u003ch2\u003eClinicians\u0026rsquo; Challenges from Decreased Information Accuracy\u003c/h2\u003e \u003cp\u003eClinicians highlighted that inaccurate or missing information in the PFCNs (e.g., test types, waiting times, or outpatient appointments) could lead to increased communication burdens and additional work. For instance, five clinicians pointed out that while an ultrasound test was not documented in the clinical notes for a patient with abdominal pain, the PFCNs included ultrasound results, which they deemed medically valid. However, they noted that this discrepancy could lead to patient complaints and increased workload.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eThe ultrasound inclusion seemed like a system error. If there\u0026rsquo;s a mismatch between tests conducted and those mentioned, patients often file complaints, and it can become a big issue.\u003c/em\u003e (Clinician 9)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAdditionally, clinicians expressed concerns about potential legal liability when PFCNs included diagnoses or tests not explicitly mentioned in their clinical notes.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eThe differential diagnoses I list might be used as evidence, and that could become problematic if communicated through the document... For example, in cases of back pain, I initially considered a fracture but didn\u0026rsquo;t record it. If it turns out to be a fracture, patients might ask why I didn\u0026rsquo;t document or address it earlier.\u003c/em\u003e (Clinician 6)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eMoreover, clinicians emphasized that providing inaccurate or incomplete information could lead to conflicts with patients and undermine trust in the doctor-patient relationship.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eIf there\u0026rsquo;s a discrepancy between the clinician\u0026rsquo;s explanation and the text received, the patient might distrust one side. This could challenge the trust between the patient and the clinician, and rebuilding that trust could cause significant stress for the clinician.\u003c/em\u003e (Clinician 3)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e\n\u003ch3\u003eDifferences in Perspective on the PFCNs Between Patients and Clinicians\u003c/h3\u003e\n\u003cp\u003eAlthough the LLM-generated documents aimed to bridge the information gap between patients and clinicians by supplementing clinician\u0026rsquo;s clinical notes, patients often found the information insufficient for understanding their symptoms and care process. While patients desired more comprehensive information, clinicians preferred to limit the content to minimize legal risks.\u003c/p\u003e \u003cp\u003e \u003cem\u003e\u0026ldquo;To me, the document felt like a simple summary of what was mentioned during the consultation. I think more detailed information could have been included. For example, during the long waiting times, I could\u0026rsquo;ve read about what a CT scan involves and how to prepare for it. Or there could\u0026rsquo;ve been more detailed post-discharge instructions, like dietary recommendations. That kind of additional guidance would\u0026rsquo;ve been really helpful, but the document only included what the doctor said, which felt like a missed opportunity.\u003c/em\u003e (Patient 8) \u003cem\u003e\u0026rdquo;\u003c/em\u003e\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eWhat I\u0026rsquo;m most worried about is that this AI generates the document based on the clinical notes I\u0026rsquo;ve written. If something gets left out, that\u0026rsquo;s not a big deal. But if something is misrepresented or, worse, if even one piece of incorrect information gets included, it means the patient is getting wrong information. As a doctor, I\u0026rsquo;d have to take on the ethical responsibility and legal risks, and that feels way too heavy to handle.\u003c/em\u003e (Clinician 10)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eAdditionally, clinicians evaluated the PFCNs based on the accuracy and alignment with their clinical intent, focusing on whether objective facts were accurately reflected. In contrast, patients primarily trusted the documents if they believed the content mirrored the clinician\u0026rsquo;s verbal explanation. This difference sometimes prevented patients from identifying errors in the documents (e.g., misdiagnoses in the initial assessment document).\u003c/p\u003e \u003cp\u003e \u003cem\u003e\u0026ldquo;Since the content matched exactly what the doctor told me, I just accepted it as is. I didn\u0026rsquo;t feel there was anything untrustworthy about it.\u0026rdquo;\u003c/em\u003e (Patient 1)\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e \u003cem\u003eIn the document, it might mention general risk factors or the severity of symptoms on its own, but sometimes these things don\u0026rsquo;t align with the doctor\u0026rsquo;s original intent. Like, when you order a test, the reason for the test, or when you prescribe a treatment, the purpose of the treatment\u0026mdash;things like what condition the patient actually has or what they need to be cautious about\u0026mdash;might end up being reflected differently from what I intended.\u003c/em\u003e (Clinician 4)\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e"},{"header":"Discussion","content":"\u003cdiv id=\"Sec32\" class=\"Section2\"\u003e \u003ch2\u003ePromoting Patient Participation Through LLM-Generated PFCNs\u003c/h2\u003e \u003cp\u003eThis study aimed to verify how LLM-generated documents could enhance patient participation in the ER setting. The key findings are as follows: (1) LLMs successfully generated PFCNs suitable for patient comprehension, as demonstrated by a PEMAT understandability score of 87.2% (\u0026gt;\u0026thinsp;70%); (2) providing LLM-generated PFCNs significantly increased patients\u0026rsquo; PPQ scores (P\u0026thinsp;\u0026lt;\u0026thinsp;0.05); and (3) the documents facilitated patient participation during ER consultations by expanding opportunities for communication through document review and question submission, improving understanding of clinical information, alleviating anxiety through clear explanations, and strengthening relationships with clinicians.\u003c/p\u003e \u003cp\u003eOur findings demonstrate that LLM technology can be effectively utilized to generate PFCNs based on clinicians\u0026rsquo; clinical notes for ED patients. Since providing PFCNs during ER consultations is a new concept, unlike previous studies \u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e,\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e, direct comparisons between clinician-written and LLM-generated documents were not considered in the study. However, when comparing the PEMAT understandability score of 81% reported by Zaretsky et al. (2024) for inpatient discharge summaries generated using GPT-4, the 87.2% score from our study demonstrates that the understandability of the PFCNs was well ensured \u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. In terms of accuracy, Zaretsky et al. reported that 54% (54 out of 100) of their documents received a top rating, whereas our study achieved 73.3% (44 out of 60) \u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. Therefore, although we did not use other LLMs (e.g. Bard, Gemini), our results supported the previous study results that LLMs is a promising tool to generate clinical documents with a quality level comparable to clinicians.\u003c/p\u003e \u003cp\u003eDespite addressing concerns about hallucinations in the preliminary study by incorporating relevant adjustments into the prompts, issues with the accuracy of LLM-generated documents due to hallucinations were still observed in the study results. Furthermore, our study identified that misalignments, where clinicians\u0026rsquo; intentions were not accurately reflected in the PFCNs, were a significant factor lowering perceived accuracy. This finding highlights the importance of not only ensuring the inclusion of accurate and well-documented clinical information but also reflecting the clinician\u0026rsquo;s intent to improve the accuracy and reliability of PFCNs provided to patients.\u003c/p\u003e \u003cp\u003eOur study results demonstrate that LLM-generated PFCNs can facilitate patient participation in the ED consultation process. Through the USE scores, we observed that users found the documents particularly helpful for acquiring information and supporting communication. Moreover, the findings indicated that clinicians who were more inclined to utilize the documents in their practice were also more likely to engage patients in the care process. These results align with prior studies demonstrating the benefits of using LLMs to deliver patient-friendly information, improving patients\u0026rsquo; understanding of medical content and their participation in decision-making \u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e,\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e,\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e. Expanding on this, our study is the first to confirm that AI-generated documents in the ED setting can enhance patient communication and strengthen the patient-physician relationship, providing patients with opportunities to actively participate in their care.\u003c/p\u003e \u003cdiv id=\"Sec33\" class=\"Section3\"\u003e \u003ch2\u003eImplications of LLM Development for ED Settings\u003c/h2\u003e \u003cp\u003eThe study highlights the potential of LLM-generated explanations to support ED consultation and workflow, particularly in busy and fast-paced environments such as tertiary hospital ED. By supplementing verbal explanations with detailed written summaries, clinicians can save time and ensure consistency in patient education. However, the need for clinicians to review and correct inaccuracies in LLM outputs may initially increase workload. Therefore, considering the challenges identified in this study and the heavy workload on emergency department workflows, we proposed implications for development of LLMs for generating and implementing PFCNs in ED settings.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec34\" class=\"Section3\"\u003e \u003ch2\u003eCollaboration Between Clinicians and LLM-based Systems to Enhance Accuracy and Alignment\u003c/h2\u003e \u003cp\u003eWe identified critical challenges related to the accuracy and alignment of LLM-generated content. Hallucinations (e.g., generation of incorrect or unmentioned information) and misalignments (e.g., failure to reflect the physician's intent or clinical reasoning) emerged as potential critical barriers. These issues not only risk patient confusion but may also strain patient-clinician relationships and increase the clinician's workload. For instance, discrepancies between clinical notes and generated explanations led to concerns about medico-legal accountability and the potential for patient complaints.\u003c/p\u003e \u003cp\u003eTo address these challenges, collaboration between clinician and AI systems during the document generation process could minimize errors and ensure explanations accurately reflect clinicians\u0026rsquo; intentions. However, considering the heavy workload of clinicians, it is essential to ensure that a robust validation process for LLM outputs does not become an excessive burden. For instance, in MedKnowts, the system assists ED clinicians to document patients\u0026rsquo; clinical information in Electronic Health Records (EHRs) rapidly by providing autocomplete suggestions for typed medical terms (e.g., WBC, WBCCAST) or displaying value trends for recently recorded items (e.g., CK, CRP) \u003csup\u003e\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e. While this prior work introduced features to support clinicians in the documentation process, we further propose incorporating editing features, such as highlighting or deletion, to better capture clinicians' intention and reflect them in LLM-generated documents.\u003c/p\u003e \u003cp\u003eTo achieve this, future research should explore how various factors\u0026mdash;such as the context of documentation (e.g., initial visit, follow-up, discharge), the clinician\u0026rsquo;s level of experience, and the patient\u0026rsquo;s perceived health literacy\u0026mdash;are integrated into the editing process. For example, during initial visits, a less experienced junior clinician might deliberately remove differential diagnoses from the generated document to minimize potential patient complaints about their medical note.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e\n\u003ch3\u003eBalancing Information Sensitivity and Comprehensiveness\u003c/h3\u003e\n\u003cp\u003eOur findings also underscore the importance of balancing information sensitivity with comprehensiveness. We identified differing perspectives on the desired depth of information between patients and clinicians, which align with findings from previous studies \u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e,\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e\u003c/sup\u003e. To meet a balance between these preferences, it is required to consider both practical and ethical factors carefully in prompting.\u003c/p\u003e \u003cp\u003eOne approach could be offering layered information, where a core explanation provides essential details (e.g. reason for inferring the diagnosis, necessity of the test), and additional resources (e.g. costs of the test, general information about the disease) are available for patients seeking deeper understanding using external links such as hyperlink connected to the hospitals\u0026rsquo; clinical education page. This modular design could accommodate varying levels of patient health literacy and interest.\u003c/p\u003e \u003cp\u003eIn addition, it is crucial to ensure that patients are fully aware of the potential inaccuracies and errors in AI-generated information. To address this, the document should clearly indicate the possibility of errors, and healthcare providers should explicitly notify patients that the content was generated by AI.\u003c/p\u003e\n\u003ch3\u003eUnderstanding Patient Interaction with PFCNs\u003c/h3\u003e\n\u003cp\u003eThe findings of this study reveal that patients actively engaged with the PFCNs to enhance their understanding of medical information and participate more effectively in their care. By gaining deeper insights into patient interaction with PFCNs, healthcare providers can bridge communication gaps, enhance patient understanding, and foster more effective and collaborative care delivery. Therefore, it is essential not only to provide these documents but also to implement strategies that maximize their utility for patients.\u003c/p\u003e \u003cp\u003eOur findings highlight the diverse ways in which patients interact with PFCNs, providing valuable insights for future research. For example, in this study, patients utilized the questioning feature to seek additional information (e.g., explanation of waiting times or the necessity of certain tests), clarify difficult terms (e.g., \"What is an autoimmune disease?\"), or correct inaccuracies in the documents (e.g., amending the number of emergency department visits). Likewise, incorporating features like question-and-answer functionalities or highlighting unclear sections in the documents could help capture how patients engage with and interpret the content. Additionally, consideration should be given to how this feedback can be effectively delivered to clinicians to improve the accuracy and relevance of the explanations provided.\u003c/p\u003e \u003cdiv id=\"Sec37\" class=\"Section2\"\u003e \u003ch2\u003eLimitations\u003c/h2\u003e \u003cp\u003eAlthough this study aimed to assess whether LLM-generated PFCNs could be applied in clinical settings to support patient participation, the study was conducted in a lab setting due to safety and ethical concerns regarding the application of AI technology to patients. To address these limitations, we not only recruited patients with prior ED experience but also employed simulated role-play consultations to reflect patients\u0026rsquo; recent ED visits. However, our in-lab experiments may not fully capture the dynamics and atmosphere of an actual emergency department. Thus, future research is required to validate the efficacy of the LLM-generated PFCNs in real-world ED settings.\u003c/p\u003e \u003cp\u003eAnother limitation is the lack of generalizability due to the study results. We only generated 120 documents using LLM during the evaluation study with 20 patients and 10 clinicians in this study. In addition, most participants in this study were in their 20s and 30s, had high levels of digital literacy, and were familiar with using LLMs, which minimized any discomfort in receiving LLM-generated documents. In contrast, real ED environments involve patients with a wide range of symptoms, low health literacy or older ages. Therefore, further research is needed to involve a more diverse range of patients.\u003c/p\u003e \u003cp\u003eIn addition, while prominent LLMs such as Bard (Google AI) and Gemini (Microsoft) exist, we did not explore how different LLMs generate PFCNs. Given the rapid advancement of LLM technology, future research should consider selecting a variety of LLMs to identify those most suitable for generating PFCNs in ED settings.\u003c/p\u003e \u003cp\u003eLastly, validation of survey instruments is another limitation. Given the novelty of using generative AI with medical records, there is a lack of validated tools to measure the accuracy and completeness of these documents. Furthermore, due to the absence of appropriate readability assessment tools for Korean documents, we could not conduct readability tests, such as the Flesch\u0026ndash;Kincaid readability tests [28,29]. Although tools like the PPQ (Patient Participation Questionnaire) are being developed to evaluate patient engagement, there remains no consistently used definition of patient-centered care (PCC) or participation in the ED [1]. Therefore, future research should aim to develop and validate instruments that are tailored to assessing LLM-generated documents and patient participation, considering the specific research settings and hospital environments.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusions","content":"\u003cp\u003eThis study aimed to determine whether PFCNs based on clinicians\u0026rsquo; clinical notes could facilitate patient participation during emergency department care. To achieve this, we conducted interviews with patients and clinicians with ED experience to explore their perspectives on patient participation and the provision of PFCNs during care. Based on the interview findings, we designed LLM prompts and developed the content and application strategies for the PFCNs. Subsequently, patients and clinicians participated in simulated ED scenarios to evaluate the quality of LLM-generated documents and their impact on patient participation. Follow-up interviews were conducted to gather feedback on the experience of using the documents and suggestions for improvement and implementation. The results of this study demonstrate that LLM-generated PFCNs can enhance patient communication, decision-making, and relationship-building with healthcare providers, thereby promoting patient participation in emergency care. Additionally, this study identified potential challenges in applying LLM-generated documents in real ED settings and proposed considerations for their implementation. In the future, the findings of this study may not only contribute to improving the stressful workflow of emergency departments but also support the realization of patient-centered care by leveraging LLM-generated PFCNs to foster patient participation. Although this study aimed to assess whether LLM-generated PFCNs could be applied in clinical settings to support patient participation, the study was conducted in a lab setting due to safety and ethical concerns regarding the application of AI technology to patients. To address these limitations, we not only recruited patients with prior ED experience but also employed simulated role-play consultations to reflect patients\u0026rsquo; recent ED visits. However, our in-lab experiments may not fully capture the dynamics and atmosphere of an actual emergency department. Thus, future research is required to validate the efficacy of the LLM-generated PFCNs in real-world ER settings.\u003c/p\u003e \u003cp\u003eAnother limitation is the lack of generalizability due to the study results. We only generated 120 documents using LLM during the evaluation study with 20 patients and 10 clinicians in this study. In addition, most participants in this study were in their 20s and 30s, had high levels of digital literacy, and were familiar with using LLMs, which minimized any discomfort in receiving LLM-generated documents. In contrast, real ED environments involve patients with a wide range of symptoms, low health literacy or older ages. Therefore, further research is needed to involve a more diverse range of patients.\u003c/p\u003e \u003cp\u003eIn addition, while prominent LLMs such as Copilot(Microsoft, copilot.microsoft.com) and Gemini (Google, gemini.google.com) exist, we did not explore how different LLMs generate PFCNs. Given the rapid advancement of LLM technology, future research should consider selecting a variety of LLMs to identify those most suitable for generating PFCNs in ED settings.\u003c/p\u003e \u003cp\u003eLastly, validation of survey instruments is another limitation. Given the novelty of using generative AI with medical records, there is a lack of validated tools to measure the accuracy and completeness of these documents. Furthermore, due to the absence of appropriate readability assessment tools for Korean documents, we could not conduct readability tests, such as the Flesch\u0026ndash;Kincaid readability tests. Although tools like the PPQ are being developed to evaluate patient engagement, there remains no consistently used definition of patient-centered care (PCC) or participation in the ED \u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. Therefore, future research should aim to develop and validate instruments that are tailored to assessing LLM-generated documents and patient participation, taking into account the specific research settings and hospital environments.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePCC\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ePatient-Centered Care\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eED\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eEmergency Department\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eLLM\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eLarge Language Model\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEMR\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eElectronic Medical Record\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eAI\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eArtificial Intelligence\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePFCNs\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003epatient-friendly clinical notes\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eIRB\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eInstitutional Review Board\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePEMAT\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ePatient Education Materials Assessment Tool\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ePPQ\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ePatient Participation Questionnaire\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eUSE\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eUsefulness Scale for Patient Information Material\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eHIVD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eHerniated intervertebral disc\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEHRs\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eElectronic Health Records\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eConflicts of Interest\u003c/h2\u003e \u003cp\u003eNone declared\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eSIK and JP conducted the studies, analyzed the data, and contributed to writing the manuscript. TWK and WS designed the interviews and user studies, as well as provided feedback on the manuscript. TRK and WCC reviewed the study goal, advised how to properly implement LLMs to generate patient friendly clinical note in ED settings and supported participant recruitment at Samsung Medical Center. HH, as the corresponding author, supervised the overall study implementation and analysis processes. All authors reviewed and approved the final manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgements\u003c/h2\u003e \u003cp\u003eThis study received no funding.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe data sets generated during this study are available from the corresponding author upon reasonable request\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eWalsh, A. et al. Patient-centered care in the emergency department: a systematic review and meta-ethnographic synthesis. Int. J. Emerg. Med. 15, 36 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBerg et al. Questionnaire measuring patient participation in health care: Scale development and psychometric evaluation. European journal of cardiovascular nursing 19, 600\u0026ndash;608 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang, L. et al. The emergency patient\u0026rsquo;s participation in medical decision-making. J. Clin. Nurs. 25, 2550\u0026ndash;2558 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFrank, C., Fridlund, B., Baigi, A. \u0026amp; Asp, M. Patient participation in the emergency department: an evaluation using a specific instrument to measure patient participation (PPED). J. Adv. Nurs. 67, 728\u0026ndash;735 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAung, E., Donald, M., Williams, G. M., Coll, J. R. \u0026amp; Doi, S. A. R. Joint influence of patient-assessed chronic illness care and patient activation on glycaemic control in type 2 diabetes. Int. J. Qual. Heal. Care 27, 117\u0026ndash;124 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarello, S., Graffigna, G. \u0026amp; Vegni, E. Patient Engagement as an Emerging Challenge for Healthcare Services: Mapping the Literature. Nurs. Res. Pr. 2012, 905934 (2012).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSharma, A. E. et al. Patient Engagement In Health Care Safety: An Overview Of Mixed-Quality Evidence. Heal. Aff. 37, 1813\u0026ndash;1820 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGillani, S. M. R., Nevill, A. M. \u0026amp; Singh, B. M. A randomised controlled trail in diabetes demonstrating the positive impact of a patient activation strategy on diabetes processes and HbA1c: The WICKED project. Br. J. Diabetes 17, 58\u0026ndash;63 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSacrist\u0026aacute;n, J. A. Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients. BMC M\u0026eacute;d. Inform. Decis. Mak. 13, 6 (2013).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRogers, S. C. et al. CARES: Improving the Care and Disposition of Psychiatric Patients in the Pediatric Emergency Department. Pediatr. Emerg. Care 31, 173\u0026ndash;177 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarzban, S., Najafi, M., Agolli, A. \u0026amp; Ashrafi, E. Impact of Patient Engagement on Healthcare Quality: A Scoping Review. J. Patient Exp. 9, 23743735221125440 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEsmaeilzadeh, P., Dharanikota, S. \u0026amp; Mirzaei, T. The role of patient engagement in patient-centric health information exchange (HIE) initiatives: an empirical study in the United States. Inf. Technol. People 37, 521\u0026ndash;552 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXiao, Z., Li, T. W., Karahalios, K. \u0026amp; Sundaram, H. Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot. arXiv (2023) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arxiv.2302.00832\u003c/span\u003e\u003cspan address=\"10.48550/arxiv.2302.00832\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYu, Q. J. \u0026amp; Pun, J. Promoting Patient Engagement in Medical Informed Consent \u0026ndash; A Qualitative Study of Chinese Doctors\u0026rsquo; Communication Strategies. Heal. Commun. 38, 71\u0026ndash;79 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarty, H., Bogenst\u0026auml;tter, Y., Franc, G., Tschan, F. \u0026amp; Zimmermann, H. How well informed are patients when leaving the emergency department? comparing information provided and information retained. Emerg. Med. J. 30, 53 (2013).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHess, E. P. et al. Shared decision making in patients with low risk chest pain: prospective randomized pragmatic trial. BMJ 355, i6165 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLateef, F. Patient expectations and the paradigm shift of care in emergency medicine. J. Emergencies, Trauma Shock 4, 163\u0026ndash;167 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePark, S. Y. \u0026amp; Chen, Y. Design Opportunities for Supporting Patient Information Needs during an Emergency Visit. Proc. 10th EAI Int. Conf. Pervasive Comput. Technol. Healthc. (2016) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.4108/eai.16-5-2016.2263339\u003c/span\u003e\u003cspan address=\"10.4108/eai.16-5-2016.2263339\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTan, D. et al. Characterizing patient-friendly \u0026ldquo;micro-explanations\u0026rdquo;of medical events. Proc. SIGCHI Conf. Hum. Factors Comput. Syst. 2011, 29\u0026ndash;32 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePark, S. Y., Chen, Y. \u0026amp; Raj, S. Beyond Health Literacy. Proc. 2017 ACM Conf. Comput. Support. Cooperative Work Soc. Comput. 2179\u0026ndash;2192 (2017) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1145/2998181.2998357\u003c/span\u003e\u003cspan address=\"10.1145/2998181.2998357\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJohnson, C., Richwine, C. \u0026amp; Patel, V. Individuals\u0026rsquo; Access and Use of Patient Portals and Smartphone Health Apps, 2020. ASTP Health IT Data Brief [Internet] (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMannhardt, N. et al. Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study. arXiv (2024) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arxiv.2401.09637\u003c/span\u003e\u003cspan address=\"10.48550/arxiv.2401.09637\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWalker, J. et al. OpenNotes After 7 Years: Patient Experiences With Ongoing Access to Their Clinicians\u0026rsquo; Outpatient Visit Notes. J. M\u0026eacute;d. Internet Res. 21, e13876 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMoramarco, F. et al. Towards more patient friendly clinical notes through language models and ontologies. AMIA Annu. Symp. Proc. AMIA Symp. 2021, 881\u0026ndash;890 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBala, S., Keniston, A. \u0026amp; Burden, M. Patient Perception of Plain-Language Medical Notes Generated Using Artificial Intelligence Software: Pilot Mixed-Methods Study. JMIR Form. Res. 4, e16670 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeo, W., Yang, C. \u0026amp; Kim, Y.-H. ChaCha: Leveraging Large Language Models to Prompt Children to Share Their Emotions about Personal Events. Proc. CHI Conf. Hum. Factors Comput. Syst. 1\u0026ndash;20 (2024) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1145/3613904.3642152\u003c/span\u003e\u003cspan address=\"10.1145/3613904.3642152\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKim, T., Shin, D., Kim, Y.-H. \u0026amp; Hong, H. DiaryMate: Understanding User Perceptions and Experience in Human-AI Collaboration for Personal Journaling. Proc. CHI Conf. Hum. Factors Comput. Syst. 1\u0026ndash;15 (2024) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1145/3613904.3642693\u003c/span\u003e\u003cspan address=\"10.1145/3613904.3642693\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDing, H. et al. Evaluation and practical application of prompt-driven ChatGPTs for EMR generation. npj Digit. Med. 8, 77 (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchuemie, M. J. et al. Standardized patient profile review using large language models for case adjudication in observational research. npj Digit. Med. 8, 18 (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAnibal, J. T., Huth, H. B., Gunkel, J., Gregurick, S. K. \u0026amp; Wood, B. J. Simulated misuse of large language models and clinical credit systems. npj Digit. Med. 7, 317 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZaretsky, J. et al. Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format. JAMA Netw. Open 7, e240357 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDecker, H. et al. Large Language Model\u0026thinsp;\u0026ndash;\u0026thinsp;Based Chatbot vs Surgeon-Generated Informed Consent Documentation for Common Procedures. JAMA Netw. Open 6, e2336997 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKulkarni, N. D. Crafting Effective Prompts: Enhancing AI Performance through Structured Input Design. J. RECENT TRENDS Comput. Sci. Eng. 12, 1\u0026ndash;10 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShoemaker, S. J., Wolf, M. S. \u0026amp; Brach, C. Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information. Patient Educ. Couns. 96, 395\u0026ndash;403 (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eH\u0026ouml;lzel, L. P. et al. Usefulness scale for patient information material (USE) - development and psychometric properties. BMC M\u0026eacute;d. Inform. Decis. Mak. 15, 34 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBraun, V. \u0026amp; Clarke, V. Using thematic analysis in psychology. Qualitative research in psychology 3, 77\u0026ndash;101 (2006).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeo, J. et al. Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study. J. M\u0026eacute;d. Internet Res. 26, e58329 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJung, H. et al. Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients. arXiv (2024) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.48550/arxiv.2404.05144\u003c/span\u003e\u003cspan address=\"10.48550/arxiv.2404.05144\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChang, H. et al. Emergency department crowding: a national data report. Clin. Exp. Emerg. Med. 11, 331\u0026ndash;334 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTung, J. Y. M. et al. Comparison of the Quality of Discharge Letters Written by Large Language Models and Junior Clinicians: Single-Blinded Study. J. M\u0026eacute;d. Internet Res. 26, e57721 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eProbst, M. A., Tschatscher, C. F., Lohse, C. M., Bellolio, M. F. \u0026amp; Hess, E. P. Factors Associated With Patient Involvement in Emergency Care Decisions: A Secondary Analysis of the Chest Pain Choice Multicenter Randomized Trial. Acad. Emerg. Med. 25, 1107\u0026ndash;1117 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMurray, L. et al. MedKnowts: Unified Documentation and Information Retrieval for Electronic Health Records. 34th Annu. ACM Symp. User Interface Softw. Technol. 1169\u0026ndash;1183 (2021) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1145/3472749.3474814\u003c/span\u003e\u003cspan address=\"10.1145/3472749.3474814\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFritz, Z., Schlindwein, A. \u0026amp; Slowther, A.-M. Patient engagement or information overload: patient and physician views on sharing the medical record in the acute setting. Clin. Med. 19, 386\u0026ndash;391 (2019).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"patient-centered care, patient participation, large language model, emergency department, clinical note, patient-friendly clinical note","lastPublishedDoi":"10.21203/rs.3.rs-5958826/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5958826/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eTo achieve patient-centered care (PCC), it is essential to provide patients with sufficient information about their medical conditions and treatment processes to actively engage in their care. However, the busy and crowded nature of the emergency department (ED) setting, combined with differences in communication styles, varying levels of health literacy, and other barriers between clinicians and patients, makes effective communication and participation challenging. Recent studies have focused on applying large language model (LLM) technologies to create patient-friendly documents that present patients\u0026rsquo; medical information in plain language with the goal of enhancing patient understanding and participation. Nevertheless, further research is needed to evaluate the impact of LLM-generated patient-friendly documents on improving patient participation, particularly in high-pressure settings like the ED.\u003c/p\u003e\u003ch2\u003eObjective\u003c/h2\u003e \u003cp\u003eThis study aimed to develop patient-friendly clinical notes (PFCNs) generated by LLM, which transform clinicians\u0026rsquo; clinical notes into patient-friendly documents for use in ED consultations, and to evaluate whether PFCNs could enhance patient participation.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eA preliminary study was conducted to identify the contents, format, and scenarios of PFCNs in the ED setting by conducting individual interviews with patients (n\u0026thinsp;=\u0026thinsp;9) and clinicians (n\u0026thinsp;=\u0026thinsp;7). Based on these findings, we developed a system which generates PFCNs using the GPT-4o model, transforming clinicians\u0026rsquo; clinical notes from ED consultations into patient-friendly documents through zero-shot learning. To evaluate the efficacy of PFCNs, we conducted a simulated ED consultation role-play with patients (n\u0026thinsp;=\u0026thinsp;20) and clinicians (n\u0026thinsp;=\u0026thinsp;10), who were provided with PFCNs. After the simulations, the participants completed the scales to assess the quality of the documents and patient participation questionnaires (PPQ) during pre- and post- simulations, followed by post-individual interviews to explore how PFCNs supported patient participation.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eWe confirmed that the LLM successfully generated PFCNs (n\u0026thinsp;=\u0026thinsp;120) suitable for patient comprehension, as indicated by a Patient Education Materials Assessment Tool (PEMAT) understandability score of 87.2 (\u0026gt;\u0026thinsp;70%) validated by both clinicians and patients. Compared to previous ED visits without PFCNs, patients who received the documents demonstrated a significantly higher PPQ scale (\u003cem\u003eP\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.05). In line with the quantitative results in patients\u0026rsquo; participation, post-interview results revealed that PFCNs supported communication with clinicians, improved understanding of clinical information, provided emotional reassurance, and strengthened patient-clinician relationships. However, two key challenges were identified regarding the content and utility of the documents: (1) potential risks from inaccuracies in PFCNs and (2) differing perspectives between patients and clinicians on the documents. Based on these findings, we suggested practical strategies for implementing PFCNs in ED settings.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThis study demonstrates that PFCNs enhanced patient participation in ED consultations, supporting patient communication, decision-making, and relationship-building with healthcare providers. These findings suggest that PFCNs not only promote patient-centered care by supporting active patient participation but also have the potential to improve ED consultation and workflow effectively.\u003c/p\u003e","manuscriptTitle":"Enhancing Patient Participation in Emergency Department through Patient-Friendly Clinical Notes Generated by Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-02-12 07:03:02","doi":"10.21203/rs.3.rs-5958826/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"621a15a9-ada7-4fb8-bafc-693697878269","owner":[],"postedDate":"February 12th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":43912034,"name":"Health sciences/Health care/Health services"},{"id":43912035,"name":"Health sciences/Medical research"},{"id":43912036,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"}],"tags":[],"updatedAt":"2025-05-23T06:23:26+00:00","versionOfRecord":[],"versionCreatedAt":"2025-02-12 07:03:02","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5958826","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5958826","identity":"rs-5958826","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}