Guidelines vs Generative AI in CKD Patient Education: The Role of Prompt Engineering and Expert Blinded Evaluation

Guidelines vs Generative AI in CKD Patient Education: The Role of Prompt Engineering and Expert Blinded Evaluation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Guidelines vs Generative AI in CKD Patient Education: The Role of Prompt Engineering and Expert Blinded Evaluation Lutfullah Zahit Koc, Sevgi Gulsen Koc, Ayca Inci, Osman Cagın Buldukoglu, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7507934/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 20 Feb, 2026 Read the published version in BMC Nephrology → Version 1 posted 9 You are reading this latest preprint version Abstract This study aimed to evaluate the accuracy, content quality, and readability of patient education responses related to chronic kidney disease (CKD) generated by large language models (ChatGPT-4o mini and Gemini) compared to clinical guidelines. Fifteen frequently asked CKD-related questions were selected using global Google Trends data and posed to both AI models and guideline-based sources. Responses were anonymized and evaluated by four independent nephrology professors using the CLEAR Tool, assessing completeness, appropriateness, evidence basis, and clarity. Both AI models significantly outperformed guideline responses across all CLEAR Tool domains (p < 0.001), with ChatGPT-4o mini achieving the highest median score (21.0 [IQR: 5.0] vs. Gemini: 17.0 [IQR: 5.0], Guideline: 13.0 [IQR: 2.0]). Initial readability analysis showed that guideline responses were easier to comprehend (FKGL: 9.40; FRE: 52.01) than AI-generated content (ChatGPT FKGL: 11.34, FRE: 36.17; Gemini FKGL: 9.62, FRE: 46.36). However, when a standardized instructional prompt was applied, AI responses demonstrated significant improvements in readability, reducing the required literacy level to approximately the 7th-grade (ChatGPT FKGL: 7.87, FRE: 64.23; Gemini FKGL: 7.13, FRE: 61.45). These findings highlight the potential of prompt-guided AI models to generate accurate, accessible educational content for CKD. Prompt engineering emerges as a practical tool to enhance clarity and usability, particularly for populations with limited health literacy. Integration with frameworks like Retrieval-Augmented Generation may further improve reliability and safety in digital health communication. Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Chronic kidney disease is an increasing global public health issue, affecting approximately 700 million people worldwide. The prevalence of CKD has reached 9.1% globally, and the mortality rate associated with this disease across all age groups increased by 41.5% from 1990 to 2017[ 1 ]. Known as the "silent epidemic," this disease is often diagnosed at advanced stages due to its asymptomatic nature in early phases and low awareness levels[ 2 ]. Furthermore, low socioeconomic status and limited health literacy accelerate the progression of the disease, increasing the risk of progression to end-stage renal disease[ 3 ]. Enhancing health literacy plays a crucial role in slowing disease progression by enabling patients to gain knowledge about CKD. Creating awareness in the early stages and implementing appropriate treatment strategies not only improve quality of life and reduce mortality and morbidity rates but also decrease the need for high-cost treatments such as dialysis[ 3 , 4 ]. In recent years, artificial intelligence (AI) technologies have introduced a new paradigm for enhancing health literacy and disease management. The rapid advancements in AI technologies hold significant potential to improve accessibility to healthcare services and democratize access to information[ 5 ]. Generative AI models, particularly large language models such as ChatGPT-4o mini (by OpenAI, San Francisco, CA) and Gemini (by Google, Mountain View, CA), have become easily accessible to a broad range of users. These models have the potential to facilitate patients' access to health information, providing personalized education and guidance[ 6 , 7 ]. However, further research is needed to evaluate the reliability and accuracy of these technologies. Strategies have been developed to enhance the accuracy of AI models in the medical field. One of the most commonly used strategies is prompt engineering, which aims to ensure that AI systems generate accurate, relevant, and reliable outputs. Prompt engineering involves designing and optimizing input instructions to Guideline AI models toward producing specific outputs. An optimal prompt ensures that the generated response contains accurate and user-appropriate information[ 8 , 9 ]. For example: "I am a patient with chronic kidney disease, and I have some questions about my condition. I would like to ask you these questions as if you are a nephrologist. Could you answer them in a way that an elementary school graduate can easily understand?" By explicitly stating the user's condition (patient with chronic kidney disease), assigning a role (as a nephrologist), and tailoring the response to the user's educational level (elementary school graduate), the likelihood of receiving accurate and relevant answers is increased. Previous studies have demonstrated the potential of AI-supported tools in diverse healthcare contexts, such as providing personalized dietary and exercise recommendations for obesity and delivering reliable, easy-to-understand information for conditions like hypothyroidism during pregnancy[ 6 , 7 ]. Building on these findings, this study is the first to systematically blinded compare AI-generated responses with guideline-based answers in the context of CKD. By integrating prompt engineering techniques, we aimed to improve both the readability and content accuracy of AI-generated educational materials. Materials and Methods This study was designed as a cross-sectional, comparative analysis to evaluate the accuracy, content quality, and readability of responses to selected patient questions about CKD. The overall methodology is summarized in Fig. 1 . Fifteen representative questions were selected based on two-year worldwide Google Trends data, focusing on globally searched patient queries related to CKD. This approach ensured thematic relevance and captured real-world informational needs observed in both clinical and digital settings. All questions are listed in Supplementary Table 1. To establish a benchmark, guideline-based reference answers were compiled from trusted nephrology sources, forming the "Guideline group." The same set of questions was then posed to both AI models: ChatGPT-4o mini and Gemini 1.5 Flash. To enhance reliability and reduce variability, each question was submitted in two independent sessions per model, and all responses were systematically documented for further evaluation. All responses—whether guideline-based or AI-generated—were anonymized and formatted in a standardized style. Four independent nephrology professors, each with over 15 years of experience, evaluated the responses in a blinded manner using the CLEAR Tool scoring system. Evaluators were unaware of the origin of each response and scored them across five subdomains: completeness, lack of false information, evidence-based accuracy, appropriateness, and relevance. This blinded design was implemented to reduce potential bias and ensure objective assessment. The CLEAR TOOL was developed based on a literature review of existing health information quality assessment frameworks, including DISCERN, PEMAT, and the CDC Clear Communication Index. While not directly adapted from a single tool, CLEAR incorporates conceptual elements such as completeness, evidence, and clarity that are shared with these validated instruments.[ 10 ]. Full scoring rubric for the CLEAR TOOL is provided in Supplementary Table 2. In addition to content quality, the readability of each response was assessed using two validated linguistic metrics: the Flesch-Kincaid Grade Level (FKGL) and the Flesch Reading Ease (FRE)[ 11 , 12 ]. These indices account for sentence length, word count, and syllable density. FKGL estimates the U.S. school grade level required to comprehend a text, while FRE assigns a score from 0 to 100, with higher scores indicating greater readability. This dual approach enabled a nuanced analysis of how accessible each response was to the average patient. To evaluate the impact of prompt engineering, the same 15 questions were re-submitted to both AI models under two conditions: without prompt and with a standardized instructional prompt. The prompt was designed based on best practices from the Gemini for Google Workspace Prompting Guide and included four key elements: persona ("a nephrologist"), task ("answer questions about chronic kidney disease"), context ("from the perspective of a patient with CKD"), and format ("in simplified language understandable by an elementary school graduate")[ 13 ]. The final prompt was: “I am a patient with chronic kidney disease, and I have some questions about my condition. Please answer them as if you are a nephrologist, but in a way that an elementary school graduate can easily understand.” This prompt was applied identically to both models. Each prompt-based query was initiated in a separate private browser session to prevent memory retention and ensure independence between responses. All answers were collected under consistent conditions using the default temperature and token settings of the public versions of ChatGPT-4o mini and Gemini, as of March 2025. Statistical Analysis Descriptive statistics were calculated. Data normality was assessed using Kolmogorov–Smirnov and Shapiro–Wilk tests. For data following a normal distribution, descriptive statistics were expressed as mean and standard deviation, and group comparisons were assessed using ANOVA, with Tukey's test applied for post hoc analysis. For data not following a normal distribution, descriptive statistics were presented as median values and interquartile ranges, and differences between groups were evaluated with the Kruskal-Wallis test. Post hoc analyses were conducted using the Mann-Whitney U test with Bonferroni correction. The evaluations of the professors regarding the responses were analyzed based on the total scores assigned for each question, using Intraclass Correlation to assess agreement. Results The Intraclass Correlation Coefficient (ICC) for the total scores was calculated to assess agreement among evaluators. The ICC value was 0.55, indicating moderate agreement among the evaluators. The groups were evaluated by comparing the 'total score,' derived from the sum of the CLEAR TOOL components, as well as each component individually. The median and interquartile range (IQR) of 'total scores' for each group were as follows: Guideline: Median = 13.0 (IQR = 2.0), ChatGPT-4o mini: Median = 21.0 (IQR = 5.0), Gemini: Median = 17.0 (IQR = 5.0) The Kruskal-Wallis test revealed a statistically significant difference in total scores between the three groups (H = 103.49, p < 0.001) (Fig. 2 ). Pairwise comparisons of the total scores using the Mann-Whitney U test with Bonferroni correction showed the following: Guideline vs. ChatGPT-4o mini: Median = 13.0 (IQR = 2.0) vs. 21.0 (IQR = 5.0), p < 0.001, Guideline vs. Gemini: Median = 13.0 (IQR = 2.0) vs. 17.0 (IQR = 5.0), p < 0.001, ChatGPT-4o mini vs. Gemini: Median = 21.0 (IQR = 5.0) vs. 17.0 (IQR = 5.0), p < 0.001 When comparing the CLEAR TOOL components across the three groups, statistically significant differences were observed for all components. Post hoc analyses with Bonferroni correction (p = 0.05/3) revealed that both AI models (ChatGPT-4o mini and Gemini) differed significantly from the guideline group across all domains. However, differences between ChatGPT-4o mini and Gemini were not statistically significant in all components. (Table 1 ). Table 1 Median and IQR scores for each CLEAR TOOL criterion across guideline, ChatGPT-4o mini, and Gemini groups. (Note: p 1 : Guideline-ChatGPT-4o mini, p 2 : Guideline-Gemini, p 3 : ChatGPT-4o mini-Gemini. Bolded comparisons indicate statistically significant differences after Bonferroni correction.) Post Hoc Analysis Guideline Median (IQR) ChatGPT-4o mini Median (IQR) Gemini Median (IQR) P value Completeness 2 (IQR: 1) 5 (IQR: 1) 4 (IQR: 1) p 1 < 0.001 p 2 < 0.001 p 3 < 0.001 Lack of False Knowledge 3 (IQR: 1) 4 (IQR: 1) 4 (IQR: 1) p 1 < 0.001 p 2 < 0.001 p 3 = 0,613 Evidence Based 3 (IQR: 1) 4 (IQR: 1) 3 (IQR: 1) p 1 < 0.001 p 2 < 0.001 p 3 < 0.001 Appropriateness 3 (IQR: 1) 4 (IQR: 1.25) 3 (IQR: 1) p 1 < 0.001 p 2 = 0,503 p 3 = 0,455 Relevance 3 (IQR: 1) 4 (IQR: 2) 3 (IQR: 1) p 1 < 0.001 p 2 = 0,293 p 3 = 0,822 Readability Readability levels were evaluated in two ways. First, questions were directly posed to AI tools (Without Prompt), and the FKGL and FRE scores of their responses, along with Guideline answers, were calculated (Table 2 ). Table 2 Readability scores (mean ± SD) of FKGL and FRE for guideline, ChatGPT-4o mini, and Gemini responses, with and without prompt use Group Without Prompt Prompt FKGL Mean ± SD FRE Mean ± SD FKGL Mean ± SD FRE Mean ± SD Guideline 9.40 ± 2.29 52.01 ± 14.85 ChatGPT-4o mini 11.34 ± 1.79 36.17 ± 10.85 7.87 ± 1.13 64.23 ± 6.16 Gemini 9.62 ± 1.55 46.36 ± 10.69 7.13 ± 1.78 61.45 ± 12.40 For the Without Prompt group, the Flesch–Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) values for the Guideline, ChatGPT-4o mini, and Gemini groups were calculated as follows: Guideline: FKGL mean = 9.40 ± 2.29 (SD), FRE mean = 52.01 ± 14.85 (SD), ChatGPT-4o mini: FKGL mean = 11.34 ± 1.79 (SD), FRE mean = 36.17 ± 10.85 (SD), Gemini: FKGL mean = 9.62 ± 1.55 (SD), FRE mean = 46.36 ± 10.69 (SD) Statistical analyses revealed a significant difference in FKGL between ChatGPT-4o mini and Guideline (p < 0.001), indicating that ChatGPT-4o mini’s responses required a higher educational level. Additionally, a significant difference was found between ChatGPT-4o mini and Gemini in FKGL scores (p < 0.001). For FRE scores, significant differences were observed between ChatGPT-4o mini and Guideline (p < 0.001) as well as between ChatGPT-4o mini and Gemini (p < 0.001). ChatGPT-4o mini’s responses exhibited significantly lower readability scores, indicating greater difficulty in comprehension. Guideline group texts were generally at a high school level (9th grade) and moderately easy to read. In contrast, ChatGPT-4o mini’s responses were at the level of a high school senior and posed greater comprehension challenges. Gemini’s texts were positioned between Guideline and ChatGPT-4o mini, requiring an educational level equivalent to the beginning of high school (Fig. 3 ). When a prompt was provided, these readability metrics for the ChatGPT-4o mini and Gemini groups were calculated as follows (Table 2 ): ChatGPT-4o mini: FKGL mean = 7.87 ± 1.13 (SD), FRE mean = 64.23 ± 6.16 (SD), Gemini: FKGL mean = 7.13 ± 1.78 (SD), FRE mean = 61.45 ± 12.40 (SD) Statistical analyses revealed a significant difference in both FKGL and FRE scores between the Prompt and Without Prompt groups for both ChatGPT-4o mini and Gemini (p < 0.001). Providing a prompt made the responses of both models significantly easier to read and reduced the required educational level to below high school (7th grade level). With the use of a prompt, ChatGPT-4o mini's responses became closer to the readability and comprehensibility levels of the Guideline group in terms of ease of understanding and educational level. Compared to Without Prompt, the use of a prompt significantly improved the readability of responses and reduced their complexity for both ChatGPT-4o mini and Gemini (Fig. 4 ). Statistical analyses confirmed these improvements: the FKGL and FRE scores of both models showed significant differences between Prompt and Without Prompt conditions (p < 0.001), based on Mann-Whitney U tests comparing paired responses across all 15 questions. These findings indicate that prompt engineering is an effective strategy to reduce linguistic complexity and enhance patient accessibility in AI-generated health content. Discussion This study is among the first known blinded comparative evaluations of AI-generated versus guideline-based responses in the context of CKD. It incorporates several methodological strengths that distinguish it from previous research. First, the evaluation was conducted by four independent nephrologists, a higher number of expert raters than typically employed in similar studies, thereby improving the reliability and robustness of the results. Second, all responses—whether generated by AI or sourced from clinical guidelines—were evaluated under blinded conditions, minimizing potential bias. Unlike prior studies that often assess AI outputs against fixed references, this study applied a fully anonymized and independent evaluation protocol. Third, the study uniquely assessed the impact of prompt engineering on the readability and comprehensibility of AI outputs, revealing significant improvements. Collectively, these contributions enhance the scientific rigor of the study and underscore its relevance for advancing AI-supported patient education in nephrology. The results of this study align with existing literature. For instance, Zhang et al. demonstrated that ChatGPT-4o mini achieved 88% accuracy in providing accurate and relevant information on total knee replacement, underscoring its effectiveness in patient education[ 14 ]. Onder et al. found ChatGPT-4o mini-4's responses reliable in hypothyroidism management, but readability analyses revealed a requirement for university-level education[ 7 ]. Similarly, Wang et al. evaluated ChatGPT-4o mini-4 and 4o-mini for clinical support in lumbar disc herniation and reported accuracy and completeness scores exceeding 75%; however, the responses were deemed “very difficult to read”[ 15 ]. Acharya et al. assessed 15 lifestyle and 20 dietary questions from KDIGO and KDOQI guidelines answered by ChatGPT-4o mini-3.5, ChatGPT-4o mini-4, Gemini AI, and Bing AI. Responses were evaluated by nephrologists for accuracy. While the answers were generally accurate, misleading statements and irrelevant references were noted, particularly in ChatGPT-4o mini-3.5, ChatGPT-4o mini-4, and Gemini. All models delivered responses at a high school readability level, highlighting potential accessibility limitations for patients with low health literacy[ 16 ]. The tendency to generate inaccurate, out-of-context responses and provide incorrect references has been documented in other studies as well[ 9 , 17 ]. In our study, the readability and comprehensibility of text generated by AI models significantly improved following prompt-guided responses compared to pre-prompt outputs. The reduction of FKGL to the 7th-grade level highlights the potential of this technology to effectively provide information to a broader patient population. By making responses more understandable, prompt engineering can facilitate access to information for individuals with low health literacy. This approach may serve as an effective tool for designing personalized patient education materials, thereby enhancing health literacy. Within this context, our study takes a step toward assessing the reliability of responses generated by large language models such as ChatGPT-4o mini and Gemini in the healthcare domain. Beyond evaluating the accuracy and trustworthiness of these systems, the application of prompt engineering techniques has been found to enhance the readability and comprehensibility of AI-generated outputs. This contributes to producing more reliable and accessible content to counter misinformation. Notably, prompt engineering can serve as a valuable tool in improving AI-generated responses, making them more accurate and user-friendly, thereby reducing the spread of misinformation in healthcare. With the widespread use of social media, the speed and scale at which misinformation spreads have greatly increased. A significant portion of fake news is crafted to be engaging and emotionally impactful, leading to higher interaction rates[ 18 ]. This, in turn, facilitates the increased use and rapid dissemination of false information across social media platforms. Such developments pose serious risks, especially in the healthcare sector. AI-generated content often relies on statistical probabilities, which may lead to misleading or out-of-context information lacking proper source validation[ 19 ]. While artificial intelligence can accelerate the spread of misinformation, it also holds significant potential for detecting and mitigating false information[ 20 ]. Various strategies have been implemented to combat misinformation and disinformation. Primarily, AI-powered misinformation detection systems on social media platforms use natural language processing and machine learning techniques to identify inaccurate content[ 21 , 22 ]. However, research suggests that these systems should not only detect falsehoods but also encourage users to critically evaluate the credibility of the information they encounter[ 23 ]. In this regard, enhancing health literacy, raising public awareness, and fostering critical thinking through targeted educational initiatives are crucial. Additionally, it is essential for healthcare professionals and institutions to actively use social media to disseminate accurate information and counteract misinformation[ 18 , 20 ]. On the other hand, relying on large language models that utilize vast and unregulated datasets increases the risk of misinformation and hallucinations. To mitigate this, defining the operational boundaries of such models and promoting the development of smaller, domain-specific AI tools under clinical supervision may offer a safer alternative[ 24 ]. Models trained on validated medical corpora are more likely to produce clinically appropriate responses. Retrieval-Augmented Generation (RAG) and prompt engineering have been shown to reduce common accuracy issues and incorrect outputs in large language models[ 8 , 9 , 25 , 26 ]. RAG offers a promising approach to mitigate misinformation by grounding AI-generated responses in verified academic sources, such as KDIGO guidelines. By enabling real-time cross-validation with trusted references, RAG systems can enhance the factual accuracy and clinical reliability of AI outputs[ 24 ]. Integrating such targeted models into patient education—under the supervision of healthcare professionals and within regulated frameworks—could significantly reduce the dissemination of inaccurate information. This study provides a foundational step toward the application of RAG-based systems in CKD education and highlights their potential to strengthen the credibility and safety of AI-assisted health communication. Despite its strengths, this study has several limitations. First, the analysis was based on only 15 questions, which may limit the generalizability of the findings. However, these questions were carefully selected from globally relevant search trends and validated sources to ensure maximum thematic representativeness. Second, the moderate level of inter-rater agreement observed in our study (ICC = 0.55) suggests that multidimensional evaluation tools may lead to variability in expert judgments, particularly in the context of AI-generated health content. This highlights the need for more structured and consensus-driven approaches when assessing such outputs. In future studies, methods such as the Delphi technique, which involve multiple rounds of feedback to achieve expert consensus, may be beneficial—especially for evaluating subjective components like 'appropriateness' and 'completeness.' Employing this approach could help clarify evaluation criteria and reduce inter-rater variability, thereby enhancing methodological reliability. Lastly, the AI models evaluated were free, publicly accessible versions, which may not fully reflect the capabilities of their enterprise-grade counterparts. Future studies should expand on these findings by incorporating more diverse and representative question sets, involving broader patient populations, and evaluating newer and clinically fine-tuned AI models. The integration of AI tools with RAG architectures and domain-specific medical corpora should be explored to further enhance the factual accuracy and contextual relevance of patient-facing content. Additionally, the development of standardized, clinically validated prompt templates may offer a practical avenue for routine implementation of AI in patient education settings. Conclusion This study demonstrates that large language models can outperform traditional Guideline-based content in both readability and overall informational quality when answering patient-centered questions about CKD. Among the evaluated tools, ChatGPT-4o mini consistently received higher scores across multiple domains, including completeness and appropriateness. In addition, prompt engineering significantly enhanced the readability of AI-generated responses by lowering linguistic complexity and making content more accessible. The reduction in FKGL by over 3 grade levels underscores prompt engineering’s potential as a low-cost intervention to enhance health accessibility. These findings highlight the growing capacity of well-designed AI tools to support patient education, particularly for populations with limited health literacy. Future studies should validate these results across broader clinical contexts and explore the safe, regulated integration of AI systems into routine health communication. Declarations Funding Declaration: The authors declare that no financial support was received from any organization or funding body for the conduct of this study. Ethics Approval and Consent to Participate This study did not involve human participants, human tissue, or clinical data, and therefore was exempt from ethical review. The study was conducted in accordance with relevant guidelines and regulations, including the Declaration of Helsinki. Ethics approval was not required as per the institutional policies. Consent to Participate: Not applicable. Consent to Publish: Not applicable. Author Contribution L.Z.K.: Conceptualization, Methodology, Data Curation, Writing – Original Draft.S.G.K.: Investigation, Formal Analysis, Writing – Review & Editing.A.I.: Supervision, Project Administration, Writing – Review & Editing.O.C.B.: Software, Visualization, Validation, Writing – Review & Editing.G.K.: Data Acquisition, Literature Review, Writing – Review & Editing.E.V.L.: Critical Review, Methodological Guidance, Writing – Review & Editing.All authors have reviewed and approved the submitted version of the manuscript and agree to be accountable for all aspects of the work. Acknowledgement We would like to express our sincere gratitude to Professor Fettah Fevzi Ersoy, M.D., Professor Kültigin Türkmen, M.D., Professor Faruk Hilmi Turgut, M.D., and Assistant Professor Feyza Bora, M.D. for their invaluable contributions to our study. Their insightful feedback and expertise have greatly enhanced the quality and depth of our research. References Bikbov B, Purcell CA, Levey AS, Smith M, Abdoli A, Abebe M, et al. Global, regional, and national burden of chronic kidney disease, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2020;395:709–33. Francis A, Harhay MN, Ong ACM, Tummalapalli SL, Ortiz A, Fogo AB, et al. Chronic kidney disease and the global public health agenda: an international consensus. Nat Rev Nephrol. 2024;20:473–85. Magadi W, Lightfoot CJ, Memory KE, Santhakumaran S, van der Veer SN, Thomas N, et al. Patient activation and its association with symptom burden and quality of life across the spectrum of chronic kidney disease stages in England. BMC Nephrol. 2022;23:45. Levin A, Ahmed SB, Carrero JJ, Foster B, Francis A, Hall RK, et al. Executive summary of the KDIGO 2024 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease: known knowns and known unknowns. Kidney Int. 2024;105:684–701. Deng J, Lin Y. The benefits and challenges of ChatGPT: An overview. Front Comput Intell Syst. 2022;2:81–3. Barlas T, Altinova AE, Akturk M, Toruner FB. Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines. Int J Obes. 2024;48:271–5. Onder CE, Koc G, Gokbulut P, Taskaldiran I, Kuskonmaz SM. Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy. Sci Rep. 2024;14:243. Patil R, Heston TF, Bhuse V. Prompt Eng Healthc Electron. 2024;13:2961. Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. npj Digit Med. 2023;6:1–6. Sallam M, Barakat M, Sallam M. Pilot Testing of a Tool to Standardize the Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models. Cureus. 2023;15:e49373. Flesch R. A new readability yardstick. J Appl Psychol. 1948;32:221–33. Kincaid J, Fishburne R, Rogers R, Chissom B. Derivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training; 1975. Google. Gemini for Google Workspace Prompting Guide: A quick-start handbook for effective prompts. Google LLC; 2024. Zhang S, Liau ZQG, Tan KLM, Chua WL. Evaluating the accuracy and relevance of ChatGPT responses to frequently asked questions regarding total knee replacement. Knee Surg Relat Res. 2024;36:15. Wang S, Wang Y, Jiang L, Chang Y, zhang S, Zhao K, et al. Assessing the clinical support capabilities of ChatGPT 4o and ChatGPT 4o mini in managing lumbar disc herniation. Eur J Med Res. 2025;30:45. Acharya PC, Alba R, Krisanapan P, Acharya CM, Suppadungsuk S, Csongradi E, et al. AI-Driven Patient Education in Chronic Kidney Disease: Evaluating Chatbot Responses against Clinical Guidelines. Diseases. 2024;12:185. Masters K. Medical Teacher’s first ChatGPT’s referencing hallucinations: Lessons for editors, reviewers, and teachers. Med Teach. 2023;45:673–5. Aïmeur E, Amri S, Brassard G. Fake news, disinformation and misinformation in social media: a review. Soc Netw Anal Min. 2023;13:30. Monteith S, Glenn T, Geddes JR, Whybrow PC, Achtyes E, Bauer M. Artificial intelligence and increasing misinformation. Br J Psychiatry. 2024;224:33–5. Suarez-Lledo V, Alvarez-Galvez J. Prevalence of Health Misinformation on Social Media: Systematic Review. J Med Internet Res. 2021;23:e17187. Vicari R, Komendatova N. Systematic meta-analysis of research on AI tools to deal with misinformation on social media during natural and anthropogenic hazards and disasters. Humanit Soc Sci Commun. 2023;10:1–14. Ozbay FA, Alatas B. Fake news detection within online social media using supervised artificial intelligence algorithms. Physica A. 2020;540:123174. Jahanbakhsh F, Katsis Y, Wang D, Popa L, Muller M. Exploring the Use of Personalized AI for Identifying Misinformation on Social Media. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery; 2023. pp. 1–27. Han J, Qiu W, Lichtfouse E. ChatGPT in Scientific Research and Writing: A Beginner’s Guide. In: Han J, Qiu W, Lichtfouse E, editors. ChatGPT in Scientific Research and Writing: A Beginner’s Guide. Cham: Springer Nature Switzerland; 2024. pp. 1–109. Meskó B. Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial. J Med Internet Res. 2023;25:e50638. Miao J, Thongprayoon C, Suppadungsuk S, Garcia Valencia OA, Cheungpasitporn W. Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications. Medicina. 2024;60:445. Additional Declarations No competing interests reported. Supplementary Files supplementarymaterial.docx Cite Share Download PDF Status: Published Journal Publication published 20 Feb, 2026 Read the published version in BMC Nephrology → Version 1 posted Editorial decision: Revision requested 10 Dec, 2025 Reviews received at journal 09 Dec, 2025 Reviewers agreed at journal 26 Nov, 2025 Reviews received at journal 23 Sep, 2025 Reviewers agreed at journal 14 Sep, 2025 Reviewers invited by journal 11 Sep, 2025 Editor assigned by journal 03 Sep, 2025 Submission checks completed at journal 03 Sep, 2025 First submitted to journal 01 Sep, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7507934","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":515418834,"identity":"5e7825df-68c5-4b4d-bba0-80e5c7f4842c","order_by":0,"name":"Lutfullah Zahit Koc","email":"","orcid":"","institution":"Antalya Eğitim ve Araştırma Hastanesi","correspondingAuthor":false,"prefix":"","firstName":"Lutfullah","middleName":"Zahit","lastName":"Koc","suffix":""},{"id":515418835,"identity":"8be63b0d-0ad1-4da1-bc0c-ca18fbf0aaaf","order_by":1,"name":"Sevgi Gulsen Koc","email":"","orcid":"","institution":"Antalya Şehir Hastanesi","correspondingAuthor":false,"prefix":"","firstName":"Sevgi","middleName":"Gulsen","lastName":"Koc","suffix":""},{"id":515418836,"identity":"5986f82a-fdbf-4fc7-9e45-e16e796a4d92","order_by":2,"name":"Ayca Inci","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA20lEQVRIiWNgGAWjYNACAwYGNvYGNhCTsYF4LTwHYFqYibVJIoFILfwzkh9+YCi4I88n+fzZYx4GG9kNB/iPfcBr9o00YwkGg2eGbdI55sY8DGnGGw4wM8/Aa82NHAaglsMJbNI5bNI8DIcTQVrw6pC/kcP8A6xF8vgzoJb/hLUY3Mhhg9giwWAG1HKAsBbDM8/MLBIMDhu28eSYG84xSDaeeZjZGK8WuePJj298+HNYXr79+LMHbyrsZPuONz7Gq4VBIIGBIQHhTiAmGJP8BwipGAWjYBSMghEPAN51QY2NxquUAAAAAElFTkSuQmCC","orcid":"","institution":"Antalya Eğitim ve Araştırma Hastanesi","correspondingAuthor":true,"prefix":"","firstName":"Ayca","middleName":"","lastName":"Inci","suffix":""},{"id":515418837,"identity":"1d6d4f5b-4a94-49f6-a9ae-01d0ab77eb2e","order_by":3,"name":"Osman Cagın Buldukoglu","email":"","orcid":"","institution":"Antalya Eğitim ve Araştırma Hastanesi","correspondingAuthor":false,"prefix":"","firstName":"Osman","middleName":"Cagın","lastName":"Buldukoglu","suffix":""},{"id":515418838,"identity":"eea04741-29f3-4e30-834c-60381fce5556","order_by":4,"name":"Gokhan Koker","email":"","orcid":"","institution":"Antalya Eğitim ve Araştırma Hastanesi","correspondingAuthor":false,"prefix":"","firstName":"Gokhan","middleName":"","lastName":"Koker","suffix":""},{"id":515418839,"identity":"c32c9699-ab91-4b16-be9e-ae7dcf03143c","order_by":5,"name":"Edgar V. Lerma","email":"","orcid":"","institution":"University of Illinois at Chicago","correspondingAuthor":false,"prefix":"","firstName":"Edgar","middleName":"V.","lastName":"Lerma","suffix":""}],"badges":[],"createdAt":"2025-09-01 11:38:21","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7507934/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7507934/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12882-026-04814-3","type":"published","date":"2026-02-20T15:59:05+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":91836157,"identity":"e398493b-fb21-4ccb-81c1-860812251f7b","added_by":"auto","created_at":"2025-09-22 09:37:02","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":384405,"visible":true,"origin":"","legend":"","description":"","filename":"AICKD21.docx","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/406c780b4329fa5099edb997.docx"},{"id":91836152,"identity":"5be0c2b2-bdc5-45f0-9a3b-7c1da2f4a46f","added_by":"auto","created_at":"2025-09-22 09:37:02","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8087,"visible":true,"origin":"","legend":"","description":"","filename":"6fbd0629e8764f9eb499bd86643220e7.json","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/97763ffa418046f030ab584b.json"},{"id":91836155,"identity":"8cb921a0-b985-4ea9-918d-da3475cc55ce","added_by":"auto","created_at":"2025-09-22 09:37:02","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":19039,"visible":true,"origin":"","legend":"","description":"","filename":"supplementarymaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/73c8d52bf7fb59654ca9ee2b.docx"},{"id":91836161,"identity":"1f455dbc-623d-4b58-9540-ed36211e17df","added_by":"auto","created_at":"2025-09-22 09:37:02","extension":"xml","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":84189,"visible":true,"origin":"","legend":"","description":"","filename":"6fbd0629e8764f9eb499bd86643220e71enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/e5f533a87ebee1c55ef48977.xml"},{"id":91835040,"identity":"3c9bd43f-3c80-487d-88dd-db8f46ae72df","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":62753,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/d1150d3948358196fc3cea4a.png"},{"id":91838429,"identity":"5dacc627-b107-42ec-99d3-8dfa9786f2f5","added_by":"auto","created_at":"2025-09-22 09:45:02","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":91192,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/7a2d8dfae64df2b14c435fe8.png"},{"id":91838431,"identity":"740c845a-de79-4c5b-a0a2-6aefcb3ea1ac","added_by":"auto","created_at":"2025-09-22 09:45:02","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":146764,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/aaec9c47063b8bc6389723f7.png"},{"id":91835046,"identity":"a70f6c08-2f4e-443b-b894-ef12ee320989","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21441,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/fab8c9fa275b1b05152c1ee1.png"},{"id":91835043,"identity":"ab435fb6-acc2-42a9-a5cf-e25f401ea4ba","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40949,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/20870d3226bb943cd9392017.png"},{"id":91835042,"identity":"6777ed55-b665-4495-92e8-7fdc6f896f5f","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":28095,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/1790b3e48ae3744f6c38ac72.png"},{"id":91835047,"identity":"b4ddd1fc-8ffc-4bda-9e26-83288901a0fc","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"xml","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":79390,"visible":true,"origin":"","legend":"","description":"","filename":"6fbd0629e8764f9eb499bd86643220e71structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/6207681a61f90a9ae547adc5.xml"},{"id":91835050,"identity":"ef436dbf-35d2-47c6-9564-a25969761dce","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"html","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":92804,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/d89f4281179a7f78bf8fc26a.html"},{"id":91835033,"identity":"81e68e21-d3b5-4d4a-b67c-209d4f7fa7d4","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":25957,"visible":true,"origin":"","legend":"\u003cp\u003eStudy design and methodological workflow.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/272930526e5f1b5201323c43.png"},{"id":91839697,"identity":"e9f77099-2a00-4fbf-ad5c-94042a8cf41b","added_by":"auto","created_at":"2025-09-22 09:53:02","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":62753,"visible":true,"origin":"","legend":"\u003cp\u003eBox plot of total CLEAR scores for guideline, ChatGPT-4o mini, and Gemini responses.\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/cf2ec9a4e3db63bce377428c.png"},{"id":91838427,"identity":"87e4c585-1746-468c-85fc-40f63a723a26","added_by":"auto","created_at":"2025-09-22 09:45:02","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":91192,"visible":true,"origin":"","legend":"\u003cp\u003eFKGL scores for guideline, ChatGPT-4o mini, and Gemini responses without prompt.\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/01db14b03c4538ef782d5b6f.png"},{"id":91836158,"identity":"a66c5a39-abdc-4ccb-8a0a-710d364ae966","added_by":"auto","created_at":"2025-09-22 09:37:02","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":146764,"visible":true,"origin":"","legend":"\u003cp\u003eEffect of prompting on FKGL and FRE scores across AI-generated responses.\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/6c98c373757a8132dbc21f8b.png"},{"id":103252218,"identity":"511e0ae1-8993-4601-9774-9900473998ab","added_by":"auto","created_at":"2026-02-23 16:13:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":817856,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/6efcda8a-4d95-4570-b321-9adb94e61a6f.pdf"},{"id":91835035,"identity":"9955ca7a-97ef-44cd-851f-3434ed0eb82c","added_by":"auto","created_at":"2025-09-22 09:29:02","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":19039,"visible":true,"origin":"","legend":"","description":"","filename":"supplementarymaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-7507934/v1/eba99f6f472c4513700e9fe0.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Guidelines vs Generative AI in CKD Patient Education: The Role of Prompt Engineering and Expert Blinded Evaluation","fulltext":[{"header":"Introduction","content":"\u003cp\u003eChronic kidney disease is an increasing global public health issue, affecting approximately 700\u0026nbsp;million people worldwide. The prevalence of CKD has reached 9.1% globally, and the mortality rate associated with this disease across all age groups increased by 41.5% from 1990 to 2017[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Known as the \"silent epidemic,\" this disease is often diagnosed at advanced stages due to its asymptomatic nature in early phases and low awareness levels[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Furthermore, low socioeconomic status and limited health literacy accelerate the progression of the disease, increasing the risk of progression to end-stage renal disease[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Enhancing health literacy plays a crucial role in slowing disease progression by enabling patients to gain knowledge about CKD. Creating awareness in the early stages and implementing appropriate treatment strategies not only improve quality of life and reduce mortality and morbidity rates but also decrease the need for high-cost treatments such as dialysis[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eIn recent years, artificial intelligence (AI) technologies have introduced a new paradigm for enhancing health literacy and disease management. The rapid advancements in AI technologies hold significant potential to improve accessibility to healthcare services and democratize access to information[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Generative AI models, particularly large language models such as ChatGPT-4o mini (by OpenAI, San Francisco, CA) and Gemini (by Google, Mountain View, CA), have become easily accessible to a broad range of users. These models have the potential to facilitate patients' access to health information, providing personalized education and guidance[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. However, further research is needed to evaluate the reliability and accuracy of these technologies.\u003c/p\u003e\u003cp\u003eStrategies have been developed to enhance the accuracy of AI models in the medical field. One of the most commonly used strategies is prompt engineering, which aims to ensure that AI systems generate accurate, relevant, and reliable outputs. Prompt engineering involves designing and optimizing input instructions to Guideline AI models toward producing specific outputs. An optimal prompt ensures that the generated response contains accurate and user-appropriate information[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. For example: \"I am a patient with chronic kidney disease, and I have some questions about my condition. I would like to ask you these questions as if you are a nephrologist. Could you answer them in a way that an elementary school graduate can easily understand?\" By explicitly stating the user's condition (patient with chronic kidney disease), assigning a role (as a nephrologist), and tailoring the response to the user's educational level (elementary school graduate), the likelihood of receiving accurate and relevant answers is increased.\u003c/p\u003e\u003cp\u003ePrevious studies have demonstrated the potential of AI-supported tools in diverse healthcare contexts, such as providing personalized dietary and exercise recommendations for obesity and delivering reliable, easy-to-understand information for conditions like hypothyroidism during pregnancy[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Building on these findings, this study is the first to systematically blinded compare AI-generated responses with guideline-based answers in the context of CKD. By integrating prompt engineering techniques, we aimed to improve both the readability and content accuracy of AI-generated educational materials.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003eThis study was designed as a cross-sectional, comparative analysis to evaluate the accuracy, content quality, and readability of responses to selected patient questions about CKD. The overall methodology is summarized in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eFifteen representative questions were selected based on two-year worldwide Google Trends data, focusing on globally searched patient queries related to CKD. This approach ensured thematic relevance and captured real-world informational needs observed in both clinical and digital settings. All questions are listed in Supplementary Table\u0026nbsp;1.\u003c/p\u003e\u003cp\u003eTo establish a benchmark, guideline-based reference answers were compiled from trusted nephrology sources, forming the \"Guideline group.\" The same set of questions was then posed to both AI models: ChatGPT-4o mini and Gemini 1.5 Flash. To enhance reliability and reduce variability, each question was submitted in two independent sessions per model, and all responses were systematically documented for further evaluation.\u003c/p\u003e\u003cp\u003eAll responses\u0026mdash;whether guideline-based or AI-generated\u0026mdash;were anonymized and formatted in a standardized style. Four independent nephrology professors, each with over 15 years of experience, evaluated the responses in a blinded manner using the CLEAR Tool scoring system. Evaluators were unaware of the origin of each response and scored them across five subdomains: completeness, lack of false information, evidence-based accuracy, appropriateness, and relevance. This blinded design was implemented to reduce potential bias and ensure objective assessment.\u003c/p\u003e\u003cp\u003eThe CLEAR TOOL was developed based on a literature review of existing health information quality assessment frameworks, including DISCERN, PEMAT, and the CDC Clear Communication Index. While not directly adapted from a single tool, CLEAR incorporates conceptual elements such as completeness, evidence, and clarity that are shared with these validated instruments.[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Full scoring rubric for the CLEAR TOOL is provided in Supplementary Table\u0026nbsp;2.\u003c/p\u003e\u003cp\u003eIn addition to content quality, the readability of each response was assessed using two validated linguistic metrics: the Flesch-Kincaid Grade Level (FKGL) and the Flesch Reading Ease (FRE)[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. These indices account for sentence length, word count, and syllable density. FKGL estimates the U.S. school grade level required to comprehend a text, while FRE assigns a score from 0 to 100, with higher scores indicating greater readability. This dual approach enabled a nuanced analysis of how accessible each response was to the average patient.\u003c/p\u003e\u003cp\u003eTo evaluate the impact of prompt engineering, the same 15 questions were re-submitted to both AI models under two conditions: without prompt and with a standardized instructional prompt. The prompt was designed based on best practices from the Gemini for Google Workspace Prompting Guide and included four key elements: persona (\"a nephrologist\"), task (\"answer questions about chronic kidney disease\"), context (\"from the perspective of a patient with CKD\"), and format (\"in simplified language understandable by an elementary school graduate\")[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. The final prompt was: \u0026ldquo;I am a patient with chronic kidney disease, and I have some questions about my condition. Please answer them as if you are a nephrologist, but in a way that an elementary school graduate can easily understand.\u0026rdquo;\u003c/p\u003e\u003cp\u003eThis prompt was applied identically to both models. Each prompt-based query was initiated in a separate private browser session to prevent memory retention and ensure independence between responses. All answers were collected under consistent conditions using the default temperature and token settings of the public versions of ChatGPT-4o mini and Gemini, as of March 2025.\u003c/p\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eStatistical Analysis\u003c/h2\u003e\u003cp\u003eDescriptive statistics were calculated. Data normality was assessed using Kolmogorov\u0026ndash;Smirnov and Shapiro\u0026ndash;Wilk tests. For data following a normal distribution, descriptive statistics were expressed as mean and standard deviation, and group comparisons were assessed using ANOVA, with Tukey's test applied for post hoc analysis. For data not following a normal distribution, descriptive statistics were presented as median values and interquartile ranges, and differences between groups were evaluated with the Kruskal-Wallis test. Post hoc analyses were conducted using the Mann-Whitney U test with Bonferroni correction. The evaluations of the professors regarding the responses were analyzed based on the total scores assigned for each question, using Intraclass Correlation to assess agreement.\u003c/p\u003e\u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eThe Intraclass Correlation Coefficient (ICC) for the total scores was calculated to assess agreement among evaluators. The ICC value was 0.55, indicating moderate agreement among the evaluators.\u003c/p\u003e\u003cp\u003eThe groups were evaluated by comparing the 'total score,' derived from the sum of the CLEAR TOOL components, as well as each component individually. The median and interquartile range (IQR) of 'total scores' for each group were as follows: Guideline: Median\u0026thinsp;=\u0026thinsp;13.0 (IQR\u0026thinsp;=\u0026thinsp;2.0), ChatGPT-4o mini: Median\u0026thinsp;=\u0026thinsp;21.0 (IQR\u0026thinsp;=\u0026thinsp;5.0), Gemini: Median\u0026thinsp;=\u0026thinsp;17.0 (IQR\u0026thinsp;=\u0026thinsp;5.0)\u003c/p\u003e\u003cp\u003eThe Kruskal-Wallis test revealed a statistically significant difference in total scores between the three groups (H\u0026thinsp;=\u0026thinsp;103.49, p\u0026thinsp;\u0026lt;\u0026thinsp;0.001) (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Pairwise comparisons of the total scores using the Mann-Whitney U test with Bonferroni correction showed the following: Guideline vs. ChatGPT-4o mini: Median\u0026thinsp;=\u0026thinsp;13.0 (IQR\u0026thinsp;=\u0026thinsp;2.0) vs. 21.0 (IQR\u0026thinsp;=\u0026thinsp;5.0), p\u0026thinsp;\u0026lt;\u0026thinsp;0.001, Guideline vs. Gemini: Median\u0026thinsp;=\u0026thinsp;13.0 (IQR\u0026thinsp;=\u0026thinsp;2.0) vs. 17.0 (IQR\u0026thinsp;=\u0026thinsp;5.0), p\u0026thinsp;\u0026lt;\u0026thinsp;0.001, ChatGPT-4o mini vs. Gemini: Median\u0026thinsp;=\u0026thinsp;21.0 (IQR\u0026thinsp;=\u0026thinsp;5.0) vs. 17.0 (IQR\u0026thinsp;=\u0026thinsp;5.0), p\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eWhen comparing the CLEAR TOOL components across the three groups, statistically significant differences were observed for all components. Post hoc analyses with Bonferroni correction (p\u0026thinsp;=\u0026thinsp;0.05/3) revealed that both AI models (ChatGPT-4o mini and Gemini) differed significantly from the guideline group across all domains. However, differences between ChatGPT-4o mini and Gemini were not statistically significant in all components. (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eMedian and IQR scores for each CLEAR TOOL criterion across guideline, ChatGPT-4o mini, and Gemini groups. (Note: p\u003csub\u003e1\u003c/sub\u003e: Guideline-ChatGPT-4o mini, p\u003csub\u003e2\u003c/sub\u003e: Guideline-Gemini, p\u003csub\u003e3\u003c/sub\u003e: ChatGPT-4o mini-Gemini. Bolded comparisons indicate statistically significant differences after Bonferroni correction.)\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"5\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e\u003cp\u003ePost Hoc Analysis\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGuideline Median (IQR)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eChatGPT-4o mini Median (IQR)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eGemini Median (IQR)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eP value\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCompleteness\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e2 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e5 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e4 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e1\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e2\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e3\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eLack of False\u003c/p\u003e\u003cp\u003eKnowledge\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e4 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e1\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e2\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003ep\u003csub\u003e3\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0,613\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eEvidence Based\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e1\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e2\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e3\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAppropriateness\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4 (IQR: 1.25)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e1\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003ep\u003csub\u003e2\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0,503\u003c/p\u003e\u003cp\u003ep\u003csub\u003e3\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0,455\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eRelevance\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e4 (IQR: 2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e3 (IQR: 1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003ep\u003c/b\u003e\u003csub\u003e\u003cb\u003e1\u003c/b\u003e\u003c/sub\u003e\u0026thinsp;\u003cb\u003e\u0026lt;\u0026thinsp;0.001\u003c/b\u003e\u003c/p\u003e\u003cp\u003ep\u003csub\u003e2\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0,293\u003c/p\u003e\u003cp\u003ep\u003csub\u003e3\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0,822\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\n\u003ch3\u003eReadability\u003c/h3\u003e\n\u003cp\u003eReadability levels were evaluated in two ways. First, questions were directly posed to AI tools (Without Prompt), and the FKGL and FRE scores of their responses, along with Guideline answers, were calculated (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eReadability scores (mean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD) of FKGL and FRE for guideline, ChatGPT-4o mini, and Gemini responses, with and without prompt use\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"5\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\"\u0026plusmn;\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e\u003cp\u003eGroup\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"2\" nameend=\"c3\" namest=\"c2\"\u003e\u003cp\u003eWithout Prompt\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u003cp\u003ePrompt\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003e\u003cb\u003eFKGL Mean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD\u003c/b\u003e\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003e\u003cb\u003eFRE Mean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD\u003c/b\u003e\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003e\u003cb\u003eFKGL Mean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD\u003c/b\u003e\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e\u003cp\u003e\u003cb\u003eFRE Mean\u0026thinsp;\u0026plusmn;\u0026thinsp;SD\u003c/b\u003e\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGuideline\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e\u003cp\u003e9.40\u0026thinsp;\u0026plusmn;\u0026thinsp;2.29\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e\u003cp\u003e52.01\u0026thinsp;\u0026plusmn;\u0026thinsp;14.85\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eChatGPT-4o mini\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e\u003cp\u003e11.34\u0026thinsp;\u0026plusmn;\u0026thinsp;1.79\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e\u003cp\u003e36.17\u0026thinsp;\u0026plusmn;\u0026thinsp;10.85\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e\u003cp\u003e7.87\u0026thinsp;\u0026plusmn;\u0026thinsp;1.13\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e\u003cp\u003e64.23\u0026thinsp;\u0026plusmn;\u0026thinsp;6.16\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGemini\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c2\"\u003e\u003cp\u003e9.62\u0026thinsp;\u0026plusmn;\u0026thinsp;1.55\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c3\"\u003e\u003cp\u003e46.36\u0026thinsp;\u0026plusmn;\u0026thinsp;10.69\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c4\"\u003e\u003cp\u003e7.13\u0026thinsp;\u0026plusmn;\u0026thinsp;1.78\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\"\u0026plusmn;\" colname=\"c5\"\u003e\u003cp\u003e61.45\u0026thinsp;\u0026plusmn;\u0026thinsp;12.40\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eFor the Without Prompt group, the Flesch\u0026ndash;Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) values for the Guideline, ChatGPT-4o mini, and Gemini groups were calculated as follows: Guideline: FKGL mean\u0026thinsp;=\u0026thinsp;9.40\u0026thinsp;\u0026plusmn;\u0026thinsp;2.29 (SD), FRE mean\u0026thinsp;=\u0026thinsp;52.01\u0026thinsp;\u0026plusmn;\u0026thinsp;14.85 (SD), ChatGPT-4o mini: FKGL mean\u0026thinsp;=\u0026thinsp;11.34\u0026thinsp;\u0026plusmn;\u0026thinsp;1.79 (SD), FRE mean\u0026thinsp;=\u0026thinsp;36.17\u0026thinsp;\u0026plusmn;\u0026thinsp;10.85 (SD), Gemini: FKGL mean\u0026thinsp;=\u0026thinsp;9.62\u0026thinsp;\u0026plusmn;\u0026thinsp;1.55 (SD), FRE mean\u0026thinsp;=\u0026thinsp;46.36\u0026thinsp;\u0026plusmn;\u0026thinsp;10.69 (SD)\u003c/p\u003e\u003cp\u003eStatistical analyses revealed a significant difference in FKGL between ChatGPT-4o mini and Guideline (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001), indicating that ChatGPT-4o mini\u0026rsquo;s responses required a higher educational level. Additionally, a significant difference was found between ChatGPT-4o mini and Gemini in FKGL scores (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001).\u003c/p\u003e\u003cp\u003eFor FRE scores, significant differences were observed between ChatGPT-4o mini and Guideline (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001) as well as between ChatGPT-4o mini and Gemini (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). ChatGPT-4o mini\u0026rsquo;s responses exhibited significantly lower readability scores, indicating greater difficulty in comprehension.\u003c/p\u003e\u003cp\u003eGuideline group texts were generally at a high school level (9th grade) and moderately easy to read. In contrast, ChatGPT-4o mini\u0026rsquo;s responses were at the level of a high school senior and posed greater comprehension challenges. Gemini\u0026rsquo;s texts were positioned between Guideline and ChatGPT-4o mini, requiring an educational level equivalent to the beginning of high school (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eWhen a prompt was provided, these readability metrics for the ChatGPT-4o mini and Gemini groups were calculated as follows (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e): ChatGPT-4o mini: FKGL mean\u0026thinsp;=\u0026thinsp;7.87\u0026thinsp;\u0026plusmn;\u0026thinsp;1.13 (SD), FRE mean\u0026thinsp;=\u0026thinsp;64.23\u0026thinsp;\u0026plusmn;\u0026thinsp;6.16 (SD), Gemini: FKGL mean\u0026thinsp;=\u0026thinsp;7.13\u0026thinsp;\u0026plusmn;\u0026thinsp;1.78 (SD), FRE mean\u0026thinsp;=\u0026thinsp;61.45\u0026thinsp;\u0026plusmn;\u0026thinsp;12.40 (SD)\u003c/p\u003e\u003cp\u003eStatistical analyses revealed a significant difference in both FKGL and FRE scores between the Prompt and Without Prompt groups for both ChatGPT-4o mini and Gemini (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). Providing a prompt made the responses of both models significantly easier to read and reduced the required educational level to below high school (7th grade level).\u003c/p\u003e\u003cp\u003eWith the use of a prompt, ChatGPT-4o mini's responses became closer to the readability and comprehensibility levels of the Guideline group in terms of ease of understanding and educational level. Compared to Without Prompt, the use of a prompt significantly improved the readability of responses and reduced their complexity for both ChatGPT-4o mini and Gemini (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eStatistical analyses confirmed these improvements: the FKGL and FRE scores of both models showed significant differences between Prompt and Without Prompt conditions (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001), based on Mann-Whitney U tests comparing paired responses across all 15 questions. These findings indicate that prompt engineering is an effective strategy to reduce linguistic complexity and enhance patient accessibility in AI-generated health content.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study is among the first known blinded comparative evaluations of AI-generated versus guideline-based responses in the context of CKD. It incorporates several methodological strengths that distinguish it from previous research. First, the evaluation was conducted by four independent nephrologists, a higher number of expert raters than typically employed in similar studies, thereby improving the reliability and robustness of the results. Second, all responses\u0026mdash;whether generated by AI or sourced from clinical guidelines\u0026mdash;were evaluated under blinded conditions, minimizing potential bias. Unlike prior studies that often assess AI outputs against fixed references, this study applied a fully anonymized and independent evaluation protocol. Third, the study uniquely assessed the impact of prompt engineering on the readability and comprehensibility of AI outputs, revealing significant improvements. Collectively, these contributions enhance the scientific rigor of the study and underscore its relevance for advancing AI-supported patient education in nephrology.\u003c/p\u003e\u003cp\u003eThe results of this study align with existing literature. For instance, Zhang et al. demonstrated that ChatGPT-4o mini achieved 88% accuracy in providing accurate and relevant information on total knee replacement, underscoring its effectiveness in patient education[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Onder et al. found ChatGPT-4o mini-4's responses reliable in hypothyroidism management, but readability analyses revealed a requirement for university-level education[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Similarly, Wang et al. evaluated ChatGPT-4o mini-4 and 4o-mini for clinical support in lumbar disc herniation and reported accuracy and completeness scores exceeding 75%; however, the responses were deemed \u0026ldquo;very difficult to read\u0026rdquo;[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eAcharya et al. assessed 15 lifestyle and 20 dietary questions from KDIGO and KDOQI guidelines answered by ChatGPT-4o mini-3.5, ChatGPT-4o mini-4, Gemini AI, and Bing AI. Responses were evaluated by nephrologists for accuracy. While the answers were generally accurate, misleading statements and irrelevant references were noted, particularly in ChatGPT-4o mini-3.5, ChatGPT-4o mini-4, and Gemini. All models delivered responses at a high school readability level, highlighting potential accessibility limitations for patients with low health literacy[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. The tendency to generate inaccurate, out-of-context responses and provide incorrect references has been documented in other studies as well[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eIn our study, the readability and comprehensibility of text generated by AI models significantly improved following prompt-guided responses compared to pre-prompt outputs. The reduction of FKGL to the 7th-grade level highlights the potential of this technology to effectively provide information to a broader patient population. By making responses more understandable, prompt engineering can facilitate access to information for individuals with low health literacy. This approach may serve as an effective tool for designing personalized patient education materials, thereby enhancing health literacy. Within this context, our study takes a step toward assessing the reliability of responses generated by large language models such as ChatGPT-4o mini and Gemini in the healthcare domain. Beyond evaluating the accuracy and trustworthiness of these systems, the application of prompt engineering techniques has been found to enhance the readability and comprehensibility of AI-generated outputs. This contributes to producing more reliable and accessible content to counter misinformation. Notably, prompt engineering can serve as a valuable tool in improving AI-generated responses, making them more accurate and user-friendly, thereby reducing the spread of misinformation in healthcare.\u003c/p\u003e\u003cp\u003eWith the widespread use of social media, the speed and scale at which misinformation spreads have greatly increased. A significant portion of fake news is crafted to be engaging and emotionally impactful, leading to higher interaction rates[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. This, in turn, facilitates the increased use and rapid dissemination of false information across social media platforms. Such developments pose serious risks, especially in the healthcare sector. AI-generated content often relies on statistical probabilities, which may lead to misleading or out-of-context information lacking proper source validation[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. While artificial intelligence can accelerate the spread of misinformation, it also holds significant potential for detecting and mitigating false information[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Various strategies have been implemented to combat misinformation and disinformation. Primarily, AI-powered misinformation detection systems on social media platforms use natural language processing and machine learning techniques to identify inaccurate content[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. However, research suggests that these systems should not only detect falsehoods but also encourage users to critically evaluate the credibility of the information they encounter[\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. In this regard, enhancing health literacy, raising public awareness, and fostering critical thinking through targeted educational initiatives are crucial. Additionally, it is essential for healthcare professionals and institutions to actively use social media to disseminate accurate information and counteract misinformation[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. On the other hand, relying on large language models that utilize vast and unregulated datasets increases the risk of misinformation and hallucinations. To mitigate this, defining the operational boundaries of such models and promoting the development of smaller, domain-specific AI tools under clinical supervision may offer a safer alternative[\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. Models trained on validated medical corpora are more likely to produce clinically appropriate responses. Retrieval-Augmented Generation (RAG) and prompt engineering have been shown to reduce common accuracy issues and incorrect outputs in large language models[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eRAG offers a promising approach to mitigate misinformation by grounding AI-generated responses in verified academic sources, such as KDIGO guidelines. By enabling real-time cross-validation with trusted references, RAG systems can enhance the factual accuracy and clinical reliability of AI outputs[\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]. Integrating such targeted models into patient education\u0026mdash;under the supervision of healthcare professionals and within regulated frameworks\u0026mdash;could significantly reduce the dissemination of inaccurate information. This study provides a foundational step toward the application of RAG-based systems in CKD education and highlights their potential to strengthen the credibility and safety of AI-assisted health communication.\u003c/p\u003e\u003cp\u003eDespite its strengths, this study has several limitations. First, the analysis was based on only 15 questions, which may limit the generalizability of the findings. However, these questions were carefully selected from globally relevant search trends and validated sources to ensure maximum thematic representativeness. Second, the moderate level of inter-rater agreement observed in our study (ICC\u0026thinsp;=\u0026thinsp;0.55) suggests that multidimensional evaluation tools may lead to variability in expert judgments, particularly in the context of AI-generated health content. This highlights the need for more structured and consensus-driven approaches when assessing such outputs. In future studies, methods such as the Delphi technique, which involve multiple rounds of feedback to achieve expert consensus, may be beneficial\u0026mdash;especially for evaluating subjective components like 'appropriateness' and 'completeness.' Employing this approach could help clarify evaluation criteria and reduce inter-rater variability, thereby enhancing methodological reliability. Lastly, the AI models evaluated were free, publicly accessible versions, which may not fully reflect the capabilities of their enterprise-grade counterparts.\u003c/p\u003e\u003cp\u003eFuture studies should expand on these findings by incorporating more diverse and representative question sets, involving broader patient populations, and evaluating newer and clinically fine-tuned AI models. The integration of AI tools with RAG architectures and domain-specific medical corpora should be explored to further enhance the factual accuracy and contextual relevance of patient-facing content. Additionally, the development of standardized, clinically validated prompt templates may offer a practical avenue for routine implementation of AI in patient education settings.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study demonstrates that large language models can outperform traditional Guideline-based content in both readability and overall informational quality when answering patient-centered questions about CKD. Among the evaluated tools, ChatGPT-4o mini consistently received higher scores across multiple domains, including completeness and appropriateness.\u003c/p\u003e\u003cp\u003eIn addition, prompt engineering significantly enhanced the readability of AI-generated responses by lowering linguistic complexity and making content more accessible. The reduction in FKGL by over 3 grade levels underscores prompt engineering\u0026rsquo;s potential as a low-cost intervention to enhance health accessibility.\u003c/p\u003e\u003cp\u003eThese findings highlight the growing capacity of well-designed AI tools to support patient education, particularly for populations with limited health literacy. Future studies should validate these results across broader clinical contexts and explore the safe, regulated integration of AI systems into routine health communication.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding Declaration:\u0026nbsp;\u003c/strong\u003eThe authors declare that no financial support was received from any organization or funding body for the conduct of this study.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics Approval and Consent to Participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study did not involve human participants, human tissue, or clinical data, and therefore was exempt from ethical review. The study was conducted in accordance with relevant guidelines and regulations, including the Declaration of Helsinki. Ethics approval was not required as per the institutional policies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to Participate:\u0026nbsp;\u003c/strong\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to Publish:\u0026nbsp;\u003c/strong\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Contribution\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eL.Z.K.: Conceptualization, Methodology, Data Curation, Writing \u0026ndash; Original Draft.S.G.K.: Investigation, Formal Analysis, Writing \u0026ndash; Review \u0026amp; Editing.A.I.: Supervision, Project Administration, Writing \u0026ndash; Review \u0026amp; Editing.O.C.B.: Software, Visualization, Validation, Writing \u0026ndash; Review \u0026amp; Editing.G.K.: Data Acquisition, Literature Review, Writing \u0026ndash; Review \u0026amp; Editing.E.V.L.: Critical Review, Methodological Guidance, Writing \u0026ndash; Review \u0026amp; Editing.All authors have reviewed and approved the submitted version of the manuscript and agree to be accountable for all aspects of the work.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe would like to express our sincere gratitude to Professor Fettah Fevzi Ersoy, M.D., Professor K\u0026uuml;ltigin T\u0026uuml;rkmen, M.D., Professor Faruk Hilmi Turgut, M.D., and Assistant Professor Feyza Bora, M.D. for their invaluable contributions to our study. Their insightful feedback and expertise have greatly enhanced the quality and depth of our research.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eBikbov B, Purcell CA, Levey AS, Smith M, Abdoli A, Abebe M, et al. Global, regional, and national burden of chronic kidney disease, 1990\u0026ndash;2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2020;395:709\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFrancis A, Harhay MN, Ong ACM, Tummalapalli SL, Ortiz A, Fogo AB, et al. Chronic kidney disease and the global public health agenda: an international consensus. Nat Rev Nephrol. 2024;20:473\u0026ndash;85.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMagadi W, Lightfoot CJ, Memory KE, Santhakumaran S, van der Veer SN, Thomas N, et al. Patient activation and its association with symptom burden and quality of life across the spectrum of chronic kidney disease stages in England. BMC Nephrol. 2022;23:45.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLevin A, Ahmed SB, Carrero JJ, Foster B, Francis A, Hall RK, et al. Executive summary of the KDIGO 2024 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease: known knowns and known unknowns. Kidney Int. 2024;105:684\u0026ndash;701.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDeng J, Lin Y. The benefits and challenges of ChatGPT: An overview. Front Comput Intell Syst. 2022;2:81\u0026ndash;3.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarlas T, Altinova AE, Akturk M, Toruner FB. Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines. Int J Obes. 2024;48:271\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOnder CE, Koc G, Gokbulut P, Taskaldiran I, Kuskonmaz SM. Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy. Sci Rep. 2024;14:243.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePatil R, Heston TF, Bhuse V. Prompt Eng Healthc Electron. 2024;13:2961.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMesk\u0026oacute; B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. npj Digit Med. 2023;6:1\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSallam M, Barakat M, Sallam M. Pilot Testing of a Tool to Standardize the Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models. Cureus. 2023;15:e49373.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFlesch R. A new readability yardstick. J Appl Psychol. 1948;32:221\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKincaid J, Fishburne R, Rogers R, Chissom B. Derivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training; 1975.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGoogle. Gemini for Google Workspace Prompting Guide: A quick-start handbook for effective prompts. Google LLC; 2024.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eZhang S, Liau ZQG, Tan KLM, Chua WL. Evaluating the accuracy and relevance of ChatGPT responses to frequently asked questions regarding total knee replacement. Knee Surg Relat Res. 2024;36:15.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWang S, Wang Y, Jiang L, Chang Y, zhang S, Zhao K, et al. Assessing the clinical support capabilities of ChatGPT 4o and ChatGPT 4o mini in managing lumbar disc herniation. Eur J Med Res. 2025;30:45.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAcharya PC, Alba R, Krisanapan P, Acharya CM, Suppadungsuk S, Csongradi E, et al. AI-Driven Patient Education in Chronic Kidney Disease: Evaluating Chatbot Responses against Clinical Guidelines. Diseases. 2024;12:185.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMasters K. Medical Teacher\u0026rsquo;s first ChatGPT\u0026rsquo;s referencing hallucinations: Lessons for editors, reviewers, and teachers. Med Teach. 2023;45:673\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eA\u0026iuml;meur E, Amri S, Brassard G. Fake news, disinformation and misinformation in social media: a review. Soc Netw Anal Min. 2023;13:30.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMonteith S, Glenn T, Geddes JR, Whybrow PC, Achtyes E, Bauer M. Artificial intelligence and increasing misinformation. Br J Psychiatry. 2024;224:33\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSuarez-Lledo V, Alvarez-Galvez J. Prevalence of Health Misinformation on Social Media: Systematic Review. J Med Internet Res. 2021;23:e17187.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eVicari R, Komendatova N. Systematic meta-analysis of research on AI tools to deal with misinformation on social media during natural and anthropogenic hazards and disasters. Humanit Soc Sci Commun. 2023;10:1\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOzbay FA, Alatas B. Fake news detection within online social media using supervised artificial intelligence algorithms. Physica A. 2020;540:123174.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJahanbakhsh F, Katsis Y, Wang D, Popa L, Muller M. Exploring the Use of Personalized AI for Identifying Misinformation on Social Media. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery; 2023. pp. 1\u0026ndash;27.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHan J, Qiu W, Lichtfouse E. ChatGPT in Scientific Research and Writing: A Beginner\u0026rsquo;s Guide. In: Han J, Qiu W, Lichtfouse E, editors. ChatGPT in Scientific Research and Writing: A Beginner\u0026rsquo;s Guide. Cham: Springer Nature Switzerland; 2024. pp. 1\u0026ndash;109.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMesk\u0026oacute; B. Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial. J Med Internet Res. 2023;25:e50638.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMiao J, Thongprayoon C, Suppadungsuk S, Garcia Valencia OA, Cheungpasitporn W. Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications. Medicina. 2024;60:445.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-nephrology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bnep","sideBox":"Learn more about [BMC Nephrology](http://bmcnephrol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bnep/default.aspx","title":"BMC Nephrology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7507934/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7507934/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis study aimed to evaluate the accuracy, content quality, and readability of patient education responses related to chronic kidney disease (CKD) generated by large language models (ChatGPT-4o mini and Gemini) compared to clinical guidelines. Fifteen frequently asked CKD-related questions were selected using global Google Trends data and posed to both AI models and guideline-based sources. Responses were anonymized and evaluated by four independent nephrology professors using the CLEAR Tool, assessing completeness, appropriateness, evidence basis, and clarity.\u003c/p\u003e\n\u003cp\u003eBoth AI models significantly outperformed guideline responses across all CLEAR Tool domains (p \u0026lt; 0.001), with ChatGPT-4o mini achieving the highest median score (21.0 [IQR: 5.0] vs. Gemini: 17.0 [IQR: 5.0], Guideline: 13.0 [IQR: 2.0]). Initial readability analysis showed that guideline responses were easier to comprehend (FKGL: 9.40; FRE: 52.01) than AI-generated content (ChatGPT FKGL: 11.34, FRE: 36.17; Gemini FKGL: 9.62, FRE: 46.36). However, when a standardized instructional prompt was applied, AI responses demonstrated significant improvements in readability, reducing the required literacy level to approximately the 7th-grade (ChatGPT FKGL: 7.87, FRE: 64.23; Gemini FKGL: 7.13, FRE: 61.45).\u003c/p\u003e\n\u003cp\u003eThese findings highlight the potential of prompt-guided AI models to generate accurate, accessible educational content for CKD. Prompt engineering emerges as a practical tool to enhance clarity and usability, particularly for populations with limited health literacy. Integration with frameworks like Retrieval-Augmented Generation may further improve reliability and safety in digital health communication.\u003c/p\u003e","manuscriptTitle":"Guidelines vs Generative AI in CKD Patient Education: The Role of Prompt Engineering and Expert Blinded Evaluation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-22 09:28:57","doi":"10.21203/rs.3.rs-7507934/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-12-10T08:13:59+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-10T02:42:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"192081204927518548606542021498282315563","date":"2025-11-26T06:38:05+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-23T15:00:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"18742620515412136238679263031376581714","date":"2025-09-15T03:54:55+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-11T07:13:12+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-03T12:12:02+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-03T12:10:48+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Nephrology","date":"2025-09-01T11:30:30+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-nephrology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bnep","sideBox":"Learn more about [BMC Nephrology](http://bmcnephrol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bnep/default.aspx","title":"BMC Nephrology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f3791571-c4ed-4388-9fa8-1002b88687b4","owner":[],"postedDate":"September 22nd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-02-23T16:10:41+00:00","versionOfRecord":{"articleIdentity":"rs-7507934","link":"https://doi.org/10.1186/s12882-026-04814-3","journal":{"identity":"bmc-nephrology","isVorOnly":false,"title":"BMC Nephrology"},"publishedOn":"2026-02-20 15:59:05","publishedOnDateReadable":"February 20th, 2026"},"versionCreatedAt":"2025-09-22 09:28:57","video":"","vorDoi":"10.1186/s12882-026-04814-3","vorDoiUrl":"https://doi.org/10.1186/s12882-026-04814-3","workflowStages":[]},"version":"v1","identity":"rs-7507934","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7507934","identity":"rs-7507934","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}