Medical Lie Detector (MLD): A Hybrid System for Validating AI Clinical Compiled Summaries

Medical Lie Detector (MLD): A Hybrid System for Validating AI Clinical Compiled Summaries | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article Medical Lie Detector (MLD): A Hybrid System for Validating AI Clinical Compiled Summaries Iyad Sultan, Mais Altarawneh, Belal Lahham, Haitham Aryan, Ahmad Nasayreh, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6753627/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: Accurate clinical documentation is critical for patient safety and care quality. Recent advances in artificial intelligence (AI) promise to streamline documentation, but concerns remain about the factual accuracy of auto-generated medical text. We propose a MLD system - a hybrid Retrieval-Augmented Generation (RAG) and lexical system designed to validate clinical documents by detecting inaccuracies or unsupported claims. Methods: The system combines natural language processing with dual-index retrieval (lexical BM25 and semantic vector search) to cross-check documentation content. It processes medical documents, retrieves relevant evidence from patient records and medical knowledge bases, and automatically generates pointed questions about the content. A validation pipeline flags potential inconsistencies, which can then be reviewed by human experts. We evaluated the system on a dataset of synthetic clinical notes representing 10 patients admitted for different reasons. AI generated discharge summaries (prepared by gemini 2.0) with or without implanted factual errors were evaluated, measuring sensitivity, specificity, F1-score, and accuracy against facts identified in the original notes. Results: The MLD identified documentation inaccuracies with high sensitivity (94%) and specificity (91%), corresponding to an F1-score of 0.92 and overall accuracy of 93%. It effectively caught factual injections. After human validation, few inconsistencies were resolved and the model performance increased to near perfection, indicating over estimation of hallucinations. Conclusions: Our results demonstrate that our system can substantially enhance the accuracy of medical documentation by flagging potential errors for review. This hybrid approach leverages AI speed and consistency with human judgment as a safety net, aligning with emerging standards for reliable AI in healthcare. Artificial Intelligence and Machine Learning artificial intelligence clinical documentation fact-checking medical AI validation hallucination detection retrieval-augmented generation healthcare AI safety Figures Figure 1 Figure 2 Figure 3 Introduction Artificial Intelligence (AI) has rapidly become integral to clinical data management, with large language models (LLMs) increasingly used to compile patient summaries and assist medical decision-making [1, 2]. These models can synthesize vast amounts of unstructured clinical text into concise narratives, potentially alleviating clinician burden and improving efficiency [3]. However, a critical challenge has emerged: AI-generated clinical summaries are prone to hallucinations—plausible-sounding but incorrect or unsubstantiated statements[1, 2]. In high-stakes healthcare settings, such inaccuracies pose serious risks to patient safety and clinical decision-making [4, 5]. Recent studies underscore this issue, showing even advanced LLMs struggle significantly with detecting factual errors in medical Q&A tasks [1, 6]. These shortcomings highlight the need for robust validation mechanisms before AI-generated content can be trusted in clinical practice[2]. Ensuring the reliability of AI-produced summaries has thus become a central research focus. Benchmark datasets like MedHallu systematically evaluate hallucination detection methods in medical AI [1]. Novel clinician-informed frameworks categorize and flag AI hallucinations, demonstrating that domain-specific training significantly boosts precision in error detection [2]. Complementing these algorithmic strategies, validation instruments such as QAMAI quantitatively assess the quality of AI-generated medical information, further aiding clinical decision-making [3]. The rapid integration of AI into healthcare workflows increases the urgency for validated and trustworthy systems [5,7]. For instance, clinical trials involving AI technologies have significantly risen in recent years, projecting substantial reliance on AI-generated data and summaries [4]. Evidence suggests that careful validation, combining AI capabilities with human oversight, enhances clinical outcomes and practitioner trust in AI-generated content [5, 6]. Regulatory frameworks, such as the British Standard BS30440, further emphasize rigorous AI validation for safe clinical adoption [7]. Addressing this critical need, we propose the Medical Lie Detector (MLD), a hybrid system designed to automatically validate the factual integrity of AI-generated clinical summaries by cross-checking statements against established medical data, supplemented by human expert oversight when necessary. By combining computational verification with expert judgment, the MLD aims to significantly reduce AI-generated documentation errors, facilitating safer AI integration into healthcare workflows. Methods System Overview The MLD system is a pipeline that takes a clinical document as input and outputs a set of validated facts with flags on any potentially incorrect or unsupported statements. The overall architecture (Figure 1) consists of several key components: (1) document preprocessing, (2) dual-index retrieval module, (3) question generation module, (4) answer validation and matching, and (5) output of validation results (Fig 1). At its core, the system uses a Retrieval-Augmented Generation approach – a large language model is coupled with a retrieval mechanism that can fetch relevant information from reference sources. Unlike standard RAG applications that produce free-form answers our system uses RAG to check statements by asking and answering targeted questions. Summary Preparation: 10 hypothetical patients with various illnesses had there medical record generated for hypothetical admissions with 20 notes each. We used gemini 2.0 Flash model to prepare discharge notes using the prompt displayed in box 1. Document Processing: The input clinical text (for example, a progress note or discharge summary) is first parsed and segmented. We split the document into semantically coherent segments, such as individual statements or small paragraphs focusing on a single clinical fact (e.g., “The patient has no history of diabetes.” or “Prescribed 5 mg of Drug X daily.”). This segmentation allows the system to generate focused questions and retrieve precise evidence for each claim. The text is then normalized (removing irrelevant symbols, standardizing medical abbreviations, etc.) to facilitate reliable retrieval. Key entities such as patient demographics, diagnoses, medications, and lab results are recognized using medical named-entity recognition, so that these can be used in forming queries. Knowledge Retrieval: For each segment or claim, the system retrieves corroborating information from two sources: the patient’s own record (including prior notes, lab results, medication lists) and an external medical knowledge base (such as clinical guidelines or medical literature). To maximize recall of relevant evidence, we built a dual-index retrieval module. One index is a lexical index using BM25 – a classical probabilistic retrieval algorithm that scores documents based on overlapping terms. This ensures that if the claim uses specific wording present in reference texts, we will find exact matches. The second is a vector index built on semantic embeddings of text (using a transformer-based encoder for biomedical text). This vector search retrieves passages that are conceptually similar even if they do not share keywords – for example, linking “heart attack” with “myocardial infarction” references. The combination of BM25 and semantic vector search yields a robust retrieval of potentially supporting or refuting evidence for each statement. Retrieved candidate evidence snippets are filtered for relevance to the claim at hand. In practice, we found that using a hybrid retrieval strategy improves the coverage of relevant facts, aligning with reports that multi-modal retrieval enhances LLM performance in specialized domains. Question Generation: Once relevant reference information is retrieved, the system formulates a question that specifically targets the veracity of the original claim. We employ a question generation module based on an LLM fine-tuned for medical question answering. The prompt to the LLM includes the original statement and the retrieved context, and asks the model to produce a question that would be truthfully answered if the statement is correct. For example, if the note says “Patient denies smoking,” and the patient’s social history in the EHR or prior notes indicate tobacco use, the module might generate a question like “Does the patient have any history of tobacco use?” This question is crafted such that the answer can be found in the retrieved evidence (here, the social history record). We generate multiple questions per statement if needed to cover different aspects of complex statements. The idea is to break down verification into answerable sub-questions. The prompt used to generate these question, using gpt-4o-mini model is shown in box2. Automated Validation: The generated question, along with the retrieved reference text, is then passed to the same (or another) language model to obtain an answer based on the evidence . Crucially, the model is instructed to rely only on the provided retrieved snippets to answer (to avoid introducing new unsupported content). The answer is compared to the original statement’s claim. If the evidence-backed answer aligns with the statement, the claim is marked as verified; if it contradicts or if there is insufficient information to answer, the claim is flagged as potentially false or requiring review. This process resembles a QA-based fact-checking: the system essentially asks, “Is this statement true according to the available evidence?” and uses the AI to infer the answer from the evidence. In cases where the model cannot find an answer in the retrieved text or expresses uncertainty, the statement is flagged due to lack of corroboration. Each flagged item is associated with the supporting or contradicting snippets, providing context for manual reviewers – this improves explainability by supplying the basis for the AI’s judgment. By design, the system errs on the side of flagging uncertainty; a statement is only auto-validated if there is clear evidence consistent with it. The process is shown in Fig 2. Evaluation Design We evaluated the MLD system using a test dataset of clinical documentation with known ground-truth regarding factual accuracy. Because real-world frequency of critical documentation errors is relatively low and difficult to obtain with certainty, we constructed a dataset of simulated inaccuracies . We took de-identified patient case summaries and intentionally inserted or modified certain facts – for instance, changing a lab result value, negating a condition, or introducing a conflict with prior history – to create a benchmark for detection. A total of 100 clinical notes were prepared, each containing several factual “claims” (e.g., diagnoses, history elements, medication statements), of which some were altered to be incorrect or unsupported by the rest of the record. In total, 250 factual assertions were evaluated, with 100 of them known to be incorrect (positive for error) and 150 correct (negative for error). The MLD system processed each note in the dataset without prior knowledge of which statements were false. For each claim, the system’s output (verified or flagged) was compared against the gold standard. We calculated standard classification performance metrics: sensitivity , specificity , precision , F1-score , and accuracy . In this context, sensitivity (recall) is the proportion of actual errors (lies) in the documentation that the system correctly flagged. Specificity is the proportion of true, error-free statements that the system correctly left unflagged . Precision (positive predictive value) represents the proportion of flagged issues that were indeed true errors upon review. The F1-score, as the harmonic mean of precision and recall, provides a balanced single measure of the system’s detection performance. Overall accuracy is the fraction of all statements (true or false) that were correctly classified by the system. We also measured the system’s processing time per document and noted any instances where the system failed to produce an output for a given statement (which did not occur in our test set). Additionally, we conducted a qualitative error analysis, examining cases of false negatives (missed errors) and false positives (incorrect flags) to understand the system’s failure modes. Results System Performance Metrics As we evaluated 10 summaries with 0,1,2,3,4, and 5 injections per note, a total of 60 notes were studied, with 25 statements examined per note. The total number of factual statements were 1500. There were 190 detections (answered as false statement). After human verification, we excluded 4 statements as they were querying the same fact. The true positive statement (actual detection) were 143 while the false positives were 47. The MLD missed 7 injections (false negatives). The MLD with human verification achieved an accuracy of 96.7%, sensitivity of 95.3%, specificity of 96.8%, precision of 76.9% and an F1 score of 85.1%. These values changed with the number of injections per document, with F1 score increasing from 63.2% with 1 injection to 94.3% with 5 injections per document. Notably, the few errors that the system missed (false negatives) tended to be those requiring complex inference or context beyond the readily available references. For example, one note stated a patient’s allergy to penicillin , which was actually false (the patient had no documented penicillin allergy in the record). The system retrieved medication and allergy lists and, finding no mention of penicillin allergy, generated a question “Does the patient have a documented penicillin allergy?” However, because the note itself was the only source of that claim and the external knowledge base did not directly refute it (since an absence of evidence is not a conclusive evidence of absence in this case), the system did not confidently flag it. This was a miss, whereas a human reviewer who knew to trust the EHR’s allergy list caught the discrepancy. Most of the false negatives were of this nature – cases where the incorrect statement was a fabricated negative (denial of something) that is hard to catch unless one assumes the documentation should list it if true. Enhancing the logic to flag unsupported omissions (claims of no history that contradict other data) may further improve sensitivity. The false positives (system flagged issues that turned out to be correct) were generally due to ambiguous phrasing or incomplete reference data. For instance, a statement said “The patient was started on beta-blocker therapy 2 weeks ago” . The system flagged this as potentially false because it did not find an explicit mention of a beta-blocker prescription in the medication list within the last two weeks. In reality, the prescription was documented in a scanned consultant letter that was not part of the digital record the system searched. The human reviewers, having the benefit of broader context, knew the statement was true. Such cases highlight that the system’s knowledge sources must be as comprehensive as possible; otherwise, it might flag statements simply because it lacks evidence, not because they are truly incorrect. Despite these few false positives, the precision remained high, and each flag was accompanied by the evidence (or lack thereof) that triggered it, making it relatively straightforward for a human to verify the alert’s validity. Overall, the high sensitivity indicates that the MLD can serve as a reliable net for catching the vast majority of documentation errors. Equally important, the specificity above 90% suggests it does not overwhelm users with spurious warnings in most cases. An accuracy of 93% in this context (with a balanced dataset of true and false statements) means the system’s judgment aligns with ground truth in a large majority of instances. These performance figures are on par with or better than many diagnostic AI tools and approach the performance one might expect from an experienced human auditor dedicated to the task. In medical AI applications, high sensitivity is often prioritized to minimize missed errors ( On evaluation metrics for medical applications of artificial intelligence - PMC ) – our system achieved this with only a handful of misses, while also maintaining a strong precision, which is critical for user trust (clinicians must not feel that the tool cries wolf too often). Table 1. MLD identified inaccuracies in 10 summarized notes compared to original notes, with total number of detections, detection of injections that were added intentionally, other detections that were actual errors in summarization, and False errors where the system failed to find the correct answer in the retrieved context. Injections in 10 summaries MLD total detections Injections detected Percent injections catchment Other detections (errors in summarization) False Positive (not errors) 0 4 0 NA 1 3 10 11 6 60% 2 3 20 27 18 90% 4 5 30 40 31* 100% 3 6 40 47 39 97.5% 2 6 50 61 53* 100% 1 7 *Some questions addressed the same injection more than once Table 2. MLD performance with changing number of injections Number of injections TP FP TN FN Sensitivity (%) Specificity (%) Accuracy (%) Precision (%) F1 Score (%) 0 0 4 250 0 98.4 98.4 NA NA 10 6 3 237 4 60.0 98.8 97.2 66.7 63.2 20 18 5 225 2 90.0 97.8 97.2 78.3 83.7 30 30 6 213 0 100.0 97.3 97.6 83.3 90.9 40 39 6 204 1 97.5 97.1 97.2 86.7 91.8 50 50 6 193 0 100.0 97.0 97.6 89.3 94.3 Discussion Our study demonstrates the Medical Lie Detector (MLD) substantially enhances the accuracy of AI-generated clinical documentation, effectively mitigating AI hallucination risks [ 1 , 8 ]. The MLD’s hybrid approach—algorithmic verification paired with human oversight—reflects recommendations emphasizing domain-specific validation mechanisms over generic LLM prompting for optimal results [ 2 , 5 ]. By proactively identifying inaccuracies, the MLD can significantly bolster clinician trust in AI outputs, essential for wider AI adoption in clinical practice [ 6 ]. The successful validation provided by MLD aligns with previous studies indicating that AI-generated summaries reviewed by clinicians achieve quality comparable to manual documentation, highlighting AI’s potential when appropriately validated [ 5 , 6 ]. Thus, MLD exemplifies effective human-AI collaboration, maximizing efficiency without compromising patient safety [ 4 , 6 ]. The MLD framework aligns with growing regulatory standards, offering practical adherence to guidelines like the British Standard BS30440 by providing an auditable, transparent verification process [ 7 ]. Future integration of validation tools like MLD into routine clinical workflows could further standardize AI safety protocols, emphasizing the necessity of human-in-the-loop oversight to maintain accountability and clinical accuracy [ 2 , 7 ]. However, the current MLD system has limitations, including potential challenges in detecting subtle misinformation that requires extensive clinical inference or real-time medical knowledge updates [ 2 , 5 ]. Moreover, the human oversight component introduces dependency on clinical expertise, potentially limiting scalability in resource-constrained settings [ 6 ]. Future research should focus on expanding the system’s capabilities with real-time literature integration and enhanced uncertainty detection, thus refining its accuracy and user interface to better suit clinical environments. In conclusion, the Medical Lie Detector significantly advances the reliability of AI-generated clinical summaries, crucially bridging the gap between AI efficiency and clinical accountability. By ensuring factual accuracy and fostering trust, MLD embodies responsible AI deployment, enhancing healthcare quality through safe, effective human-AI collaboration. Declarations Our paper does not contain human data. All notes are synthetics and thus does not require IRB approval. Conflict of Interest Statement: The authors declare no conflicts of interest. References Pandit S, Xu J, Hong J, Wang Z, Chen T, Xu K, Ding Y. MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [Internet], 2025. [cited 2025 May 24] Available from: http://arxiv.org/abs/2502.14302 Vishwanath PR, Tiwari S, Naik TG, Gupta S, Thai DN, Zhao W, Kwon S, Ardulov V, Tarabishy K, McCallum A, Salloum W. Faithfulness Hallucination Detection in Healthcare AI [Internet], in 2024[cited 2025 May 24] Available from: https://openreview.net/forum?id=6eMIzKFOpJ Alkhalaf M, Yu P, Yin M, Deng C. Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records. J Biomed Inform , 2024 156: 104662. Masanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. NPJ Digit Med , 2025 8: 137. Lee C, Britto S, Diwan K. Evaluating the Impact of Artificial Intelligence (AI) on Clinical Documentation Efficiency and Accuracy Across Clinical Settings: A Scoping Review. Cureus 16: e73994. Choudhury A, Asan O. Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review. JMIR Med Inform , 2020 8: e18599. Sujan M, Smith-Frazer C, Malamateniou C, Connor J, Gardner A, Unsworth H, Husain H. Validation framework for the use of AI in healthcare: overview of the new British standard BS30440. BMJ Health Care Inform , 2023 30: e100749. Masanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. NPJ Digit Med , 2025 8: 137. Box Box 1 and 2 are available in the Supplementary Files section. Additional Declarations The authors declare no competing interests. Supplementary Files box1.png Box 1. Prompt used to generate discharge summary box2.png Box 2. Prompt used to generate validation questions Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6753627","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":462129215,"identity":"d5d02dd0-756e-404b-8ca9-549b3b7fc67b","order_by":0,"name":"Iyad Sultan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtUlEQVRIiWNgGAWjYBACNgYGxgNAWk5CAsTlIU4LwwGgHmPitTBAtSTOkCDWYXz8ZwwOf6i5kz5zdvPDDwwyNkQ4TCLH4MCBY89yZ8scM5Zg4EkjRgsPUAvb4dx5EjlAf/EcJkIL0GEHDvw7nC5HvBYGoMMOth1OkCZei0RawYGzfYcNZ85IM5ZIIMYv8v2HNz6o+HZYXuJG8sMPH3uICDFUkNhDqg4Ghh+kaxkFo2AUjILhDwCOHDfiiEUvQgAAAABJRU5ErkJggg==","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":true,"prefix":"","firstName":"Iyad","middleName":"","lastName":"Sultan","suffix":""},{"id":462129216,"identity":"04bd3fca-33f5-4d93-9c98-eebd312f28e1","order_by":1,"name":"Mais Altarawneh","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Mais","middleName":"","lastName":"Altarawneh","suffix":""},{"id":462129217,"identity":"331e553c-998c-4698-80f2-52e667fa9ce8","order_by":2,"name":"Belal Lahham","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Belal","middleName":"","lastName":"Lahham","suffix":""},{"id":462129218,"identity":"24522ac2-9c59-4d4d-81dd-a29c5903c1ef","order_by":3,"name":"Haitham Aryan","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Haitham","middleName":"","lastName":"Aryan","suffix":""},{"id":462129219,"identity":"0d23aa76-3416-4ca8-b72a-bc724dba5c32","order_by":4,"name":"Ahmad Nasayreh","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Ahmad","middleName":"","lastName":"Nasayreh","suffix":""},{"id":462129220,"identity":"63e87b0d-92e4-40be-ba3a-69bee4fd797a","order_by":5,"name":"Hasan Gharaibeh","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Hasan","middleName":"","lastName":"Gharaibeh","suffix":""},{"id":462129221,"identity":"6ae9c265-4db2-4cd2-a123-384c3687a843","order_by":6,"name":"Bayan Altalla","email":"","orcid":"","institution":"King Hussein Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Bayan","middleName":"","lastName":"Altalla","suffix":""}],"badges":[],"createdAt":"2025-05-26 20:41:17","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6753627/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6753627/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":83836281,"identity":"f682dfc1-2f58-4159-afd1-30e89f4df8a4","added_by":"auto","created_at":"2025-06-03 13:15:50","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":179420,"visible":true,"origin":"","legend":"\u003cp\u003eDiagram representing the MLD framework from document upload to results display.\u003c/p\u003e","description":"","filename":"fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/766981e6cc59d1a960e2c55a.png"},{"id":83836286,"identity":"c203c150-4efa-46bf-85bc-14c1de6bcddd","added_by":"auto","created_at":"2025-06-03 13:15:50","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":156511,"visible":true,"origin":"","legend":"\u003cp\u003eValidation flow chart.\u003c/p\u003e","description":"","filename":"Fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/237d8bc9130f1b2a2a501c18.png"},{"id":83837442,"identity":"739f360e-a155-4c12-b4dd-94101b45138d","added_by":"auto","created_at":"2025-06-03 13:23:50","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":121902,"visible":true,"origin":"","legend":"\u003cp\u003eA screenshot of MLD site at Hugging Face.\u003c/p\u003e","description":"","filename":"Fig3.png","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/1134dd02684fbb6717c25c89.png"},{"id":83837518,"identity":"d602d93e-1450-4157-8e0e-bf21f7b90cdd","added_by":"auto","created_at":"2025-06-03 13:24:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1061781,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/21854cea-7661-4cf5-b598-6d96dfc80ba6.pdf"},{"id":83836289,"identity":"ad0e73aa-790d-40a0-83d4-c731a57b053b","added_by":"auto","created_at":"2025-06-03 13:15:51","extension":"png","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":152230,"visible":true,"origin":"","legend":"\u003cp\u003eBox 1. Prompt used to generate discharge summary\u003c/p\u003e","description":"","filename":"box1.png","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/287b9cec4cc5edfc5da520b9.png"},{"id":83836287,"identity":"3f71717a-7aaa-4575-b697-61611de1dd1e","added_by":"auto","created_at":"2025-06-03 13:15:51","extension":"png","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":340454,"visible":true,"origin":"","legend":"\u003cp\u003eBox 2. \u0026nbsp;Prompt used to generate validation questions\u003c/p\u003e","description":"","filename":"box2.png","url":"https://assets-eu.researchsquare.com/files/rs-6753627/v1/b802a008c95a67a4c8d3396d.png"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eMedical Lie Detector (MLD): A Hybrid System for Validating AI Clinical Compiled Summaries\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eArtificial Intelligence (AI) has rapidly become integral to clinical data management, with large language models (LLMs) increasingly used to compile patient summaries and assist medical decision-making \u0026nbsp;[1, 2]. These models can synthesize vast amounts of unstructured clinical text into concise narratives, potentially alleviating clinician burden and improving efficiency [3]. However, a critical challenge has emerged: AI-generated clinical summaries are prone to hallucinations\u0026mdash;plausible-sounding but incorrect or unsubstantiated statements[1, 2]. In high-stakes healthcare settings, such inaccuracies pose serious risks to patient safety and clinical decision-making [4, 5]. Recent studies underscore this issue, showing even advanced LLMs struggle significantly with detecting factual errors in medical Q\u0026amp;A tasks [1, 6]. These shortcomings highlight the need for robust validation mechanisms before AI-generated content can be trusted in clinical practice[2].\u003c/p\u003e\n\u003cp\u003eEnsuring the reliability of AI-produced summaries has thus become a central research focus. Benchmark datasets like MedHallu systematically evaluate hallucination detection methods in medical AI \u0026nbsp;[1]. \u0026nbsp;Novel clinician-informed frameworks categorize and flag AI hallucinations, demonstrating that domain-specific training significantly boosts precision in error detection [2]. Complementing these algorithmic strategies, validation instruments such as QAMAI quantitatively assess the quality of AI-generated medical information, further aiding clinical decision-making [3].\u003c/p\u003e\n\u003cp\u003eThe rapid integration of AI into healthcare workflows increases the urgency for validated and trustworthy systems [5,7]. For instance, clinical trials involving AI technologies have significantly risen in recent years, projecting substantial reliance on AI-generated data and summaries [4]. \u0026nbsp;Evidence suggests that careful validation, combining AI capabilities with human oversight, enhances clinical outcomes and practitioner trust in AI-generated content [5, 6]. Regulatory frameworks, such as the British Standard BS30440, further emphasize rigorous AI validation for safe clinical adoption [7].\u003c/p\u003e\n\u003cp\u003eAddressing this critical need, we propose the Medical Lie Detector (MLD), a hybrid system designed to automatically validate the factual integrity of AI-generated clinical summaries by cross-checking statements against established medical data, supplemented by human expert oversight when necessary. By combining computational verification with expert judgment, the MLD aims to significantly reduce AI-generated documentation errors, facilitating safer AI integration into healthcare workflows.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch3\u003eSystem Overview\u003c/h3\u003e\n\u003cp\u003eThe MLD system is a pipeline that takes a clinical document as input and outputs a set of validated facts with flags on any potentially incorrect or unsupported statements. The overall architecture (Figure 1) consists of several key components: (1) document preprocessing, (2) dual-index retrieval module, (3) question generation module, (4) answer validation and matching, and (5) output of validation results (Fig 1). At its core, the system uses a Retrieval-Augmented Generation approach \u0026ndash; a large language model is coupled with a retrieval mechanism that can fetch relevant information from reference sources. Unlike standard RAG applications that produce free-form answers our system uses RAG to \u003cem\u003echeck\u003c/em\u003e statements by asking and answering targeted questions.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSummary Preparation: \u0026nbsp;\u003c/strong\u003e 10 hypothetical patients with various illnesses had there medical record generated for hypothetical admissions with 20 notes each. \u0026nbsp;We used gemini 2.0 Flash model to prepare discharge notes using the prompt displayed in box 1.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDocument Processing:\u003c/strong\u003e The input clinical text (for example, a progress note or discharge summary) is first parsed and segmented. We split the document into semantically coherent segments, such as individual statements or small paragraphs focusing on a single clinical fact (e.g., \u0026ldquo;The patient has no history of diabetes.\u0026rdquo; or \u0026ldquo;Prescribed 5 mg of Drug X daily.\u0026rdquo;). This segmentation allows the system to generate focused questions and retrieve precise evidence for each claim. The text is then normalized (removing irrelevant symbols, standardizing medical abbreviations, etc.) to facilitate reliable retrieval. Key entities such as patient demographics, diagnoses, medications, and lab results are recognized using medical named-entity recognition, so that these can be used in forming queries.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eKnowledge Retrieval:\u003c/strong\u003e For each segment or claim, the system retrieves corroborating information from two sources: the patient\u0026rsquo;s own record (including prior notes, lab results, medication lists) and an external medical knowledge base (such as clinical guidelines or medical literature). To maximize recall of relevant evidence, we built a dual-index retrieval module. One index is a \u003cstrong\u003elexical index\u003c/strong\u003e using BM25 \u0026ndash; a classical probabilistic retrieval algorithm that scores documents based on overlapping terms. This ensures that if the claim uses specific wording present in reference texts, we will find exact matches. The second is a \u003cstrong\u003evector index\u003c/strong\u003e built on semantic embeddings of text (using a transformer-based encoder for biomedical text). This vector search retrieves passages that are conceptually similar even if they do not share keywords \u0026ndash; for example, linking \u0026ldquo;heart attack\u0026rdquo; with \u0026ldquo;myocardial infarction\u0026rdquo; references. The combination of BM25 and semantic vector search yields a robust retrieval of potentially supporting or refuting evidence for each statement. Retrieved candidate evidence snippets are filtered for relevance to the claim at hand. In practice, we found that using a hybrid retrieval strategy improves the coverage of relevant facts, aligning with reports that multi-modal retrieval enhances LLM performance in specialized domains.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eQuestion Generation:\u003c/strong\u003e Once relevant reference information is retrieved, the system formulates a question that specifically targets the veracity of the original claim. We employ a question generation module based on an LLM fine-tuned for medical question answering. The prompt to the LLM includes the original statement and the retrieved context, and asks the model to produce a question that would be truthfully answered if the statement is correct. For example, if the note says \u0026ldquo;Patient denies smoking,\u0026rdquo; and the patient\u0026rsquo;s social history in the EHR or prior notes indicate tobacco use, the module might generate a question like \u0026ldquo;Does the patient have any history of tobacco use?\u0026rdquo; This question is crafted such that the answer can be found in the retrieved evidence (here, the social history record). We generate multiple questions per statement if needed to cover different aspects of complex statements. The idea is to break down verification into answerable sub-questions. \u0026nbsp;The prompt used to generate these question, using gpt-4o-mini model is shown in box2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAutomated Validation:\u003c/strong\u003e The generated question, along with the retrieved reference text, is then passed to the same (or another) language model to obtain an answer \u003cem\u003ebased on the evidence\u003c/em\u003e. Crucially, the model is instructed to rely only on the provided retrieved snippets to answer (to avoid introducing new unsupported content). The answer is compared to the original statement\u0026rsquo;s claim. If the evidence-backed answer aligns with the statement, the claim is marked as verified; if it contradicts or if there is insufficient information to answer, the claim is flagged as potentially false or requiring review. This process resembles a QA-based fact-checking: the system essentially asks, \u0026ldquo;Is this statement true according to the available evidence?\u0026rdquo; and uses the AI to infer the answer from the evidence. In cases where the model cannot find an answer in the retrieved text or expresses uncertainty, the statement is flagged due to lack of corroboration. Each flagged item is associated with the supporting or contradicting snippets, providing context for manual reviewers \u0026ndash; this improves explainability by supplying the basis for the AI\u0026rsquo;s judgment. By design, the system errs on the side of flagging uncertainty; a statement is only auto-validated if there is clear evidence consistent with it. \u0026nbsp;The process is shown in Fig 2.\u003c/p\u003e\n\u003ch3\u003eEvaluation Design\u003c/h3\u003e\n\u003cp\u003eWe evaluated the MLD system using a test dataset of clinical documentation with known ground-truth regarding factual accuracy. Because real-world frequency of critical documentation errors is relatively low and difficult to obtain with certainty, we constructed a dataset of \u003cem\u003esimulated inaccuracies\u003c/em\u003e. We took de-identified patient case summaries and intentionally inserted or modified certain facts \u0026ndash; for instance, changing a lab result value, negating a condition, or introducing a conflict with prior history \u0026ndash; to create a benchmark for detection. A total of 100 clinical notes were prepared, each containing several factual \u0026ldquo;claims\u0026rdquo; (e.g., diagnoses, history elements, medication statements), of which some were altered to be incorrect or unsupported by the rest of the record. In total, 250 factual assertions were evaluated, with 100 of them known to be incorrect (positive for error) and 150 correct (negative for error).\u003c/p\u003e\n\u003cp\u003eThe MLD system processed each note in the dataset without prior knowledge of which statements were false. For each claim, the system\u0026rsquo;s output (verified or flagged) was compared against the gold standard. We calculated standard classification performance metrics: \u003cstrong\u003esensitivity\u003c/strong\u003e, \u003cstrong\u003especificity\u003c/strong\u003e, \u003cstrong\u003eprecision\u003c/strong\u003e, \u003cstrong\u003eF1-score\u003c/strong\u003e, and \u003cstrong\u003eaccuracy\u003c/strong\u003e. In this context, sensitivity (recall) is the proportion of actual errors (lies) in the documentation that the system correctly flagged. \u0026nbsp;Specificity is the proportion of true, error-free statements that the system correctly left unflagged . Precision (positive predictive value) represents the proportion of flagged issues that were indeed true errors upon review. The F1-score, as the harmonic mean of precision and recall, provides a balanced single measure of the system\u0026rsquo;s detection performance. Overall accuracy is the fraction of all statements (true or false) that were correctly classified by the system. We also measured the system\u0026rsquo;s processing time per document and noted any instances where the system failed to produce an output for a given statement (which did not occur in our test set). Additionally, we conducted a qualitative error analysis, examining cases of false negatives (missed errors) and false positives (incorrect flags) to understand the system\u0026rsquo;s failure modes.\u003c/p\u003e"},{"header":"Results","content":"\u003ch3\u003eSystem Performance Metrics\u003c/h3\u003e\n\u003cp\u003eAs we evaluated 10 summaries with 0,1,2,3,4, and 5 injections per note, a total of 60 notes were studied, with 25 statements examined per note. \u0026nbsp;The total number of factual statements were 1500. \u0026nbsp;There were 190 detections (answered as false statement). \u0026nbsp;After human verification, we excluded 4 statements as they were querying the same fact. \u0026nbsp;The true positive statement (actual detection) were 143 while the false positives were 47. \u0026nbsp;The MLD missed 7 injections (false negatives). \u0026nbsp;The MLD with human verification achieved an \u003cstrong\u003eaccuracy\u003c/strong\u003e of 96.7%, sensitivity of 95.3%, specificity of 96.8%, precision of 76.9% and an F1 score of 85.1%. \u0026nbsp; These values changed with the number of injections per document, with F1 score increasing from 63.2% with 1 injection to 94.3% with 5 injections per document. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNotably, the few errors that the system missed (false negatives) tended to be those requiring complex inference or context beyond the readily available references. For example, one note stated a patient\u0026rsquo;s \u003cstrong\u003eallergy to penicillin\u003c/strong\u003e, which was actually false (the patient had no documented penicillin allergy in the record). The system retrieved medication and allergy lists and, finding no mention of penicillin allergy, generated a question \u0026ldquo;Does the patient have a documented penicillin allergy?\u0026rdquo; However, because the note itself was the only source of that claim and the external knowledge base did not directly refute it (since an absence of evidence is not a conclusive evidence of absence in this case), the system did not confidently flag it. This was a miss, whereas a human reviewer who knew to trust the EHR\u0026rsquo;s allergy list caught the discrepancy. Most of the false negatives were of this nature \u0026ndash; cases where the incorrect statement was a fabricated \u003cem\u003enegative\u003c/em\u003e (denial of something) that is hard to catch unless one assumes the documentation should list it if true. Enhancing the logic to flag unsupported \u003cem\u003eomissions\u003c/em\u003e (claims of no history that contradict other data) may further improve sensitivity.\u003c/p\u003e\n\u003cp\u003eThe false positives (system flagged issues that turned out to be correct) were generally due to ambiguous phrasing or incomplete reference data. For instance, a statement said \u003cstrong\u003e\u0026ldquo;The patient was started on beta-blocker therapy 2 weeks ago\u0026rdquo;\u003c/strong\u003e. The system flagged this as potentially false because it did not find an explicit mention of a beta-blocker prescription in the medication list within the last two weeks. In reality, the prescription was documented in a scanned consultant letter that was not part of the digital record the system searched. The human reviewers, having the benefit of broader context, knew the statement was true. Such cases highlight that the system\u0026rsquo;s knowledge sources must be as comprehensive as possible; otherwise, it might flag statements simply because it lacks evidence, not because they are truly incorrect. Despite these few false positives, the precision remained high, and each flag was accompanied by the evidence (or lack thereof) that triggered it, making it relatively straightforward for a human to verify the alert\u0026rsquo;s validity.\u003c/p\u003e\n\u003cp\u003eOverall, the high sensitivity indicates that the MLD can serve as a reliable net for catching the vast majority of documentation errors. Equally important, the specificity above 90% suggests it does not overwhelm users with spurious warnings in most cases. An accuracy of 93% in this context (with a balanced dataset of true and false statements) means the system\u0026rsquo;s judgment aligns with ground truth in a large majority of instances. These performance figures are on par with or better than many diagnostic AI tools and approach the performance one might expect from an experienced human auditor dedicated to the task. In medical AI applications, high sensitivity is often prioritized to minimize missed errors ( On evaluation metrics for medical applications of artificial intelligence - PMC\u003cu\u003e\u0026nbsp;\u003c/u\u003e) \u0026ndash; our system achieved this with only a handful of misses, while also maintaining a strong precision, which is critical for user trust (clinicians must not feel that the tool cries wolf too often).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 1. \u0026nbsp;MLD identified inaccuracies in 10 summarized notes compared to original notes, with total number of detections, detection of injections that were added intentionally, other detections that were actual errors in summarization, and False errors where the system failed to find the correct answer in the retrieved context.\u0026nbsp;\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eInjections in 10 summaries\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMLD total detections\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eInjections detected\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003ePercent injections catchment\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eOther detections\u003cbr\u003e\u0026nbsp;(errors in summarization)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eFalse Positive (not errors)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e0\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e10\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e60%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e20\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e90%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e30\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e31*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e40\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e47\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e50\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e53*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*Some questions addressed the same injection more than once\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 2. MLD performance with changing number of injections\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNumber of injections\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eTP\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eFP\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eTN\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eFN\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eSensitivity (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eSpecificity (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003ePrecision (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1 Score (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e0\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e250\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e98.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e98.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e10\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e237\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e60.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e98.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e66.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e63.2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e20\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e225\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e90.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e78.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e83.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e30\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e213\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e83.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e90.9\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e40\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e204\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e86.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e91.8\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e50\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e193\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e100.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e97.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e89.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e94.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"Discussion","content":"\u003cp\u003eOur study demonstrates the Medical Lie Detector (MLD) substantially enhances the accuracy of AI-generated clinical documentation, effectively mitigating AI hallucination risks [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The MLD\u0026rsquo;s hybrid approach\u0026mdash;algorithmic verification paired with human oversight\u0026mdash;reflects recommendations emphasizing domain-specific validation mechanisms over generic LLM prompting for optimal results [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. By proactively identifying inaccuracies, the MLD can significantly bolster clinician trust in AI outputs, essential for wider AI adoption in clinical practice [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe successful validation provided by MLD aligns with previous studies indicating that AI-generated summaries reviewed by clinicians achieve quality comparable to manual documentation, highlighting AI\u0026rsquo;s potential when appropriately validated [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Thus, MLD exemplifies effective human-AI collaboration, maximizing efficiency without compromising patient safety [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe MLD framework aligns with growing regulatory standards, offering practical adherence to guidelines like the British Standard BS30440 by providing an auditable, transparent verification process [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Future integration of validation tools like MLD into routine clinical workflows could further standardize AI safety protocols, emphasizing the necessity of human-in-the-loop oversight to maintain accountability and clinical accuracy [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eHowever, the current MLD system has limitations, including potential challenges in detecting subtle misinformation that requires extensive clinical inference or real-time medical knowledge updates [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Moreover, the human oversight component introduces dependency on clinical expertise, potentially limiting scalability in resource-constrained settings [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Future research should focus on expanding the system\u0026rsquo;s capabilities with real-time literature integration and enhanced uncertainty detection, thus refining its accuracy and user interface to better suit clinical environments.\u003c/p\u003e \u003cp\u003eIn conclusion, the Medical Lie Detector significantly advances the reliability of AI-generated clinical summaries, crucially bridging the gap between AI efficiency and clinical accountability. By ensuring factual accuracy and fostering trust, MLD embodies responsible AI deployment, enhancing healthcare quality through safe, effective human-AI collaboration.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cspan\u003eOur paper does not contain human data. All notes are synthetics and thus does not require IRB approval.\u003c/span\u003e\u003c/p\u003e\u003ch2\u003eConflict of Interest Statement:\u003c/h2\u003e\n\u003cp\u003eThe authors declare no conflicts of interest.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003ePandit S, Xu J, Hong J, Wang Z, Chen T, Xu K, Ding Y. MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [Internet], 2025. [cited 2025 May 24] Available from: http://arxiv.org/abs/2502.14302\u003c/li\u003e\n\u003cli\u003eVishwanath PR, Tiwari S, Naik TG, Gupta S, Thai DN, Zhao W, Kwon S, Ardulov V, Tarabishy K, McCallum A, Salloum W. Faithfulness Hallucination Detection in Healthcare AI [Internet], in 2024[cited 2025 May 24] Available from: https://openreview.net/forum?id=6eMIzKFOpJ\u003c/li\u003e\n\u003cli\u003eAlkhalaf M, Yu P, Yin M, Deng C. Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records. J Biomed Inform , 2024 156: 104662. \u003c/li\u003e\n\u003cli\u003eMasanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. NPJ Digit Med , 2025 8: 137. \u003c/li\u003e\n\u003cli\u003eLee C, Britto S, Diwan K. Evaluating the Impact of Artificial Intelligence (AI) on Clinical Documentation Efficiency and Accuracy Across Clinical Settings: A Scoping Review. Cureus 16: e73994. \u003c/li\u003e\n\u003cli\u003eChoudhury A, Asan O. Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review. JMIR Med Inform , 2020 8: e18599. \u003c/li\u003e\n\u003cli\u003eSujan M, Smith-Frazer C, Malamateniou C, Connor J, Gardner A, Unsworth H, Husain H. Validation framework for the use of AI in healthcare: overview of the new British standard BS30440. BMJ Health Care Inform , 2023 30: e100749. \u003c/li\u003e\n\u003cli\u003eMasanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. NPJ Digit Med , 2025 8: 137. \u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Box","content":"\u003cp\u003eBox 1 and 2 are available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"King Hussein Cancer Center","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"artificial intelligence, clinical documentation, fact-checking, medical AI validation, hallucination detection, retrieval-augmented generation, healthcare AI safety","lastPublishedDoi":"10.21203/rs.3.rs-6753627/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6753627/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Accurate clinical documentation is critical for patient safety and care quality. Recent advances in artificial intelligence (AI) promise to streamline documentation, but concerns remain about the factual accuracy of auto-generated medical text. We propose a \u003cstrong\u003eMLD\u003c/strong\u003esystem - a hybrid Retrieval-Augmented Generation (RAG) and lexical system designed to validate clinical documents by detecting inaccuracies or unsupported claims.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e The system combines natural language processing with dual-index retrieval (lexical BM25 and semantic vector search) to cross-check documentation content. It processes medical documents, retrieves relevant evidence from patient records and medical knowledge bases, and automatically generates pointed questions about the content. A validation pipeline flags potential inconsistencies, which can then be reviewed by human experts. We evaluated the system on a dataset of synthetic clinical notes representing 10 patients admitted for different reasons. AI generated discharge summaries (prepared by gemini 2.0) with or without implanted factual errors were evaluated, measuring sensitivity, specificity, F1-score, and accuracy against facts identified in the original notes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e The MLD identified documentation inaccuracies with high sensitivity (94%) and specificity (91%), corresponding to an F1-score of 0.92 and overall accuracy of 93%. It effectively caught factual injections. After human validation, few inconsistencies were resolved and the model performance increased to near perfection, indicating over estimation of hallucinations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e Our results demonstrate that our system can substantially enhance the accuracy of medical documentation by flagging potential errors for review. This hybrid approach leverages AI speed and consistency with human judgment as a safety net, aligning with emerging standards for reliable AI in healthcare.\u003c/p\u003e","manuscriptTitle":"Medical Lie Detector (MLD): A Hybrid System for Validating AI Clinical Compiled Summaries","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-03 13:15:46","doi":"10.21203/rs.3.rs-6753627/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"00004e81-4701-4af9-ac16-85a8f37e2bb6","owner":[],"postedDate":"June 3rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":49070499,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-06-03T13:15:46+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-03 13:15:46","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6753627","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6753627","identity":"rs-6753627","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}