Enhancing Clinical Decision-Making in Rapid Response Systems: Integrating AI-Based Predictions and Early Pattern Clustering

Enhancing Clinical Decision-Making in Rapid Response Systems: Integrating AI-Based Predictions and Early Pattern Clustering | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Enhancing Clinical Decision-Making in Rapid Response Systems: Integrating AI-Based Predictions and Early Pattern Clustering Hyunsun Lim, Kyung Hyun Lee, Jung Hwa Hong, Sae Hwan An This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6866987/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background The AI solution used in the hospital's Rapid Response System is designed to predict critical events such as cardiopulmonary resuscitation (CPR), unexpected deaths, and unanticipated transfers to the intensive care unit (ICU) in real time. However, given the limitations perceived by the clinical staff regarding its effectiveness, we considered whether it could provide additional information to support decision-making. Objective This study supplemented decision-making support by utilizing predictive patterns produced by a critical event AI solution. Methods This study assessed the predictive performance of three decision-making strategies using data from Ilsan Hospital (n = 292,981; admissions between April 1, 2019, and February 28, 2024): a cut-off value–based strategy utilizing AI solution outputs, a clustering-based strategy categorizing patients into 10 classes according to early prediction patterns during the first five days of hospitalization, and a hybrid strategy combining both approaches. Results Although the AI solution–based decision-making approach yielded the highest AUROC (74.54%), its AUPRC (6.21%), F 1 score (17.86), and recall (10.72%) were relatively low. Including Classes 2, 5, and 6 along with AI solution further improved recall to 32.4% and the F 1 score or 34.54% although the AUROC decreased to 66.70%. Conclusions Supplementing the existing AI solution–based decision-making process with pattern-based predictions over a defined observation period may enhance the practical utility and real-world applicability of clinical decision-making in hospital settings. A.I. solution Latent class pattern machine learning deep learning Prediction performance Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction In-hospital clinical deterioration, encompassing events such as cardiopulmonary resuscitation (CPR), unexpected death, and unplanned transfers to intensive care units (ICUs), continues to pose significant challenges despite the widespread adoption of early warning systems (EWS) [ 1 ]. Traditional EWS models, which rely heavily on static thresholds for vital signs, often suffer from limitations in sensitivity and specificity, leading to missed events or an excessive number of false alarms [ 2 ]. In response, artificial intelligence (AI)-based solutions have emerged as promising tools to enhance the early detection of clinical deterioration. These AI models leverage a wide range of routinely collected electronic health record (EHR) data—including vital signs, laboratory results, and demographic information—to continuously monitor patients and predict the likelihood of adverse events [ 9 ]. Numerous retrospective studies have demonstrated the potential of AI-driven prediction models to outperform conventional scoring systems in terms of predictive accuracy [ 5 ]. As a result, following regulatory approvals, the implementation of AI-based early warning systems in real-world hospital settings has accelerated, often in conjunction with pilot programs for insurance reimbursement [ 6 ]. However, several important challenges have been identified during the real-world deployment of AI solutions. First, many AI models are designed to maximize specificity and precision, focusing on reducing false positives at the expense of recall [ 4 ]. In clinical practice, this approach may fail to capture rare but clinically significant events, which can have profound consequences for patients and healthcare systems. Second, the universal application of AI solutions across all hospitalized patients—including those with do-not-resuscitate (DNR) orders or receiving palliative care—can dilute the models' effectiveness for detecting unexpected and actionable deterioration [ 6 ]. Third, static predictions based solely on initial admission characteristics may inadequately reflect the dynamic and evolving nature of patient status during hospitalization. In light of these challenges, there is an urgent need for supplementary strategies that enhance existing AI-based early warning systems by providing additional, dynamic, and clinically meaningful information to frontline healthcare providers [ 4 ]. Rather than replacing established AI models, these strategies should aim to complement them, addressing gaps in sensitivity while maintaining an acceptable balance between alarm burden and clinical benefit [ 5 ]. Real-time pattern analysis of AI prediction outputs offers a promising approach to meet this need. By continuously analyzing prediction trends over an extended observation period, it may be possible to identify latent subgroups of patients at elevated risk who might otherwise be overlooked by threshold-based alerts. Latent class analysis (LCA) can uncover hidden structures within patient trajectories, potentially revealing clinically relevant heterogeneity that static models fail to capture. Therefore, in this study, we aimed to investigate whether analyzing real-time AI prediction patterns over a 5-day post-admission period and applying latent class analysis could supplement existing AI solutions. We sought to determine whether this approach could improve recall and enrich clinical decision-making by identifying higher-risk patient groups without abandoning the foundational structure of the current predictive models [ 5 ]. Ultimately, our goal was to offer a practical framework that could be integrated into AI graphical user interfaces (GUIs) or hospital electronic medical record (EMR) systems, enhancing the clinical applicability of AI-driven early warning systems in dynamic and complex real-world hospital environments. Methods Study cohort The study cohort included hospitalized patients at Ilsan Hospital between January 1, 2012, and June 30, 2024. Data collection began after the initiation of the Rapid Response System (RRS) pilot program on April 1, 2019, and included all eligible admissions thereafter [ 6 ]. Although the AI solution system was officially implemented in October 2022, predicted values were retrospectively generated from April 2019 to September 2022 to ensure sufficient data volume (n = 292,981, Fig. 1 ). Discharge records were collected from April 1, 2019, to July 12, 2024. Critical events were defined as the occurrence of cardiopulmonary resuscitation (CPR), death, or ICU transfer during hospitalization. To extract real-time predictions from the AI solution, a total of 18 features, including sex, age, vital signs, and laboratory results, were utilized [ 9 ]. In summary, the dataset consisted of admission and discharge records, critical events (CPR, death, ICU transfer), vital signs, and laboratory test results. This study was approved by the Institutional Review Board of NHIS Ilsan Hospital (NHIMC-2024-10-014-002). All methods were performed in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines. This study was conducted in accordance with the Declaration of Helsinki. Clustering To analyze the patterns of AI solution predictions, real-time measurements were organized at 1-hour intervals between hospital admission and either critical events or discharge. When multiple measurements were available within the same hour, the average value was used. Missing values were imputed using the last observation carried forward (LOCF) or the last observation backward (LOBF) methods. A minimum observation period of 5 days (120 hours) was secured for inclusion. For clustering, principal component analysis (PCA) was first performed to reduce the dimensionality of the data, producing 10 principal components that explained approximately 97.0% of the total variance (explained variance ratios for the first 10 components were 85.5%, 5.8%, 2.0%, 1.0%, 0.7%, 0.6%, 0.4%, 0.3%, 0.3%, and 0.3%, respectively) [ 8 ]. Subsequently, Gaussian Mixture Modeling (GMM) with 10 components and full covariance matrices was applied to the PCA-transformed data to estimate latent clusters. Each patient was assigned a latent class label based on the GMM clustering results. Performance measurements To evaluate the potential utility of the clustering results, we examined the distribution of composite events—defined as the combined occurrence of cardiopulmonary resuscitation (CPR), death, or ICU transfer—as well as the distribution of each individual event across the latent classes. Performance was assessed under three different conditions: first, by applying the AI solution prediction threshold (≥ 34); second, by using the latent classes obtained from clustering; and third, by combining the threshold and clustering information. For each condition, we calculated the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), recall, specificity, precision, negative predictive value (NPV), and F 1 score. In addition, the total number of alarms and the number of false alarms were analyzed for each strategy. All analyses were performed using Python (version 3.11.9) [ 12 , 13 ]. Results Demographic characteristics Baseline characteristics of the study population, stratified by the latent classes identified through clustering, are summarized. A total of 292,981 patients were included in the analysis. Demographic and clinical variables, corresponding to the features used in the AI solution predictions—including gender, age, vital signs, and laboratory test results—were compared across the 10 latent classes, which were derived based on composite event outcomes. The number of patients in each latent class was as follows: Class 0, 171,046 (58.4%); Class 1, 17,086 (5.8%); Class 2, 7,560 (2.6%); Class 3, 10,946 (3.7%); Class 4, 24,406 (8.3%); Class 5, 2,026 (0.7%); Class 6, 2,828 (1.0%); Class 7, 14,502 (5.0%); Class 8, 16,372 (5.6%); and Class 9, 26,209 (8.9%). Notably, variations were observed in age, mental status, and inflammatory markers (e.g., CRP and lactate) among different clusters. These findings suggest that the identified latent classes captured meaningful clinical heterogeneity within the cohort (Table 1 ). Clustering The patterns of the 10 latent classes based on composite events were visualized over a 5-day (120-hour) period (Fig. 2 ). Graphs for CPR, death, and ICU transfer are provided in the supplementary files (Supplementary Fig. 1–3). The distribution of the 10 latent classes and their corresponding event rates for composite events, CPR, death, and ICU transfer are summarized (Table 2 ). Composite events were most frequently observed in Class 5 (54.6%), followed by Class 6 (36.0%), Class 2 (33.3%), and Class 8 (20.8%). CPR events were most common in Class 7 (8.3%), followed by Class 8 (3.7%), Class 3 (1.4%), and Classes 9 and 6 (each 0.4%). Mortality rates were highest in Class 2 (51.2%), followed by Class 8 (24.4%), Class 5 (7.6%), and Class 6 (3.4%). ICU transfers occurred most frequently in Class 6 (36.7%), followed by Class 2 (20.7%), Class 8 (5.8%), and Class 4 (5.7%). Table 2 Latent class Distribution and Event Rates for Each Critical Event. Composite CPR Death ICU Class n(%) Event(%) Class n(%) Event(%) Class n(%) Event(%) Class n(%) Event(%) All 292,981(100.0) 14,640(5.0) All 292,981(100.0) 1,254(0.4) All 292,981(100.0) 5,756(2.0) All 292,981(100.0) 10,745(3.7) (5) 2,026(0.7) 1,107(54.6) (7) 1,581(0.5) 131(8.3) (2) 1,741(0.6) 891(51.2) (6) 7,528(2.6) 2,761(36.7) (6) 2,828(1.0) 1,019(36.0) (8) 3,163(1.1) 118(3.7) (8) 3,661(1.3) 893(24.4) (3) 2,493(0.9) 551(22.1) (2) 7,560(2.6) 2,520(33.3) (3) 9,401(3.2) 128(1.4) (5) 7,689(2.6) 587(7.6) (2) 16,894(5.8) 3,491(20.7) (8) 16,372(5.6) 3,404(20.8) (9) 25,652(8.8) 111(0.4) (6) 12,987(4.4) 445(3.4) (9) 3,698(1.3) 214(5.8) (4) 24,406(8.3) 1,559(6.4) (6) 14,153(4.8) 59(0.4) (3) 10,751(3.7) 321(3.0) (4) 16,791(5.7) 960(5.7) (7) 14,502(5.0) 843(5.8) (1) 10,623(3.6) 42(0.4) (9) 23,994(8.2) 708(3.0) (1) 11,760(4.0) 451(3.8) (3) 10,946(3.7) 624(5.7) (5) 24,304(8.3) 86(0.4) (1) 26,669(9.1) 727(2.7) (7) 13,098(4.5) 431(3.3) (1) 17,086(5.8) 966(5.7) (2) 12,654(4.3) 42(0.3) (0) 170,585(58.2) 1,144(0.7) (5) 28,936(9.9) 887(3.1) (9) 26,209(9.0) 1,229(4.7) (0) 170,687(58.3) 500(0.3) (4) 20,841(7.1) 37(0.2) (8) 20,828(7.1) 586(2.8) (0) 171,046(58.4) 1,369(0.8) (4) 20,763(7.1) 37(0.2) (7) 14,063(4.8) 3(0.0) (0) 170,955(58.4) 422(0.3) Performance Model performance for predicting composite events, CPR, death, and ICU transfer is summarized (Fig. 3, Supplementary Table 1, 2). For composite events, the AI solution alone achieved an AUROC of 74.54% and an AUPRC of 6.21%, with a recall of 10.72% a specificity of 99.51%, and F 1 score of 17.86%. When focusing only on patients within Class 5, performance was similar (AUROC 74.99%), but recall was lower (7.56%). Including Classes 2, 5, and 6 along with AI solution further improved recall to 32.4% and the F 1 score or 34.54% although the AUROC decreased to 66.70%. For CPR events, the AI solution alone achieved an AUROC of 52.25% and an AUPRC of 2.35%, with a recall of 17.78%, a specificity of 98.50%, and the F 1 score of 7.64%. When focusing on patients in Class 7, the AUROC slightly improved to 53.95%, but recall decreased to 10.45%. Expanding to Classes 3, 7, and 8 along with AI solution improved recall to 30.54% and the F 1 score to 4.79%, although the AUROC decreased to 51.14%. For mortality (death) events, the AI solution alone achieved an AUROC of 66.94% and an AUPRC of 11.76%, with a recall of 30.43%, a specificity of 98.93%, and an F 1 score of 31.86%. Targeting patients in Class 2 achieved a higher AUROC (74.80%) but lower recall (16.25%). Including Classes 2 and 8 along with AI solution improved recall to 36.61% and the F 1 score to 31.86%, with a reduced AUROC of 63.49%. For ICU transfer prediction, the AI solution alone achieved an AUROC of 60.06% and an AUPRC of 2.32%, with a recall of 6.53%, a specificity of 99.19%, and an F 1 score of 10.23%. Focusing on Class 6 alone improved AUROC to 66.94% and recall to 25.67%. Expanding to Classes 3, and 6 with AI solution achieved the highest recall (32.03%) and an F 1 score of 31.04%, with a modest AUROC of 63.73%. To examine whether performance varies depending on the threshold setting, a sensitivity analysis was conducted to compare results obtained under different thresholds with those derived from the proposed AI and clustering-based approach. Based on the F 1 score, which is particularly suitable for disease prediction, the proposed method demonstrated higher performance. Alarm The alarm characteristics for composite events, CPR, death, and ICU transfer are summarized (Fig. 4 , Supplementary Table 3). For composite events, the AI solution alone generated 2,928 alarms, with 53.59% identified as appropriate and 46.41% classified as false alarms. Restricting to Class 5 produced 2,026 alarms with a similar appropriateness rate of 54.64% and a false alarm rate of 45.36%. Including Classes 2, 5, and 6 along with the AI solution further increased the total alarms to 12,858, with a decreased appropriateness of 36.93% and a false alarm rate of 63.07%. For CPR events, the AI solution alone generated 4,585 alarms, with only 4.86% identified as appropriate and 95.14% classified as false alarms. Restricting to Class 7 produced 1,581 alarms with a slightly higher appropriateness rate of 8.29% and a false alarm rate of 91.71%. Including Classes 3, 7, and 8 along with the AI solution increased the total alarms to 14,749, while the appropriateness rate decreased to 2.60% and the false alarm rate rose to 97.40%. For mortality (death) events, the AI solution alone generated 4,738 alarms, with 35.20% identified as appropriate and 64.80% classified as false alarms. Restricting to Class 2 produced 1,741 alarms with an appropriateness rate of 51.18% and a false alarm rate of 48.82%. Including Classes 2 and 8 along with the AI solution increased the total alarms to 7,116, while the appropriateness rate decreased to 28.20% and the false alarm rate increased to 71.80%. For ICU transfer, the AI solution alone generated 2,976 alarms, with 23.59% identified as appropriate and 76.41% classified as false alarms. Restricting to Class 6 produced 7,528 alarms with a higher appropriateness rate of 36.68% and a false alarm rate of 63.32%. Including Classes 3 and 6 along with the AI solution increased the total alarms to 11,458, with an appropriateness rate of 30.06% and a false alarm rate of 69.94%. Discussion In recent years, the development and evaluation of AI-based hardware and software targeting hospitalized patients have been actively pursued in clinical fields. Particularly following regulatory approvals, there has been a surge in real-world validation studies conducted prior to insurance reimbursement [ 2 , 6 ]. At this hospital, an AI solution aimed at reducing critical events via a rapid response system has been implemented since November 2022. Despite initial expectations, real-world clinical utility, as perceived by frontline clinicians, appeared limited after deployment. One major issue was a mismatch in the at-risk population. Since the AI solution was applied universally to all inpatients—including those in hospice care or with do-not-resuscitate (DNR) orders—the predictive performance for unforeseen critical events was inevitably diluted. While the development of a new prediction model to overcome these limitations would be ideal, this study instead focused on analyzing real-time AI prediction outputs to extract patterns that could provide more actionable and clinically relevant information. Recognizing that meaningful pattern detection requires a certain observation period, we designed a model based on 5-day post-admission data rather than immediate prediction. Although the proposed approach may not outperform existing models in terms of immediate prediction accuracy, it offers potential value by providing additional information beyond five days of hospitalization [ 4 ]. Furthermore, conventional AI solutions tend to prioritize specificity and precision, focusing only on clear and definitive predictions [ 7 ]. However, in clinical practice, even rare critical events are significant, given their profound impacts on patients, families, and healthcare staff [ 8 ]. In this context, strategies that minimize alarms and prioritize only highly certain events may not be appropriate. Thus, rather than aiming to replace existing AI models, our approach seeks to supplement them by applying a latent class analysis to increase recall [ 11 ]. Even though this strategy may involve greater manpower, time, and resource consumption, it offers clinicians additional information that can guide decision-making without disregarding low-frequency but high-impact events [ 9 , 13 ]. This study utilized data collected from April 2019 to June 2024. Although hospital data were available from 2012, we restricted our analysis to post-2019, when the hospital’s rapid response system was actively introduced, ensuring more consistent monitoring practices. It should be noted that although the AI solution was officially introduced in November 2022, retrospective prediction outputs were calculated for the period from 2019 to 2022 for the purpose of data completeness, which may introduce some unknown bias. Additionally, because this approach relies on a 5-day post-admission patterning, it is not designed for immediate event prediction within the early hospitalization period, unlike most conventional models that rely on short-term windows before discharge or event occurrence. External validation using data from other institutions was not conducted, which remains a key limitation of this study. As a result, the generalizability of the findings to other clinical environments remains uncertain. Nevertheless, considering the common issue of domain dependency in AI prediction models, providing additional institution-specific information tailored to each hospital’s operational context may help address the heterogeneity across different sites. Rather than proposing a new prediction model, the goal of this study was to offer a framework for enhancing existing AI solutions by supplying supplementary information based on real-time patterns. We anticipate that incorporating this analytical approach into AI solution graphical user interfaces (GUI) or hospital EMR systems may help mitigate the known limitations of current AI models, improving their clinical utility. Conclusion This study investigated a novel approach for supplementing AI-based early warning systems in hospitalized patients by utilizing real-time prediction pattern analysis over a 5-day observation period. Rather than aiming to replace existing AI solutions, we proposed a framework to enhance their clinical utility by offering additional information through latent class analysis, thereby improving recall without compromising decision-making in critical care settings. Although the proposed method requires a longer observational window and may generate a higher number of alarms, it emphasizes the clinical importance of detecting rare but high-impact events. Incorporating this strategy into existing AI solution interfaces or hospital EMR systems may contribute to overcoming current limitations in specificity-focused models and better align predictive tools with real-world clinical needs. Declarations Acknowledgements Not applicable. Author contributions Conceptualization, Validation: Lim, Lee; Data curation: Lim, Hong; Methodology, formal analysis, and investigation: Lim, Hong; Funding acquisition: Lim; Project administration: Lim Resources: Lim; Supervision, Visualization: Lim, Seo; Writing-original draft: All authors; Writing-review & editing: All authors Funding This work was supported by an NHIS (National Health Insurance Service) Ilsan Hospital grant (NHIMC-2024-CR-077). Availability of data and materials Due to the data protection policy of Ilsan Hospital, the datasets cannot be shared publicly. Ethics approval and consent to participate The requirement for informed consent was waived by the Institutional Review Board (IRB). The authors declare that they have no conflict of interest with the NHIS. This study was approved by the Institutional Review Board of NHIS Ilsan Hospital (NHIMC 2024-10-014-002). Consent for publication Not applicable. Competing interests All authors declare that they have no competing interests. Author details 1 Department of Research and Analysis, National Health Insurance Service Ilsan Hospital, Goyang, Republic of Korea References Pimentel MAF, Redfern OC, Malycha J, et al. Detecting deteriorating patients in the hospital: A comparison of early warning scores with machine learning models. J Crit Care. 2020;57:156–61. Chromik J, Flint AR, Prendke M, Arnrich B, Poncette AS. Enabling machine learning models in alarm fatigue research: Creation of a large relevance-annotated oxygen saturation alarm data set. Comput Biol Med. 2024;183:109244. Gallo R, Wang J, Choppala S, et al. Effectiveness of an artificial intelligence–enabled intervention for inpatient clinical deterioration. JAMA Intern Med. 2024;184(9):1137. Görges M, Markewitz BA, Westenskow DR. Improving Alarm Performance in the Medical Intensive Care Unit Using Delays and Clinical Context. Anesth Analg. 2009;108(5):1546–52. Au-Yeung WM, Sahani AK, Isselbacher EM, Armoundas AA. Reduction of false alarms in the intensive care unit using an optimized machine learning based approach. NPJ Digit Med. 2019;2:86. Ministry of Health and Welfare. Guidelines for the Pilot Program of the Rapid Response System. (Revised on April 1, 2023). Cvach M. Monitor alarm fatigue: An integrative review. Biomed Instrum Technol. 2012;26(4):214–22. Ben-Hur A, Guyon I. Detecting stable clusters using principal component analysis. Methods Mol Biol. 2003;224:159–82. Liu Z, Yu L, Hsiao JH, Chan AB. Parametric manifold learning of Gaussian mixture models. Proc Int Joint Conf Artif Intell. 2019;28:3079–85. Wang Y, Zhang Y, Zhang L, et al. A principal component analysis-boosted dynamic Gaussian mixture clustering model for ignition factors of Brazil’s rainforests. IEEE Access. 2021;9:144835–47. Clifford GD, Liu C, Moody B, Lehman LH, Silva I, Li Q, et al. AF Classification from a Short Single Lead ECG Recording: The PhysioNet/Computing in Cardiology Challenge 2017. Comput Cardiol. 2017;44:1–4. Python 3 Reference Manual. https:// docs.python.org/3/reference/ Accessed 2023-01-19. Abadi M, Agarwal A, Barham P, Brevo E, Chen Z, Citro C et al. TensorFlow: Large-scale machine learning on heterogeneous systems. (2015). Accessed 01–19. Table 1 Table 1 is available in the Supplementary Files section. Additional Declarations No competing interests reported. Supplementary Files SuppleMAESpattern.docx Table1.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6866987","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":483971953,"identity":"41661245-2440-4436-b576-a0e40d9b45e0","order_by":0,"name":"Hyunsun Lim","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAxUlEQVRIiWNgGAWjYBACAxDxwIBBjkGCsQEixEOMlgQDBmNStTAwJDZIwIQIaTHnP2P2IKFgW/p26ebWDT8YbPIZeM4+wKvFckaOuUGCwe3cnXMOtt3sYUizbOBtN8DvsBs8ZhIgLRtuJLbdZmA4bMDAz0bAL+fPgLWkGxCv5UAOWEsCQgtvGwEtN9LKQFoMQQ672WOQZsDGc4yQww5vk/jw57a8wY30Zzd+VNgY8POk4deCbgIDAwGfjIJRMApGwSggBgAAtylD2qTG4UcAAAAASUVORK5CYII=","orcid":"","institution":"National Health Insurance Service Ilsan Hospital","correspondingAuthor":true,"prefix":"","firstName":"Hyunsun","middleName":"","lastName":"Lim","suffix":""},{"id":483971954,"identity":"b4bb63b4-e6c9-4e2a-9c67-077d195ec47d","order_by":1,"name":"Kyung Hyun Lee","email":"","orcid":"","institution":"AITRICS","correspondingAuthor":false,"prefix":"","firstName":"Kyung","middleName":"Hyun","lastName":"Lee","suffix":""},{"id":483971955,"identity":"21b6a511-9e90-4319-85d3-1cd6965f9189","order_by":2,"name":"Jung Hwa Hong","email":"","orcid":"","institution":"National Health Insurance Service Ilsan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Jung","middleName":"Hwa","lastName":"Hong","suffix":""},{"id":483971956,"identity":"df57ea64-ee45-42c8-9976-6766b0cefcda","order_by":3,"name":"Sae Hwan An","email":"","orcid":"","institution":"National Health Insurance Service Ilsan Hospital","correspondingAuthor":false,"prefix":"","firstName":"Sae","middleName":"Hwan","lastName":"An","suffix":""}],"badges":[],"createdAt":"2025-06-11 02:08:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6866987/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6866987/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":87029094,"identity":"f4c2a910-0aa5-4507-aa29-827d65d497cd","added_by":"auto","created_at":"2025-07-18 12:36:35","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":17728,"visible":true,"origin":"","legend":"\u003cp\u003eOverview of the model structure\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/88d80fb8bd05496461e543e6.png"},{"id":87029095,"identity":"9abdb056-97d2-4680-a9ca-dfc2dc3febc7","added_by":"auto","created_at":"2025-07-18 12:36:35","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":103979,"visible":true,"origin":"","legend":"\u003cp\u003eLatent class pattern in composite events\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/86de4336e4af925601f90da9.png"},{"id":87029097,"identity":"3a9739d5-e977-479e-9173-dccda97d50ee","added_by":"auto","created_at":"2025-07-18 12:36:35","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":17215,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance results according to decision-making approaches for composite events.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/805bd8d10aad60d80104f53c.png"},{"id":87029101,"identity":"ee643a9a-4061-443b-804c-3a6c9e95a7d2","added_by":"auto","created_at":"2025-07-18 12:36:35","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":23695,"visible":true,"origin":"","legend":"\u003cp\u003eAlarm results according to decision-making approaches for composite events.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/1b84cb825098d9a16d66effa.png"},{"id":88973812,"identity":"f38aa309-4d6c-4531-8e33-fdda40a47e35","added_by":"auto","created_at":"2025-08-13 10:02:11","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":717097,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/1f71e881-2237-4e0f-94e2-efee203e714d.pdf"},{"id":87030480,"identity":"16bb0b67-98a5-4f33-b470-f759a653aaf3","added_by":"auto","created_at":"2025-07-18 12:44:35","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":428211,"visible":true,"origin":"","legend":"","description":"","filename":"SuppleMAESpattern.docx","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/56dcf388e8b51b16b93f24f2.docx"},{"id":87031433,"identity":"826e871c-c2a7-4c45-b140-41fdfaa3cf01","added_by":"auto","created_at":"2025-07-18 12:52:35","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":21771,"visible":true,"origin":"","legend":"","description":"","filename":"Table1.docx","url":"https://assets-eu.researchsquare.com/files/rs-6866987/v1/fabd0caeda529fed86e31a5b.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhancing Clinical Decision-Making in Rapid Response Systems: Integrating AI-Based Predictions and Early Pattern Clustering","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn-hospital clinical deterioration, encompassing events such as cardiopulmonary resuscitation (CPR), unexpected death, and unplanned transfers to intensive care units (ICUs), continues to pose significant challenges despite the widespread adoption of early warning systems (EWS) [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Traditional EWS models, which rely heavily on static thresholds for vital signs, often suffer from limitations in sensitivity and specificity, leading to missed events or an excessive number of false alarms [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. In response, artificial intelligence (AI)-based solutions have emerged as promising tools to enhance the early detection of clinical deterioration. These AI models leverage a wide range of routinely collected electronic health record (EHR) data\u0026mdash;including vital signs, laboratory results, and demographic information\u0026mdash;to continuously monitor patients and predict the likelihood of adverse events [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Numerous retrospective studies have demonstrated the potential of AI-driven prediction models to outperform conventional scoring systems in terms of predictive accuracy [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. As a result, following regulatory approvals, the implementation of AI-based early warning systems in real-world hospital settings has accelerated, often in conjunction with pilot programs for insurance reimbursement [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eHowever, several important challenges have been identified during the real-world deployment of AI solutions. First, many AI models are designed to maximize specificity and precision, focusing on reducing false positives at the expense of recall [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In clinical practice, this approach may fail to capture rare but clinically significant events, which can have profound consequences for patients and healthcare systems. Second, the universal application of AI solutions across all hospitalized patients\u0026mdash;including those with do-not-resuscitate (DNR) orders or receiving palliative care\u0026mdash;can dilute the models' effectiveness for detecting unexpected and actionable deterioration [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Third, static predictions based solely on initial admission characteristics may inadequately reflect the dynamic and evolving nature of patient status during hospitalization. In light of these challenges, there is an urgent need for supplementary strategies that enhance existing AI-based early warning systems by providing additional, dynamic, and clinically meaningful information to frontline healthcare providers [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Rather than replacing established AI models, these strategies should aim to complement them, addressing gaps in sensitivity while maintaining an acceptable balance between alarm burden and clinical benefit [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Real-time pattern analysis of AI prediction outputs offers a promising approach to meet this need. By continuously analyzing prediction trends over an extended observation period, it may be possible to identify latent subgroups of patients at elevated risk who might otherwise be overlooked by threshold-based alerts. Latent class analysis (LCA) can uncover hidden structures within patient trajectories, potentially revealing clinically relevant heterogeneity that static models fail to capture.\u003c/p\u003e\u003cp\u003eTherefore, in this study, we aimed to investigate whether analyzing real-time AI prediction patterns over a 5-day post-admission period and applying latent class analysis could supplement existing AI solutions. We sought to determine whether this approach could improve recall and enrich clinical decision-making by identifying higher-risk patient groups without abandoning the foundational structure of the current predictive models [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Ultimately, our goal was to offer a practical framework that could be integrated into AI graphical user interfaces (GUIs) or hospital electronic medical record (EMR) systems, enhancing the clinical applicability of AI-driven early warning systems in dynamic and complex real-world hospital environments.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eStudy cohort\u003c/h2\u003e\u003cp\u003eThe study cohort included hospitalized patients at Ilsan Hospital between January 1, 2012, and June 30, 2024. Data collection began after the initiation of the Rapid Response System (RRS) pilot program on April 1, 2019, and included all eligible admissions thereafter [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Although the AI solution system was officially implemented in October 2022, predicted values were retrospectively generated from April 2019 to September 2022 to ensure sufficient data volume (n\u0026thinsp;=\u0026thinsp;292,981, Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Discharge records were collected from April 1, 2019, to July 12, 2024. Critical events were defined as the occurrence of cardiopulmonary resuscitation (CPR), death, or ICU transfer during hospitalization. To extract real-time predictions from the AI solution, a total of 18 features, including sex, age, vital signs, and laboratory results, were utilized [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. In summary, the dataset consisted of admission and discharge records, critical events (CPR, death, ICU transfer), vital signs, and laboratory test results. This study was approved by the Institutional Review Board of NHIS Ilsan Hospital (NHIMC-2024-10-014-002). All methods were performed in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines. This study was conducted in accordance with the Declaration of Helsinki.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eClustering\u003c/h3\u003e\n\u003cp\u003eTo analyze the patterns of AI solution predictions, real-time measurements were organized at 1-hour intervals between hospital admission and either critical events or discharge. When multiple measurements were available within the same hour, the average value was used. Missing values were imputed using the last observation carried forward (LOCF) or the last observation backward (LOBF) methods. A minimum observation period of 5 days (120 hours) was secured for inclusion. For clustering, principal component analysis (PCA) was first performed to reduce the dimensionality of the data, producing 10 principal components that explained approximately 97.0% of the total variance (explained variance ratios for the first 10 components were 85.5%, 5.8%, 2.0%, 1.0%, 0.7%, 0.6%, 0.4%, 0.3%, 0.3%, and 0.3%, respectively) [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Subsequently, Gaussian Mixture Modeling (GMM) with 10 components and full covariance matrices was applied to the PCA-transformed data to estimate latent clusters. Each patient was assigned a latent class label based on the GMM clustering results.\u003c/p\u003e\n\u003ch3\u003ePerformance measurements\u003c/h3\u003e\n\u003cp\u003eTo evaluate the potential utility of the clustering results, we examined the distribution of composite events\u0026mdash;defined as the combined occurrence of cardiopulmonary resuscitation (CPR), death, or ICU transfer\u0026mdash;as well as the distribution of each individual event across the latent classes. Performance was assessed under three different conditions: first, by applying the AI solution prediction threshold (\u0026ge;\u0026thinsp;34); second, by using the latent classes obtained from clustering; and third, by combining the threshold and clustering information. For each condition, we calculated the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), recall, specificity, precision, negative predictive value (NPV), and F\u003csub\u003e1\u003c/sub\u003e score. In addition, the total number of alarms and the number of false alarms were analyzed for each strategy. All analyses were performed using Python (version 3.11.9) [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\u003ch2\u003eDemographic characteristics\u003c/h2\u003e\u003cp\u003eBaseline characteristics of the study population, stratified by the latent classes identified through clustering, are summarized. A total of 292,981 patients were included in the analysis. Demographic and clinical variables, corresponding to the features used in the AI solution predictions\u0026mdash;including gender, age, vital signs, and laboratory test results\u0026mdash;were compared across the 10 latent classes, which were derived based on composite event outcomes. The number of patients in each latent class was as follows: Class 0, 171,046 (58.4%); Class 1, 17,086 (5.8%); Class 2, 7,560 (2.6%); Class 3, 10,946 (3.7%); Class 4, 24,406 (8.3%); Class 5, 2,026 (0.7%); Class 6, 2,828 (1.0%); Class 7, 14,502 (5.0%); Class 8, 16,372 (5.6%); and Class 9, 26,209 (8.9%). Notably, variations were observed in age, mental status, and inflammatory markers (e.g., CRP and lactate) among different clusters. These findings suggest that the identified latent classes captured meaningful clinical heterogeneity within the cohort (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\u003ch2\u003eClustering\u003c/h2\u003e\u003cp\u003eThe patterns of the 10 latent classes based on composite events were visualized over a 5-day (120-hour) period (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Graphs for CPR, death, and ICU transfer are provided in the supplementary files (Supplementary Fig.\u0026nbsp;1\u0026ndash;3).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThe distribution of the 10 latent classes and their corresponding event rates for composite events, CPR, death, and ICU transfer are summarized (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Composite events were most frequently observed in Class 5 (54.6%), followed by Class 6 (36.0%), Class 2 (33.3%), and Class 8 (20.8%). CPR events were most common in Class 7 (8.3%), followed by Class 8 (3.7%), Class 3 (1.4%), and Classes 9 and 6 (each 0.4%). Mortality rates were highest in Class 2 (51.2%), followed by Class 8 (24.4%), Class 5 (7.6%), and Class 6 (3.4%). ICU transfers occurred most frequently in Class 6 (36.7%), followed by Class 2 (20.7%), Class 8 (5.8%), and Class 4 (5.7%).\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eLatent class Distribution and Event Rates for Each Critical Event.\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"12\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c11\" colnum=\"11\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c12\" colnum=\"12\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c3\" namest=\"c1\"\u003e\u003cp\u003eComposite\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c6\" namest=\"c4\"\u003e\u003cp\u003eCPR\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c9\" namest=\"c7\"\u003e\u003cp\u003eDeath\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c12\" namest=\"c10\"\u003e\u003cp\u003eICU\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eClass\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003en(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEvent(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eClass\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003en(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eEvent(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eClass\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003en(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eEvent(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eClass\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003en(%)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003eEvent(%)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eAll\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e292,981(100.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e14,640(5.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eAll\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e292,981(100.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e1,254(0.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eAll\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e292,981(100.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e5,756(2.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eAll\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e292,981(100.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e10,745(3.7)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003e(5)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e2,026(0.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e1,107(54.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e\u003cb\u003e(7)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e1,581(0.5)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e131(8.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e\u003cb\u003e(2)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e1,741(0.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e891(51.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e\u003cb\u003e(6)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e7,528(2.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e2,761(36.7)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003e(6)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e2,828(1.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e1,019(36.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e\u003cb\u003e(8)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e3,163(1.1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e118(3.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e\u003cb\u003e(8)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e3,661(1.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e893(24.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e\u003cb\u003e(3)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e2,493(0.9)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e551(22.1)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003e(2)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e7,560(2.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e2,520(33.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e\u003cb\u003e(3)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e9,401(3.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e128(1.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e\u003cb\u003e(5)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e7,689(2.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e587(7.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e\u003cb\u003e(2)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e16,894(5.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e3,491(20.7)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e\u003cb\u003e(8)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e16,372(5.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e3,404(20.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e\u003cb\u003e(9)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e25,652(8.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e111(0.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e\u003cb\u003e(6)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e12,987(4.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e445(3.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e\u003cb\u003e(9)\u003c/b\u003e\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e3,698(1.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e214(5.8)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e24,406(8.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e1,559(6.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e14,153(4.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e59(0.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e10,751(3.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e321(3.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e16,791(5.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e960(5.7)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e14,502(5.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e843(5.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e10,623(3.6)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e42(0.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(9)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e23,994(8.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e708(3.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e11,760(4.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e451(3.8)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e10,946(3.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e624(5.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(5)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e24,304(8.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e86(0.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e26,669(9.1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e727(2.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e13,098(4.5)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e431(3.3)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e17,086(5.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e966(5.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e12,654(4.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e42(0.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e170,585(58.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e1,144(0.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(5)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e28,936(9.9)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e887(3.1)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(9)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e26,209(9.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e1,229(4.7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e170,687(58.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e500(0.3)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e20,841(7.1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e37(0.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e20,828(7.1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e586(2.8)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003e(0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e171,046(58.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e1,369(0.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e(4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e20,763(7.1)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003e37(0.2)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003e(7)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e14,063(4.8)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003e3(0.0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003e(0)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c11\"\u003e\u003cp\u003e170,955(58.4)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c12\"\u003e\u003cp\u003e422(0.3)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003ePerformance\u003c/h3\u003e\n\u003cp\u003eModel performance for predicting composite events, CPR, death, and ICU transfer is summarized (Fig.\u0026nbsp;3, Supplementary Table\u0026nbsp;1, 2). For composite events, the AI solution alone achieved an AUROC of 74.54% and an AUPRC of 6.21%, with a recall of 10.72% a specificity of 99.51%, and F\u003csub\u003e1\u003c/sub\u003e score of 17.86%. When focusing only on patients within Class 5, performance was similar (AUROC 74.99%), but recall was lower (7.56%). Including Classes 2, 5, and 6 along with AI solution further improved recall to 32.4% and the F\u003csub\u003e1\u003c/sub\u003e score or 34.54% although the AUROC decreased to 66.70%.\u003c/p\u003e\u003cp\u003eFor CPR events, the AI solution alone achieved an AUROC of 52.25% and an AUPRC of 2.35%, with a recall of 17.78%, a specificity of 98.50%, and the F\u003csub\u003e1\u003c/sub\u003e score of 7.64%. When focusing on patients in Class 7, the AUROC slightly improved to 53.95%, but recall decreased to 10.45%. Expanding to Classes 3, 7, and 8 along with AI solution improved recall to 30.54% and the F\u003csub\u003e1\u003c/sub\u003e score to 4.79%, although the AUROC decreased to 51.14%. For mortality (death) events, the AI solution alone achieved an AUROC of 66.94% and an AUPRC of 11.76%, with a recall of 30.43%, a specificity of 98.93%, and an F\u003csub\u003e1\u003c/sub\u003e score of 31.86%. Targeting patients in Class 2 achieved a higher AUROC (74.80%) but lower recall (16.25%). Including Classes 2 and 8 along with AI solution improved recall to 36.61% and the F\u003csub\u003e1\u003c/sub\u003e score to 31.86%, with a reduced AUROC of 63.49%. For ICU transfer prediction, the AI solution alone achieved an AUROC of 60.06% and an AUPRC of 2.32%, with a recall of 6.53%, a specificity of 99.19%, and an F\u003csub\u003e1\u003c/sub\u003e score of 10.23%. Focusing on Class 6 alone improved AUROC to 66.94% and recall to 25.67%. Expanding to Classes 3, and 6 with AI solution achieved the highest recall (32.03%) and an F\u003csub\u003e1\u003c/sub\u003e score of 31.04%, with a modest AUROC of 63.73%.\u003c/p\u003e\u003cp\u003eTo examine whether performance varies depending on the threshold setting, a sensitivity analysis was conducted to compare results obtained under different thresholds with those derived from the proposed AI and clustering-based approach. Based on the F\u003csub\u003e1\u003c/sub\u003e score, which is particularly suitable for disease prediction, the proposed method demonstrated higher performance.\u003c/p\u003e\n\u003ch3\u003eAlarm\u003c/h3\u003e\n\u003cp\u003eThe alarm characteristics for composite events, CPR, death, and ICU transfer are summarized (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e4\u003c/span\u003e, Supplementary Table\u0026nbsp;3). For composite events, the AI solution alone generated 2,928 alarms, with 53.59% identified as appropriate and 46.41% classified as false alarms. Restricting to Class 5 produced 2,026 alarms with a similar appropriateness rate of 54.64% and a false alarm rate of 45.36%. Including Classes 2, 5, and 6 along with the AI solution further increased the total alarms to 12,858, with a decreased appropriateness of 36.93% and a false alarm rate of 63.07%. For CPR events, the AI solution alone generated 4,585 alarms, with only 4.86% identified as appropriate and 95.14% classified as false alarms. Restricting to Class 7 produced 1,581 alarms with a slightly higher appropriateness rate of 8.29% and a false alarm rate of 91.71%. Including Classes 3, 7, and 8 along with the AI solution increased the total alarms to 14,749, while the appropriateness rate decreased to 2.60% and the false alarm rate rose to 97.40%. For mortality (death) events, the AI solution alone generated 4,738 alarms, with 35.20% identified as appropriate and 64.80% classified as false alarms. Restricting to Class 2 produced 1,741 alarms with an appropriateness rate of 51.18% and a false alarm rate of 48.82%. Including Classes 2 and 8 along with the AI solution increased the total alarms to 7,116, while the appropriateness rate decreased to 28.20% and the false alarm rate increased to 71.80%. For ICU transfer, the AI solution alone generated 2,976 alarms, with 23.59% identified as appropriate and 76.41% classified as false alarms. Restricting to Class 6 produced 7,528 alarms with a higher appropriateness rate of 36.68% and a false alarm rate of 63.32%. Including Classes 3 and 6 along with the AI solution increased the total alarms to 11,458, with an appropriateness rate of 30.06% and a false alarm rate of 69.94%.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn recent years, the development and evaluation of AI-based hardware and software targeting hospitalized patients have been actively pursued in clinical fields. Particularly following regulatory approvals, there has been a surge in real-world validation studies conducted prior to insurance reimbursement [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. At this hospital, an AI solution aimed at reducing critical events via a rapid response system has been implemented since November 2022. Despite initial expectations, real-world clinical utility, as perceived by frontline clinicians, appeared limited after deployment. One major issue was a mismatch in the at-risk population. Since the AI solution was applied universally to all inpatients\u0026mdash;including those in hospice care or with do-not-resuscitate (DNR) orders\u0026mdash;the predictive performance for unforeseen critical events was inevitably diluted.\u003c/p\u003e\u003cp\u003eWhile the development of a new prediction model to overcome these limitations would be ideal, this study instead focused on analyzing real-time AI prediction outputs to extract patterns that could provide more actionable and clinically relevant information. Recognizing that meaningful pattern detection requires a certain observation period, we designed a model based on 5-day post-admission data rather than immediate prediction. Although the proposed approach may not outperform existing models in terms of immediate prediction accuracy, it offers potential value by providing additional information beyond five days of hospitalization [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e\u003cp\u003eFurthermore, conventional AI solutions tend to prioritize specificity and precision, focusing only on clear and definitive predictions [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. However, in clinical practice, even rare critical events are significant, given their profound impacts on patients, families, and healthcare staff [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. In this context, strategies that minimize alarms and prioritize only highly certain events may not be appropriate. Thus, rather than aiming to replace existing AI models, our approach seeks to supplement them by applying a latent class analysis to increase recall [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. Even though this strategy may involve greater manpower, time, and resource consumption, it offers clinicians additional information that can guide decision-making without disregarding low-frequency but high-impact events [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. This study utilized data collected from April 2019 to June 2024. Although hospital data were available from 2012, we restricted our analysis to post-2019, when the hospital\u0026rsquo;s rapid response system was actively introduced, ensuring more consistent monitoring practices. It should be noted that although the AI solution was officially introduced in November 2022, retrospective prediction outputs were calculated for the period from 2019 to 2022 for the purpose of data completeness, which may introduce some unknown bias. Additionally, because this approach relies on a 5-day post-admission patterning, it is not designed for immediate event prediction within the early hospitalization period, unlike most conventional models that rely on short-term windows before discharge or event occurrence.\u003c/p\u003e\u003cp\u003eExternal validation using data from other institutions was not conducted, which remains a key limitation of this study. As a result, the generalizability of the findings to other clinical environments remains uncertain. Nevertheless, considering the common issue of domain dependency in AI prediction models, providing additional institution-specific information tailored to each hospital\u0026rsquo;s operational context may help address the heterogeneity across different sites. Rather than proposing a new prediction model, the goal of this study was to offer a framework for enhancing existing AI solutions by supplying supplementary information based on real-time patterns. We anticipate that incorporating this analytical approach into AI solution graphical user interfaces (GUI) or hospital EMR systems may help mitigate the known limitations of current AI models, improving their clinical utility.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study investigated a novel approach for supplementing AI-based early warning systems in hospitalized patients by utilizing real-time prediction pattern analysis over a 5-day observation period.\u003c/p\u003e\u003cp\u003eRather than aiming to replace existing AI solutions, we proposed a framework to enhance their clinical utility by offering additional information through latent class analysis, thereby improving recall without compromising decision-making in critical care settings. Although the proposed method requires a longer observational window and may generate a higher number of alarms, it emphasizes the clinical importance of detecting rare but high-impact events. Incorporating this strategy into existing AI solution interfaces or hospital EMR systems may contribute to overcoming current limitations in specificity-focused models and better align predictive tools with real-world clinical needs.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualization, Validation: Lim, Lee; Data curation: Lim, Hong; Methodology, formal analysis, and investigation: Lim, Hong; Funding acquisition: Lim; Project administration: Lim\u003c/p\u003e\n\u003cp\u003eResources: Lim; Supervision, Visualization: Lim, Seo; Writing-original draft: All authors; Writing-review \u0026amp; editing: All authors\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported by an NHIS (National Health Insurance Service) Ilsan Hospital grant (NHIMC-2024-CR-077).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDue to the data protection policy of Ilsan Hospital, the datasets cannot be shared publicly.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe requirement for informed consent was waived by the Institutional Review Board (IRB). The authors declare that they have no conflict of interest with the NHIS. This study was approved by the Institutional Review Board of NHIS Ilsan Hospital (NHIMC 2024-10-014-002).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor details\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e1\u003c/sup\u003e Department of Research and Analysis, National Health Insurance Service Ilsan Hospital, Goyang, Republic of Korea\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003ePimentel MAF, Redfern OC, Malycha J, et al. Detecting deteriorating patients in the hospital: A comparison of early warning scores with machine learning models. J Crit Care. 2020;57:156\u0026ndash;61.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChromik J, Flint AR, Prendke M, Arnrich B, Poncette AS. Enabling machine learning models in alarm fatigue research: Creation of a large relevance-annotated oxygen saturation alarm data set. Comput Biol Med. 2024;183:109244.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGallo R, Wang J, Choppala S, et al. Effectiveness of an artificial intelligence\u0026ndash;enabled intervention for inpatient clinical deterioration. JAMA Intern Med. 2024;184(9):1137.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eG\u0026ouml;rges M, Markewitz BA, Westenskow DR. Improving Alarm Performance in the Medical Intensive Care Unit Using Delays and Clinical Context. Anesth Analg. 2009;108(5):1546\u0026ndash;52.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAu-Yeung WM, Sahani AK, Isselbacher EM, Armoundas AA. Reduction of false alarms in the intensive care unit using an optimized machine learning based approach. NPJ Digit Med. 2019;2:86.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMinistry of Health and Welfare. Guidelines for the Pilot Program of the Rapid Response System. (Revised on April 1, 2023).\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCvach M. Monitor alarm fatigue: An integrative review. Biomed Instrum Technol. 2012;26(4):214\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBen-Hur A, Guyon I. Detecting stable clusters using principal component analysis. Methods Mol Biol. 2003;224:159\u0026ndash;82.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLiu Z, Yu L, Hsiao JH, Chan AB. Parametric manifold learning of Gaussian mixture models. Proc Int Joint Conf Artif Intell. 2019;28:3079\u0026ndash;85.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWang Y, Zhang Y, Zhang L, et al. A principal component analysis-boosted dynamic Gaussian mixture clustering model for ignition factors of Brazil\u0026rsquo;s rainforests. IEEE Access. 2021;9:144835\u0026ndash;47.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eClifford GD, Liu C, Moody B, Lehman LH, Silva I, Li Q, et al. AF Classification from a Short Single Lead ECG Recording: The PhysioNet/Computing in Cardiology Challenge 2017. Comput Cardiol. 2017;44:1\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePython 3 Reference Manual. https://\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003edocs.python.org/3/reference/\u003c/span\u003e\u003cspan address=\"http://docs.python.org/3/reference/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e Accessed 2023-01-19.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAbadi M, Agarwal A, Barham P, Brevo E, Chen Z, Citro C et al. TensorFlow: Large-scale machine learning on heterogeneous systems. (2015). Accessed 01\u0026ndash;19.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Table 1","content":"\u003cp\u003eTable 1 is available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"A.I. solution, Latent class pattern, machine learning, deep learning, Prediction performance","lastPublishedDoi":"10.21203/rs.3.rs-6866987/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6866987/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e\u003cp\u003eThe AI solution used in the hospital's Rapid Response System is designed to predict critical events such as cardiopulmonary resuscitation (CPR), unexpected deaths, and unanticipated transfers to the intensive care unit (ICU) in real time. However, given the limitations perceived by the clinical staff regarding its effectiveness, we considered whether it could provide additional information to support decision-making.\u003c/p\u003e\u003ch2\u003eObjective\u003c/h2\u003e\u003cp\u003eThis study supplemented decision-making support by utilizing predictive patterns produced by a critical event AI solution.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e\u003cp\u003eThis study assessed the predictive performance of three decision-making strategies using data from Ilsan Hospital (n\u0026thinsp;=\u0026thinsp;292,981; admissions between April 1, 2019, and February 28, 2024): a cut-off value\u0026ndash;based strategy utilizing AI solution outputs, a clustering-based strategy categorizing patients into 10 classes according to early prediction patterns during the first five days of hospitalization, and a hybrid strategy combining both approaches.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e\u003cp\u003eAlthough the AI solution\u0026ndash;based decision-making approach yielded the highest AUROC (74.54%), its AUPRC (6.21%), F\u003csub\u003e1\u003c/sub\u003e score (17.86), and recall (10.72%) were relatively low. Including Classes 2, 5, and 6 along with AI solution further improved recall to 32.4% and the F\u003csub\u003e1\u003c/sub\u003e score or 34.54% although the AUROC decreased to 66.70%.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e\u003cp\u003eSupplementing the existing AI solution\u0026ndash;based decision-making process with pattern-based predictions over a defined observation period may enhance the practical utility and real-world applicability of clinical decision-making in hospital settings.\u003c/p\u003e","manuscriptTitle":"Enhancing Clinical Decision-Making in Rapid Response Systems: Integrating AI-Based Predictions and Early Pattern Clustering","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-18 12:36:30","doi":"10.21203/rs.3.rs-6866987/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ce15f039-feb4-41a7-8e5b-2dadbd2eeb31","owner":[],"postedDate":"July 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-08-13T09:54:02+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-18 12:36:30","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6866987","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6866987","identity":"rs-6866987","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}