Using Machine Learning and Natural Language Processing in Triage for Prediction of Clinical Disposition in the Emergency Department

Abstract BACKGROUND: Accurate triage is required for efficient allocation of resources and to decrease patients’ length of stay. Triage decisions are often subjective and vary by provider, leading to patients being over-triaged or under-triaged. This study developed machine learning models that incorporated natural language processing to predict patient disposition. The models were assessed by comparing their performance with the judgements of emergency physicians (EPs). METHOD: This retrospective study obtained data from patients visiting EDs between January 2018 and December 2019. Internal validation data came from China Medical University Hospital (CMUH), while external validation data were obtained from Asia University Hospital (AUH). Nontrauma patients aged ≥ 20 years were included. The models were trained using structured data and unstructured data (free-text notes). The primary outcome was death in the ED or admission to the intensive care unit, and the secondary outcome was either admission to a general ward or transferal to another hospital. Six machine learning models (CatBoost, light gradient boosting machine, logistic regression, random forest classifier, extremely randomized trees, and gradient boosting models) and one logistic regression model were developed and evaluated using EPs' predictions as reference. RESULT: A total of 168,235 and 27,645 patients were enrolled from CMUH and AUH, respectively. EPs achieved AUC values of 0.752 and 0.659 for the primary and secondary outcomes, respectively. Among the machine learning models, CatBoost exhibited the highest AUCs (0.935 for primary outcome and 0.857 for secondary outcome) and F1 scores (0.488 for primary outcome and 0.610 for secondary outcome). In external validation, gradient boosting achieved the highest AUC (0.933) for the primary outcome, while light gradient boosting machine achieved the highest AUC (0.817) for the secondary outcome. CONCLUSION: The machine learning models outperformed the reference model in predicting clinical outcomes. Integrating these models into triage systems may enhance triage processes and resource allocation.