Natural Language Processing as a Predictor of Mortality in Intensive Care Unit Patients

Background: Artificial intelligence, specifically machine learning, has led to a series of medical publications in recent years. Several studies have used physiologic parameters with machine learning algorithms to derive robust clinical prediction tools. The goal of this study was to use natural language processing in conjunction with physiologic parameters to analyze clinical text data and create a clinical prediction tool. Research Question: Can a natural language processing machine learning based clinical prediction tool accurately predict ICU mortality with relatively limited sample size? Study Design and Methods: Three machine learning classifiers including a support vector machine (SVM), XGBoost Tree, and logistic regression were used to determine probability of hospital mortality in patients admitted to the intensive care unit (ICU) using physician notes and patient physiologic parameters. Discrimination for the binary outcome of life or death will be measured using receiver operating characteristic-area under the curve (AUC). Calibration will be assessed using chi-square goodness of fit. Results: : 7,555 patients were available for analysis with 5,288 being included in the derivation set and 2,267 in the validation set. Using the SVM algorithm, the AUC for hospital mortality was 0.895 (95% CI, 0.850-0.940). Calibration was also acceptable with chi square of 7.94 and p = 0.54. The XGBoost Tree algorithm resulted in the best discrimination for mortality with an AUC of 0.912 (95% CI, 0.881-0.943), but calibration was poor. Logistic regression resulted in an AUC of 0.868 (95% CI, 0.827-0.909) but calibration was also poor. Conclusions: : A support vector machine learning algorithm can use patient chart data, lab values, and physiologic parameters to generate a clinical prediction tool to predict mortality in ICU patients with a relatively limited dataset. For this study, SVM was superior to logistic regression which is used in many traditional ICU risk predictors.