BERT-based model to predict cardiovascular disease by analyzing healthcare utilization behavior of patients newly diagnosed with metabolic diseases

Background: The rise in cardiovascular disease (CVD) worldwide is causing enormous social and economic costs. Accordingly, the field of precision medicine aims to improve care through personalized prediction and prevention. In South Korea, we have health insurance claims data (HIRA) covering almost every citizen, which provides all the information about healthcare utilization behavior. Health insurance users can access their data through a simple authentication process. This data can be used to predict their personalized risk factors. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous success in the natural language processing domain. We adapt the BERT framework originally developed for the text domain to the structured HIRA data. The study aimed to predict secondary cardiovascular diseases in patients with newly diagnosed metabolic diseases (hypertension, diabetes, hyperlipidemia) using HIRA claims data and BERT. Methods: Each disease was assigned to the training, validation, and test sets in the ratio of 7:2:1 through data augmentation. Patients' diagnoses and prescribed medications were embedded as input sequences, and age was used for positional encoding to distinguish visits. The predictive ability of the model was evaluated by measuring the area under curve(AUC). Result: In each group of patients diagnosed with hypertension, diabetes, and dyslipidemia, BERT achieved mean receiver operating characteristic curve (AUC) areas of 97.9%, 97.8%, and 97.8%, respectively. We found that the top-ranked conditions for self-attendance were hypertension, diabetes, dyslipidemia, and diagnoses and medications that are more common in older adults. Conclusion: BERT performs good cardiovascular disease prediction using only diagnosis names and medication prescriptions on a relatively small training dataset. This study suggests that BERT can be used to advance personalized predictive healthcare models and patient care.