Natural Language Processing to Build a Multicenter Computable Phenotype Library for Adults with Congenital Heart Disease

Objective Our objective was to build classifiers for multiple phenotypes that categorize a cohort of adults with congenital heart disease (ACHD), that can be used to populate variables in a biobank.

A dataset of 1492 ACHD patients, with expert-created labels for eight phenotypes, was created and used to train classifiers with three different architectures. A larger unlabeled dataset containing 15869 patients was used to pre-train the classifiers, and a 20% subset of the unlabeled dataset was used to validate the classifier predictions.

On held out labeled data, F1 scores for the eight target phenotypes of interest ranged from 0.66 to 1. Of those, the six phenotypes with best classification performance were then validated on unlabeled data, where positive predictive value ranged from 81.5% to 100%.

We were able to classify six out of eight phenotypes with satisfactory performance. Challenging phenotypes included cyanosis and New York Heart Association functional class. Both vary over time and in the latter case there is limited agreement between human observers. Different phenotypes benefited from different model architectures to some degree, but the differences are small enough that uniformity of deployment may be a more important factor in choosing what models to deploy. We saw no benefit to joint training, but some phenotypes benefited from a multiclass model.

Human-curated data can be used to train NTLP-based ACHD phenotype classifiers with excellent test characteristics acceptable for application in quality improvement efforts and to populate ACHD registry data. Competing Interest Statement DL has worked as a consultant for TRPV Pharmaceuticals, Inc. TM is on the scientific advisory board for Lavita.ai. Funding Statement This work was conducted with support from: Harvard Catalyst/The Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award UL1 TR001102, Harvard University, and its affiliated academic healthcare centers), the Heart Institute Research Core (HIRC) at Cincinnati Children's Hospital, and National Heart, Lung and Blood Institute award R01HL151604. ARO and DML were supported by the Samuel and Molly Kaplan Fund for Adult Congenital Heart Disease. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The study was approved by Boston Children's Hospital's Institutional Review Board, and there was a formal reliance agreement with the Partners HealthCare/Brigham and Women's Hospital Institutional Review Board. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Footnotes ↵# ARO and TAM are co-senior authors. Funding sources: This work was conducted with support from: Harvard Catalyst/The Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award UL1 TR001102, Harvard University, and its affiliated academic healthcare centers), the Heart Institute Research Core (HIRC) at Cincinnati Children’s Hospital, and National Heart, Lung and Blood Institute award R01HL151604. ARO and DML were supported by the Samuel and Molly Kaplan Fund for Adult Congenital Heart Disease. Data Availability The primary data produced in the present study is not able to be made publicly available, because it contains patient details that cannot be legally or ethically shared. Abbreviations - AA - history of atrial arrhythmia - ACHD - adult congenital heart disease - BERT - Bidirectional Encoder Representations from Transformers - CNN - Convolutional neural network - HierCNLPT+FT - Hierarchical clinical natural language transformers model with fine tuning - CHD - congenital heart disease - PH - pulmonary hypertension - NYHA FC - New York Heart Association functional class - TGA - Transposition of the great arteries - cc-TGA - congenitally corrected transposition of the great arteries - d-TGA - dextro-TGA (complete transposition of the great arteries) - MRN - medical record number - SVM TF-IDF - Support vector machine classifier with term frequency-inverse document frequency-based features