Development of an accessible 10-year Digital CArdioVAscular (DiCAVA) risk assessment: a UK Biobank study.

Nikola Dolezalova works in the Innovation & AI team at Huma Therapeutics as a Clinical Data Scientist. In 2019, she completed her PhD at the University of Cambridge, Department of Surgery on cryopreservation of pancreatic islets and then worked as a post-doctoral Research Associate, researching anti-inflammatory reflex, and a bioelectronic therapy for inflammatory diseases in collaboration with Galvani Bioelectronics.

The UK Biobank (UKB) dataset is a prospective cohort study of 502 488 UK participants 18 recruited between 2006 and 2010. It comprises baseline information from an initial assessment on participants who were subsequently followed up via additional assessment sessions, medical records, and other health-related records. Use of data for this study was approved by UKB (application number 55668). The sole exclusion criterion was a pre-existing diagnosis of a CVD (defined identically to the outcome definition below, with non-limited look-back). Censoring was done at the date of first CVD diagnosis, when lost to follow-up due to death or other reasons, or at the date of last available update on the data (30 September 2020), whichever came first. Definition of CVD in this study includes myocardial infarction, heart failure, angina pectoris, stroke, and transient ischaemic attack. A diagnosis of CVD was confirmed using several fields of UKB-defined First Occurrences (includes primary care, hospital admissions, death register data, and self-report), Algorithmically defined outcomes (hospital admissions, death register data, and self-report), and remaining ICD-10 codes from inpatient records. The full list of UKB fields used for outcome definition can be found in Supplementary material online, Table S2 . While there was >75% overlap between the Algorithmically defined outcomes and the corresponding fields for first occurrences, taking into account both fields ensured a more comprehensive selection of CVD cases. Earliest date for any cardiovascular disease on record was used for each participant. For the baseline set of variables, we used UKB variables available for the majority of participants. This dataset was further enriched by including first occurrences fields for all available ICD-10 (10th revision of the International Statistical Classification of Diseases and Related Health Problems) codes and hospital inpatient records for remaining ICD-10 codes ( Supplementary material online, Table S3 ). Diagnoses with <0.2% occurrence in the dataset were excluded from the feature list. Considering our use-case, a key determinant was the possibility of its input or assessment using only a smartphone and the possibility to apply these findings to other countries. This led to exclusion of blood tests and other biological measurements which cannot be acquired via smartphone, as well as UK-specific variables, such as Townsend index, income, and certain qualification levels. The exceptions were total cholesterol, HDL cholesterol, and blood pressure measurements owing to their significant predictive value in CVD risk. Analysis of data completeness is provided in Supplementary material online, Table S3 . The majority of predictor variables were used as provided by the UKB, with the addition of several variables which were derived from the data (waist-to-hip circumference ratio, total cholesterol/HDL cholesterol ratio, and total alcohol intake). Imputation of missing values was performed by substitution with the mean. Binary variables for pre-existing diseases were derived from a combination of corresponding fields in first occurrences, self-reported medical conditions, medications, and in-patient hospital diagnosis, with the exclusion of cases where diagnosis occurred after the assessment date. Categorical and ordinal variables were one-hot encoded and continuous variables were scaled. Prior to training the model, the dataset was split into two parts: train dataset (75%) and test dataset (25%), stratified on the outcome. The train dataset was further split into train (75%) and validation (25%) for the purposes of DeepSurv hyperparameter search. There were no statistical differences between the different datasets (data not shown). Our modelling approach involved the use of Cox Proportional Hazard (CPH) models, implemented in the Python ‘lifelines’ library, and Cox proportional hazards deep neural network (DeepSurv), implemented in the ‘pycox’ package using PyTorch ( Supplementary material online, Table S4 ). DeepSurv models, 22 as opposed to CPH model, are able to capture non-linear interaction between variables but require large-scale hyperparameter optimization to achieve a good performance. Using the full set, and later the reduced set of features, optimal hyperparameters were searched among those described in Table 1 via Tree-Structured Parzen Estimator (TPE) algorithm 20 from the Optuna Library ( Supplementary material online, Table S4 ). TPE is a Bayesian hyperparameter optimization method which uses the results from past evaluations to build a probabilistic model used to identify the best candidate hyperparameter values for future searches. DeepSurv hyper-parameter search space The search space consisted of 10 different neural network topologies, up to three layers deep, and a choice of three activation functions for these layers. Regularization techniques included drop-out (ignores randomly selected neurons in the network) and weight decay (L2 regularization, shrinks weights). The option to utilize batch normalization was offered to accelerate training via standardizing the inputs’ changing distribution. The choice of an optimizer for gradient descent included standard stochastic gradient descent (SGD) with or without Momentum, or Adam (adaptive moment optimization). Uniform distributions. Feature selection was performed using the CPH model to decrease the risk of overfitting and ensure suitability for use in a digital solution. In the first step, univariate CPH models were trained for each of the features in the baseline model and those with P -value >0.01 were excluded. The remaining features were processed through stepwise backward elimination. In brief, in each elimination round, a batch of features was removed and performance evaluated. If the concordance index (c-index) decreased by <0.001, features were eliminated, otherwise a smaller subset was removed until the final set of features whose removal would cause performance degradation was found. These highly contributing features were then used in the reduced models. Both CPH and DeepSurv models were then trained using the final set of features, along with a model variant excluding all cholesterol variables and systolic blood pressure (substituted by heart rate for its digital feasibility), to enable calculation of risk for individuals who are not able to obtain these measurements. Feature selection using DeepSurv was not performed as a consequence of its high computational requirements and because performance of the baseline DeepSurv model was not notably superior to the CPH model. Details of variable derivation for replication of Framingham risk score on the UKB dataset is summarized in Supplementary material online, Table S5 . Framingham score for each participant was calculated using the formulas for males and females published in the original article, 8 and c-index was calculated using the predicted and actual time-to-CVD-event. Additionally, a CPH model was re-trained using the seven Framingham variables on the UKB dataset and compared to our findings. Sex-specific variants of our final models were trained to allow for closer comparison to these two scores. In the summaries of cohort characteristics, participant numbers and percentages of total are shown for categorical and ordinal variables, whereas median and first and third quartiles are shown for continuous variables. Statistical comparisons were performed using the χ 2 test for categorical and ordinal variables and Kruskal–Wallis test for continuous variables. Where detailed analysis of the results of CPH models is provided, hazard ratios (HR) with 95% confidence intervals (CIs), as well as the coefficients, are provided. P -values test the null hypothesis that the coefficient of each variable is equal to zero. Significance level was set to 0.05. C-index was used as the metric for both models, with 95% CIs calculated using the percentile bootstrap resampling method (50 resampling rounds). Where detailed analysis of the results of CPH models is provided, log(HR) with 95% CIs are shown. P -values test the null hypothesis that the coefficient of each variable is equal to zero and significance level was set to 0.05. Calibration was evaluated at the 10-year time point using calibration plots and the integrated calibration index (ICI), which is a mean weighted difference between observed and predicted probabilities, implemented in the Python lifelines library. Time-specific evaluation of model discrimination was done using the cumulative/dynamic area under the ROC curve (AUC C/D ), implemented in the scikit-survival package. This article was written following the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines, which are further elaborated in Supplementary material online, Table S6 .

After exclusion of participants with pre-existing CVD, the study was conducted with 466 052 UKB participants. There were 42 377 participants (9.09%) who developed CVD during the observation period. The most common CVDs were chronic ischaemic heart disease (42.1%), myocardial infarction (20.3%), and stroke (15.4%). The recruitment and data funnel is illustrated in Figure 1 . The full breakdown can be found in Supplementary material online, Table S2 . Median follow-up time was 11.5 years (IQR 10.7–12.3 years) and maximum follow-up time was 14.6 years. The studied dataset includes 44.1% men and 55.9% women (sex-specific demographic data shown in Supplementary material online, Table S7 ), aged 37–73 at the time of recruitment (mean 56.2 ± 8.09). The participants were predominantly white, with a small proportion of other ethnic groups (2.2% Asian, 1.6% Black, and 1.5% Others). Detailed demographic analysis for the variables selected in the final model can be found in Table 2 . Summary of demographic characteristics of the studied cohort grouped by the outcomes Last column shows P -value after comparing the incident CVD group with the non-CVD group. Comparisons were performed using the χ 2 test for categories and Kruskal–Wallis test for continuous variables. This variable is used only as a substitute for systolic blood pressure in the model excluding cholesterol and systolic blood pressure. CPH and DeepSurv models were trained using the pre-processed dataset, containing 608 features. This baseline CPH model achieved a c-index of 0.7431 (95% CI 0.7422–0.7441) on the test dataset. DeepSurv, with optimized hyperparameters ( Supplementary material online, Table S8 ), achieved a c-index of 0.7461 (95% CI 0.7452–0.7469) on the test dataset. The CPH model was used for the subsequent feature selection from this baseline dataset. Initially, 93 features were eliminated based on a P -value >0.01 in a univariate CPH model. The subsequent stepwise backward elimination excluded a further 465 features. The remaining 50 features were subjected to review by a clinician, further excluding three features (First Occurrence of ICD-10 codes F17 and H25, lamb/mutton intake 2–4 times a week). The remaining 47 features in the final reduced models include demographic measures (age, sex, holding a university degree), anthropometrics (waist-to-hip ratio), systolic blood pressure, total cholesterol, cholesterol ratio, range of pre-existing conditions, medications, symptoms, family history of heart disease, lifestyle measures (e.g. smoking, alcohol consumption), and self-rated health. A CPH model trained using these features achieved a c-index 0.7443 (95% CI 0.7441–0.7445) on the test dataset. A DeepSurv model showed concordance of 0.7446 (95% CI 0.7441–0.7452) on the test dataset. The AUC CD calculated at years 1 through 10 showed above-average discrimination at years 1–5 and stable AUC of ∼0.76 from year 6 onwards ( Figure 3A and B ). Sensitivity analysis ( Supplementary material online, Figure S1 ) revealed that features representing symptoms (e.g. abdominal pain, wheeze in chest) contribute more to shorter term predictions (1–3 years). The details of CPH feature coefficients and statistical analysis can be found in Figure 2 and Supplementary material online, Table S9 . Based on the P -value calculated in the CPH model, the top five risk factors include age, systolic blood pressure, diagnosis of atrial fibrillation and flutter (ICD-10 code I48), cholesterol ratio, and father with heart disease. The top five protective features include being a female, non-smoker, not experiencing any chest pain, brisk usual walking pace, and excellent self-rated health. Numbers of participants included in the study with the breakdown of CVD incidence in the train and test datasets. Plot of cox proportional hazards model coefficients for the general population. Values are shown as log(HR) ± 95% CI. CI, confidence interval; HR, hazard ratio. The CPH model showed good calibration, with slight overestimation of the higher probabilities which were more sparsely represented in the dataset at 10 years ( Figure 3C ). The DeepSurv model slightly overestimated lower probabilities at 10 years and underestimated higher probabilities at 5 years ( Figure 3D ). The integrated calibration index (ICI) was 0.00295 for the CPH model and 0.00567 for DeepSurv model (details in Supplementary material online, Table S10 ). Time-specific performance and calibration plots for CPH and DeepSurv models. AUC C/D was calculated for years 1 through 10 using the CPH ( A ) or the DeepSurv ( B ) model. Calibration plots for probabilities of developing CVD at year 1, 5, or 10 using the CPH ( C ) or the DeepSurv ( D ) model. Smoothed calibration curves and histograms of the predicted probabilities of developing CVD are shown in the respective colours for each evaluated time point. ICI, integrated calibration index. To enable calculation of risk scores for individuals without access to cholesterol and blood pressure measurements, the models were re-trained after excluding total cholesterol and cholesterol ratio. Heart rate replaced systolic blood pressure as it is measurable via mobile device. The c-index of the CPH model decreased to 0.741 and c-index of the DeepSurv model decreased to 0.739 ( Table 3 ). C-index of trained models for test dataset predictions A 95% confidence interval is shown in square brackets. Cholesterol measurements include total cholesterol and cholesterol ratio, systolic blood pressure was substituted with heart rate. BP, blood pressure. Established risk scores, such as Framingham, provide separate models for males and females. To provide closer comparison to these scores, we trained sex-specific models using our final set of variables (excluding sex). The c-index of the male-specific model (both CPH and DeepSurv) was 0.72 (detailed results in Supplementary material online, Table S11 ) and the female-specific model was 0.75 (detailed results in Supplementary material online, Table S12 ). Risk scores calculated using the Framingham formula achieved a c-index of 0.68 and 0.70 for male and females, respectively. The c-indices rose slightly after re-training the CPH model using the seven Framingham variables on the UKB dataset. Comparison of all c-indices can be found in Table 4 . Results of comparison of our model for Framingham score Test dataset c-indices was shown. 95% confidence intervals are shown in the square brackets. UKB, UK Biobank.

Through our investigation of the UKB’s sizable dataset, we were able to develop a model with a c-index of 0.745, showing very good predictive ability for CVD events over a 10-year period. The main features in the model are well-established across other risk scores: age, sex, hypertensive medication, systolic blood pressure, smoking status, and cholesterol. However, minimal differences were observed when cholesterol-related variables were excluded (0.744 vs. 0.741 using the Cox model), strengthening the case for use of our model in a remote setting and without mandatory cholesterol screening. The clinical demand for this exclusion is illustrated by the incorporation of a non-laboratory-based score along with the original Framingham CVD risk score. 8 The observed modest decrease in c-index after exclusion of cholesterol may be explained by multicollinearity with other factors with high importance in our model: waist-to-height ratio, smoking status, family history, and hypertensive medication. Anthropometric measures, while traditionally viewed as valued parameters only of obesity and diabetes, 30 are gaining traction in risk modelling across a wider range of non-communicable and communicable diseases alike. 31 The expanding impact of the digital revolution on healthcare makes these features especially relevant owing to novel, accessible technologies which can capture a broad range of anthropometric information using only a smartphone camera. 32 Alcohol consumption and smoking habits, while traditional factors, present in an interesting manner. For the former, as categorical UKB fields were one-hot encoded, only alcohol abstinence passed feature selection, with other degrees of consumption not significant. The detrimental predictive significance of alcohol abstinence can be attributed to the average poorer health status and higher prevalence of chronic conditions and neurological problems of never- and former-drinkers when compared to ever-drinkers. 33 For tobacco, as expected, currently not smoking is protective and increasing pack years confers increased risk of CVD. However, having smoked occasionally was unexpectedly calculated as protective, likely as this group is enriched for people who have since ceased smoking, having never smoked regularly. In addition to traditional risk factors and anthropometric variables, several other variables not present in any conventional risk model have shown to be significant contributors to our model. Notably, the inclusion of reported symptoms in our analysis sets the resultant model apart from others. The symptoms of dyspnoea, syncope, dizziness, wheezing, and chest pain are recognized as clinical indicators of possible cardiac disease. Our model supports their potential to highlight subclinical, underlying cardiovascular pathology. Similarly, abdominal and pelvic pain may represent undiagnosed comorbidity associated with our CVD outcomes, including inflammatory bowel disease, endometriosis, or aortic aneurysm. 34 , 35 We performed a sensitivity analysis to investigate the influence of these features for short- and long-term predictions which revealed that removing these features mostly impairs shorter term predictions. The extent of one’s education has also emerged as an important contributing social factor in our model, reinforcing the findings of recent studies which show education as a significant predictor of CVD acquisition and mortality. 36 Furthermore, several diagnoses not traditionally associated with the incidence of CVD were found to be significant contributors to the CPH model. A diagnosis of urinary tract infections (UTI) or incontinence, as well as epilepsy, are significant contributors independent of all other features. UTI and incontinence may be explained by the raised cardiovascular risk of the proinflammatory state, but warrants further investigation. Epidemiological studies have shown a raised CVD risk profile in those with epilepsy, but this is mostly attributed to concomitant risk factors. 37 Self-rated health and prior diagnosis of a depressive episode, both associated with increased CVD risk, 38 , 39 are also significant features, stressing the assessment of psychological aspects of health when calculating risk for CVDs. To a smaller, but still important extent, two physical activity-related behaviours—walking pace and hours spent outdoors (in winter)—have contributed to the performance of the model, both known to influence the risk of CVD-related events. 40 , 41 Lastly, a family history of heart disease showed strong predictive power, with sibling heart disease ranking significantly higher than parental. This is likely as a sibling diagnosis has been caused both by similar genetic and environmental factors, whereas parental diagnoses reflect pertinent environmental contributors to a lesser extent. Despite its well-known contribution to CVD occurrence, the QRISK3 is the only risk model that currently incorporates family history in the calculation, but our results show that more granular coverage of family history could lead to better risk assessment. 7 By using CPH and Deepsurv to develop risk scores, we compared an established statistical method with a more novel machine learning approach to survival analysis. We hypothesized that use of Deepsurv might result in superior predictions by identifying complex interactions between variables. However, performances of both models were similar, implying a minor contribution of non-linear interactions within the large feature space, or insufficient number of training examples to give advantage to a more data-intensive deep learning model. In the case of equal performance of both models, CPH models provide several advantages: they are less computationally intensive and easier to implement and update with new data or recalibrate for other demographics owing to a low number of pre-set hyperparameters. Importantly, CPH models also provide easily interpretable feature coefficients, making them more trustworthy for clinicians who may not be familiar with academic data science. The minimal performance drop when excluding the less accessible features of cholesterol and blood pressure, together with the fact that many features are modifiable, supports the feasibility of feature collection through a digital application, in a remote setting, and on a regular basis. This enables more engaging risk management and eliminates the need for interaction with a healthcare provider. Our digitally-collected score slightly overperformed Framingham risk score (c-index 0.745 vs. 0.704). Due to increasing ownership, mobile phones and smart wearables have become a priority medium for digital health interventions. The use of digitally collected features to calculate CVD risk will allow patient-facing solutions, transferring responsibility of health status and lifestyle choices to patients rather than clinical teams. Furthermore, gamification and goal setting has been shown to increase the success of lifestyle change interventions. 42 Several risk variables, particularly those related to diet (salt added to food, nutritional supplement intake), smoking, alcohol consumption, as well as activity (hours spent outdoors in winter) are adjustable in the short term, giving direct feedback to users. A patient-facing digital risk score that responds to and reports positive change in a short time span, as a consequence of such lifestyle improvements, could increase motivation in patients with high CVD risk. Furthermore, the risk score can be easily accompanied by educational content from credible sources that is tailored to support the patient along that journey, providing further support on their path to improving their cardiovascular health. The assessment of cardiovascular risk is a mainstay of clinical guidelines, 5 and the ability to calculate this remotely with very limited face-to-face clinical resources could have significant beneficial cost and patient accessibility implications. A key challenge in any digital health intervention is being able to maintain patient adherence across time, which is tackled by considering key usability and design aspects early in the development process. The number of variables incorporated into this model can be challenging for patients to recall, especially those related to non-modifiable factors, and can lead to patient drop-off over time. By creating a simple, user-friendly interface that asks clear questions that are easy to understand, we strongly believe that all of the necessary information can be comfortably obtained, especially in the setting of repeated assessments. A notable limitation of our study stems from the UKB cohort being unrepresentative of the general population across various domains. 43 First, ethnic diversity is very low with 94% of all participants identifying as White and, second, age at recruitment was restricted to 37–73. Our analysis, therefore, was not able to account for differences in CVD risk across ethnicities, which is a significant known factor for CVD incidence, nor in younger age groups, thus use of our model in these populations should be interpreted with some reserve. Importantly, the restricted age distribution also represents a higher risk population, implying validation of the model in a more representative sample would result in a higher performance, as seen when QRISK3 is calculated for UKB participants (representative population: 0.88; UKB: 0.76). 7 , 44 Last, the UKB cohort is reported to be healthier and wealthier than the general population. 43 This is a significant factor when genetic aspects account for only ∼32% of coronary artery disease occurrence, 45 thereby may result in an underappreciation of the influence of detrimental lifestyle choices. Future work will concern external validation of the model which will allow conclusions to be drawn about its use across geographies and populations. While our aim was to limit model bias by applying a data-driven approach to feature selection and excluding features with a large proportion of missing data, choosing CPH model-based feature selection may have biased the final features in the reduced model. It is likely that this has also limited the performance of the DeepSurv model, by selecting features that specifically enhanced the performance of the CPH model. In conclusion, DiCAVA, our 10-year CVD risk model, has very good predictive capacity and contains significant predictors not previously described by existing risk scores in the literature. We demonstrated its feasible utility in a remote setting where cholesterol and blood pressure measurements may not always be convenient, highlighting that even the most established predictors are not always essential.

Cardiovascular diseases (CVDs) are the most common cause of mortality globally, with 18.6 million attributed deaths in 2017. 1 The increase of CVD-related mortality in low- and middle-income countries, 2 together with increased life expectancy and a growing CVD morbidity worldwide, 3 poses a significant challenge in managing this group of diseases. Many risk factors for acquiring CVDs are well-established: hypertension, obesity, diabetes, poor physical activity, hypercholesterolaemia, smoking, alcohol intake, among others. 4 Given that up to 70% of cases and deaths from CVDs are attributed to modifiable factors, 4 primary prevention of CVD, rather than treatment, has become a mainstay of public health strategies and is extensively described in leading guidelines. 5 These strategies can be highly effective in reducing the number of occurrences and, thereby, the corresponding mortality. The use of risk scores for CVDs in clinical practice is commonplace. 6–10 Their primary use is to identify individuals who are at high risk of development of either a fatal and/or non-fatal CVD event in the next 10 years. Their goal is to highlight that risk, so it can be mitigated through either lifestyle adjustment or pharmacological treatment of associated conditions, such as hypertension and hypercholesterolaemia. These interventions have indeed been shown to be cost-effective, 11 but the pursuit of better screening programmes and economic assessments is necessary. 12 There are several limitations of the cardiovascular risk scores used in practice. In addition to overestimating the proportion of people in the high-risk category, 13 recent systematic reviews show low-quality evidence that use of existing CVD risk scores may have minimal effect on incidence of CVD events. 14 Moreover, the majority of existing risk scores utilize a small and homogenous set of non-malleable factors in their calculation ( Supplementary material online, Table S1 ). Risk factors for CVDs can be easily captured through web-based platforms, with standard practice often integrating a risk model as a function within electronic healthcare records (EHRs). 15 However, more granular insight into cardiovascular health can be achieved through smartphone-based apps and integrated questionnaires, or wearable devices that allow passive and continuous assessments. 16 Furthermore, the analysis of such data using modern methods, such as machine learning (ML), allows for deeper insight into the risk factors that contribute to CVDs, capturing non-linear variable interactions uncapturable by classical statistical methods. 17 The primary aim of our study, therefore, is to develop a CVD risk model using both traditional statistical and ML approaches, guided by clinical intuition, which is viable in a remote setting. The secondary goal is the identification of new variables which should be considered by the wider community for incorporation into future CVD risk models to improve their utility. To evidence the value of non-traditional variables, we compare our model to the Framingham risk score that is currently used in clinical practice.

Click here for additional data file.