Choosing variant interpretation tools for clinical applications: context matters

Our inability to solve the Variant Interpretation Problem (VIP) has become a bottleneck in the biomedical/clinical application of Next-Generation Sequencing. This situation has favored the development and use of bioinformatics tools for the VIP. However, choosing the optimal tool for our purposes is difficult because of the high variability of clinical contexts across and within countries. Here, we introduce the use of cost models as a new approach to compare pathogenicity predictors that considers clinical context. An interesting feature of this approach, absent in standard performance measures, is that it treats pathogenicity predictors as rejection classifiers. These classifiers, commonly found in machine learning applications to healthcare, reject low-confidence predictions. Finally, to explore whether context has any impact on predictor selection, we have developed a computational procedure that solves the problem of comparing an arbitrary number of tools across all possible clinical scenarios. We illustrate our approach using a set of seventeen pathogenicity predictors for missense variants. Our results show that there is no optimal predictor for all possible clinical scenarios. We also find that considering rejection gives a view of classifiers contrasting with that of standard performance measures. The Python code for comparing pathogenicity predictors across the clinical space using cost models is available to any interested user at: https://github.com/ClinicalTranslationalBioinformatics/clinical_space_partition Summaries Josu Aguirre earned his doctorate at the Clinical and Translational Bioinformatics group, at the Vall d’Hebron Institute of Research (VHIR). Natàlia Padilla earned is a post-doctoral researcher at the Clinical and Translational Bioinformatics group, at the Vall d’Hebron Institute of Research (VHIR). Selen Özkan is a Ph.D. student at the Clinical and Translational Bioinformatics group, at the Vall d’Hebron Institute of Research (VHIR). Casandra Riera earned her doctorate at the Clinical and Translational Bioinformatics group, at the Vall d’Hebron Institute of Research (VHIR). Lidia Feliubadalo earned her doctorate at the Universitat de Barcelona, presently she is a high-level technician working at the Catalan Institute of Oncology (ICO) in the diagnosis of hereditary cancers. Xavier de la Cruz is ICREA Research Professor at the Vall d’Hebron Institute of Research (VHIR). His research interests revolve around the application of machine learning methods to healthcare problems.