{"paper_id":"0c736211-abc5-4e98-b7d2-5bb16cdac750","body_text":"Capturing Unanticipated Drug Toxicities Using an Ensemble Machine Learning Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Capturing Unanticipated Drug Toxicities Using an Ensemble Machine Learning Approach Nicole Zatorski, Avner Schlessinger This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6999821/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 9 You are reading this latest preprint version Abstract Despite rigorous safety evaluations during development, numerous drugs have been withdrawn from the market due to serious toxicities. Here we investigate the features found in drugs with these unanticipated toxicities and apply a machine learning approach to predict if a drug is likely to be withdrawn due to intolerable side effects without the need for human trial data. Our best preforming classifier was an ensemble predictor trained on protein targets, protein structure features, chemical fingerprints, and chemical features that achieved 92% accuracy and 0.845 Matthews Correlation Coefficient with 10-fold holdout test set cross validation. Analysis of features predictive of unanticipated toxicity revealed both known factors such as inhibition of cytochrome P450 as well as yet uninvestigated factors including the inhibition of bile salt export pumps. This predictor and subsequent feature analysis pave the way for the larger role of computational methods in screening potential candidates during drug development. Physical sciences/Chemistry Biological sciences/Computational biology and bioinformatics Biological sciences/Drug discovery Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 INTRODUCTION Unanticipated toxicities can emerge when drugs are incorporated into standard of care so numerous regulatory bodies monitor the safety of drugs for side effects after the end of their official clinical testing phase 1 – 4 . If unexpected side effects are observed, these agencies, such as the United States Federal Drug Administration (FDA) or the European Medicines Agency (EMA), can withdraw the drug from the market 1 – 4 . In addition to the impact unexpected toxicities have on patients, the decision to discontinue a drug further creates economic burden that prevents recoupment of research and development investment that increase costs of healthcare for patients 5 . Due to the difficulties resulting from unexpected toxicities and subsequent drug withdrawals, many approaches have been used in an effort to predict toxicity earlier in the development pipeline 6 . Currently, the industry standard relies on laboratory and animal testing followed by Phase 1 trials in healthy volunteers to assess toxic side effects 7 . Traditional computation toxicity prediction approaches have centered around the use of drug Quantitative Structure-Activity Relationships (QSAR) 8 , which typically involve statistical or machine learning models that characterize the relationship between chemical properties of a drug and a phenotypes associated with toxicity. These features are used, for example, to predict a drug’s likelihood of inhibiting a particular protein, such as the hERG channel in the heart, which can precipitate arrhythmias when modulated by a drug 9 . While this system is fairly robust, it fails to capture the long-term use toxicities and very rare side effects uncovered in post marketing surveillance. In addition, emerging big data analysis and machine learning techniques can be applied preclinically, during the drug development process, to provide a supplemental approach to toxicity prediction and eliminate potentially toxic drug candidates 10 . For example, deep neural network models applied to whole-genome DNA microarray data from the Open Toxicogenomics Project-Genomics Assisted Toxicity Evaluation System and DrugMatrix have predicted three liver toxicity endpoints with a range of 56–89% accuracy 11 . Cardio ToxCSM, is a webserver that predicts six cardiac toxicity outcomes from chemical drug structure representations and molecular descriptors achieving areas under the receiver operating characteristic curve (AUROC) ranging from 0.63 to 0.92 12 . ProTox-II provides predictions for 33 toxicity endpoints based on drug chemical data and 15 assayed targets 13 . These existing predictors, which are not yet used in established drug development pipelines, focus on detecting organ system or pathway toxicities rather than labeling drugs based on their likelihood of resulting in any side effects that necessitate withdrawal. Structural features of protein drug targets have great potential for bolstering toxicity predictive models. Because these features describe the underlying structures of proteins, they are more readily able to capture similarities in different proteins 14 . These structural similarities, due to the interconnectedness of structure and function, therefore allow structural features to capture functional similarities and therefore similarities in drug effects. Structural features have been used, along with machine learning approaches, to predict protein-protein interactions 15 and to construct genetic disease protein networks 16 . Protein structural features also capture patterns in normal human tissues 17 , cancer 14 , 18 , and drug perturbations 14 , 17 from gene expression and proteomics data. These features therefore, combined with current machine learning techniques, hold great potential for revealing unanticipated drug toxicities. Here we leverage data from 948 protein targets and their structural features with drug structural information to predict drug toxicity as defined by the compound’s withdrawal status. Drug withdrawal based on the recommendation of global drug safety regulatory agencies is a metric for drug toxicity that captures both extremely rare and severe drug side effects that have not been anticipated by traditional screening approaches. We begin with an examination of the different characteristics between withdrawn drugs and those still deemed safe to use. Indeed, through rigorous feature analysis, we identify drug and drug target features correlated with withdrawal drug status which can be used to inform future drug development. We then develop a predictor, named Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH), to determine the likelihood of a drug having unanticipated toxicity leading to regulatory withdrawal. Furthermore, we discuss how WATCH can be applied as a screening tool to anticipate toxicities that may emerge in drugs under investigation by applying the predictor to drugs in clinical trials. RESULTS Drug withdrawals categorized by organ system The 120 drugs, found in the ChEMBL database 19 , that have been withdrawn by regulatory agencies worldwide have targeted a variety of organ systems (Supplementary Fig. 1). Interestingly, side effects affecting three organ systems make up a majority of reasons for drug withdrawal from the market. Cardiovascular (19.5%), hepatic (19.2%), and neuropsychiatric toxicity (13.2%) are the three largest contributors to drug withdrawal (Fig. 1 ). Cardiovascular reasons for withdrawal are largely due to arrhythmias and conditions that predispose to arrhythmias: arrhythmias (12%), QT prolongation (6%), ventricular tachycardia (6%), torsade de pointes (6%), ventricular arrhythmias (5%), and repolarization abnormalities (3%) (Supplementary Fig. 2A). This is supported by the focus of early cardiotoxicity screens, which primarily investigate compound interactions with arrhythmia inducing channels such as hERG 9 . Besides general hepatotoxicity, liver failure is the second largest listed reason for hepatotoxicity (8%) (Supplementary Fig. 2B). In contrast to hepatotoxicity and cardiotoxicity, neuropsychiatric toxicity has more variety of side effects that range from abuse (21%) to central nervous system stimulation (2%) (Supplementary Fig. 2C). This reflects the wide range of observed neuropsychiatric toxicities and the difficulty in the development of in vivo screening tests which can be applied prior to lengthy animal studies 20 . Neuropsychiatric drugs are well known for their polypharmacology, where individual drugs bind many proteins, which contributes to increased frequency and severity of toxicity 21 . Comparison between withdrawn and not withdrawn drugs by indication A fisher exact test analysis of drug Anatomical Therapeutic Chemical (ATC) codes 22 , a hierarchical classification system of medicines maintained by the World Health Organization (WHO), reveals differences in target organ system and disease indication between drugs that are in use and those that have been withdrawn (Fig. 2 ). Withdrawn drugs have different frequencies of therapeutic indications from drugs that are not withdrawn for various treatment types. For example, neither the antineoplastic nor antiparasitic categories have any withdrawn drugs. Antineoplastics account for 10% of not withdrawn drugs (p = 1.66e-5) and antiparasitic drugs comprise 3% of not withdrawn drugs (p = 0.0448) (Fig. 2 ). For other ATC indications, more drugs have been withdrawn than not withdrawn. Drugs that are indicated for diseases of the alimentary and metabolic systems (22% of withdrawn and 9% of not withdrawn p = 3.63e-5), cardiovascular system (19% of withdrawn and 12% of not withdrawn p = 0.0294), dermatologic system (12% of withdrawn and 6% of not withdrawn p = 0.0331), musculo-skeletal system (22% of withdrawn and 5% of not withdrawn p = 4.40e-10), and nervous system (45% of withdrawn and 18% of not withdrawn p = 1.90e-10) have a higher frequency of withdrawals. A sub analysis of drugs actively in clinical trials reveals that the primary indications of focus are: alimentary and metabolism (16%), antineoplastic and immune (13%), cardiovascular (12%), and nervous system (13%). Features of withdrawn drugs and their protein targets To determine the characteristics that mark a drug removed from use due to toxicity, we investigated drug structural and pharmacological features through recursive feature elimination with cross validation (RFECV) 23 using a random forest model, variance threshold, and generic univariate selection (Methods). Feature selection of drug chemical features demonstrates that route of administration, formulation as a prodrug, acidity (as measured by apKa), quantitative estimate of drug-likeness 24 , drug identity as a base, number of hydrogen bond acceptors (hba), molecular weight (MW), polar surface area (tPSA), lipophilicity (i.e. logP, logD), and frequency of heavy atoms are all highly distinguishing features between withdrawn and not withdrawn drugs (Fig. 3 , Supplementary Table 1). Consistent with these results, underrepresented features in withdrawn drugs include molecular weight, frequency of heavy atoms, polar surface area, and quantitative estimate of drug-likeness (Fig. 3 ). We next applied the feature selection approach to the protein targets, including the protein name, as well as protein structural and functional features, such as secondary structure elements, disordered regions, and coiled-coils. These proteins include intended and unintended targets of the drug. The most informative drug targets include proteins involved in drug absorption, disposition, metabolism, and excretion (ADME) that are screened in drug development as part of drug toxicity target panels, such as the metabolic enzyme Cytochrome P450 (3A4, 2C19, 2J2) 25 ; and the membrane transporters 26 , 27 the ATP-binding cassette sub-family C (members 2–4), the bile salt Protein Target A Gene Name B Protein Target Name C Biological Function D Drugs with lowest IC 50 standard values E O15438 ABCC3 ATP-binding cassette sub-family C member 3 transports drugs across cell membranes Cyclosporin A, amg-009, olmesartan medoxomil O95342 ABCB11 ATP-binding cassette sub-family B member 11 transports bile salts across hepatocyte membrane Alisporivir, BDBM172715, Pioglitazone O15439 ABCC4 Multidrug resistance-associated protein 4 transports compounds out of cells Estradiol 3,17-disulfate, Tranilast, Sulfasalazine P08684 CYP3A4 Cytochrome P450 3A4 mediates metabolism of steroid hormones BDBM144637, BDBM50448963, BDBM144636 Q92887 ABCC2 ATP-binding cassette sub-family C member 2 transports drugs across cell membranes Leukotrien C4, Bilirubin monoglucuronide, Bilirubin bisglucuronide O15245 SLC22A1 Solute carrier family 22 member 1 transports organic cations chlorhexidine, SKF550, Decynium-22 Q96FL8 SLC47A1 Multidrug and toxin extrusion protein 1 transports cationic compounds ondansetron, imatinib, ritonavir P33261 CYP2C19 Cytochrome P450 2C19 mediates metabolism of fatty acids Sulconazole, loratadine, Miconazole Q01959 SLC6A3 Sodium-dependent dopamine transporter mediates dopamine transport hydroperoxycadiforin, Rhenium complex, BAY-390 P51589 CYP2J2 Cytochrome P450 2J2 mediates metabolism of fatty acids Chlorzoxazone, Danazol, BDBM235143 export pump (ABCB11), SLC family 22 member 1 (SLC22A1, OCT1), multidrug and toxin extrusion protein 1 (SLC47A1, MATE1), and the ATP-dependent translocase ABCB1 (Table 1, Supplementary Table 3). The feature selection paradigm also uncovers proteins not regularly screened by for activity during drug development, such as the sodium dependent dopamine transporter DAT (SLC6A3). Feature analysis of the structural characteristics of the proteins inhibited by drugs indicate an importance in composition of helical regions, turn regions, and number of amino acid contacts (Supplementary Table 3). Prediction of drug withdrawal status We hypothesized that these differences between withdrawn and not withdrawn drugs can be exploited to predict if a drug is likely to be withdrawn by regulatory agencies due to unanticipated toxicity using a machine learning. We developed various classifiers trained on protein targets, protein target structural features, as well as molecular fingerprints of the drug, and other structural features of drugs. Models were selected and optimized for each dataset separately. Top models were determined by maximizing the sum of model accuracy, F1, Precision, Recall, and MCC. The various datasets, which each contain different types of features, individually display a wide range of performances (Fig. 4BC). Of the individual predictors, the classifier trained on inhibited protein features, a multilayer perceptron, performed the best with an accuracy of 84.3%, F1 of 0.867, and an MCC of 0.725 (Table 2). The classifier trained on drug features, a stacked extra trees and decision tree, performed with an accuracy of 77.0% (Table 2). The classifier trained on molecular fingerprints, classified with random forest, and inhibited proteins, classified with an RBF sampler and gradient boosting, performed with accuracies of 68.3% and 62.8% respectively (Table 2). The predictions from the individually trained classifiers were combined using an ensemble predictor, the Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH) (Fig. 4 A). To minimize overfitting, holdout data was used in the training of WATCH (Methods). Application of this ensemble learning approach improved performance of drug prediction of withdrawal status compared to individual models trained on chemical fingerprints, chemical features, inhibited proteins, and features of inhibited proteins respectively (Fig. 4BC). Overall accuracy of this K nearest neighbor classifier on the balanced test data set was 92.0%, F1 was 0.919, and MCC was 0.845 (Table 2). For comparison, averaging the individual model predictions achieved accuracy of 85.5% and a MCC of 0.720 (Table 2). Therefore, WATCH demonstrated improved performance compared to the best individual classifier and the simple averaging of all the predicted probabilities from each individual model. Thus, using a combination of feature types derived from drug chemical structure as well as protein targets, WATCH was best able to predict withdrawal status of a compound of all tested models. Data Model Accuracy F1 Precision Recall MCC Ensemble of all data sets K nearest neighbors 0.920 0.919 0.929 0.915 0.845 Averaged prediction of all models on each data set Average 0.855 0.859 0.839 0.890 0.720 Features of protein targets Multilayer perceptron 0.843 0.867 0.769 1.000 0.725 Protein targets RBF sampler and gradient boosting 0.628 0.704 0.589 0.880 0.294 Molecular fingerprints Random forest 0.683 0.690 0.680 0.710 0.370 Drug features Stacked extra trees and decision tree 0.770 0.766 0.850 0.750 0.581 Drug withdrawal status predictor applied to drugs in clinical trials Once WATCH was trained and validated on drug data collected it was tasked with predicting the likelihood that drugs in investigative trials at that time which would ultimately be withdrawn by regulatory agencies due to toxicity. Certain drugs in phase 1–3 clinical trials 19 were predicted to belong to the withdrawn drug class (Supplementary Table 4). Following ten iterations of prediction, a majority of compounds were not predicted to be withdrawn (4 withdrawal predictions or less) (Supplementary Table. 4). Just over 25% of drugs were labeled as likely to be withdrawn in 50% of iterations. There were 5 drugs however that were predicted to be withdrawn in each of the ten iterations: Flindokalner, Levosulpiride, Amithiozone, AZD1981, and Incyclinide (Fig. 5 ). DISCUSSION Drug withdrawal from the market is disruptive to patient care and drug development. Regulatory agencies decide to withdraw a drug after serious and unexpected toxicities in patients are observed. These rare or late manifesting toxicities have not been detected during the normal rigorous drug design process. Here we characterize withdrawn drugs using various features and apply this knowledge in combination with an ensemble predictor to anticipate drug withdrawal status earlier in drug development. Our study provides an overview for landscape of withdrawn drugs. Side-effects associated with withdrawn drugs (derived from ChEMBL) primarily affect three organ systems including the liver, the cardiovascular system, and the neuropsychiatric system (Fig. 1 ). This is consistent with multiple studies investigating drug toxicity 28 , 29 . Moreover, comprehensive analysis of ATC codes (Fig. 2 ), enabled comparisons of trends in withdrawn and non-withdrawn drugs. In the case of antineoplastic drugs, underlying disease severity and resistance development may influence withdrawal status. For example, although vincristine is associated with serious neuropathy, it is still an important part of the regime used to treat childhood cancers such as acute lymphoblastic leukemia, Hodgkin lymphoma, rhabdomyosarcoma, nephroblastoma, medulloblastoma, and low-grade glioma 30 . Similarly, the potential for multidrug resistance increases the need for numerous approved antineoplastic therapies in order to allow for multidrug therapies 31 . Antiparasitic drugs also have no withdrawn treatments. However, in contrast to antineoplastic drugs, the shorter treatment course of certain antiparasitic drugs- less than 90 days 32 - is hypothesized to play a role in this trend. The high number of withdrawn drugs in other categories could potentially result from the number of other existing treatments for diseases in these categories combined with a different assessment of a toxicity to benefit ratio 33 . Taken together, this analysis suggests the possible underlying influence of treatment course and disease severity on withdrawal status, as well as demonstrates current priorities in research and development strategies. To gain more insight into what makes a drug toxic, we characterized drugs based on their chemical and pharmacological features. Feature selection using a combination of three methods distinguished important characteristics of globally withdrawn drugs that can be employed in future drug development. Interestingly, the top four chemical features overrepresented in withdrawn drugs as compared to drugs still in use are related to drug lipophilicity. In particular, logP and logD could be informative because they capture drug distribution throughout the body, with higher concentrations of drug in more locations increasing the potential for binding off-target increasing side effects. Consistent with these results, underrepresented features in withdrawn drugs include molecular weight, frequency of heavy atoms, polar surface area, and quantitative estimate of drug-likeness, which also affect the drug’s ability to cross lipid membrane bilayers, translating to drug oral bioavailability 24 , 34 . Route of administration, another set of features extracted by this analysis, is vital for the understanding of the speed and level of systemic reach of a compound upon use. Indeed, chemical features highly correlated with drug withdrawal including route of administration and pKa reflect known pharmacological ADME design trends (Fig. 3 ). These features can be used in the chemical design process and provide additional confidence in our approach as they overlap with existing drug chemical structure rules 24 , 34 . In the future, drugs with properties that have values in the withdrawn range may become candidate for more thorough screening or may be altered chemically during early stages of development. Similarly, our feature selection approach highlighted the importance of certain drug proteins targets, some of which have known and experimentally validated toxicity. Cytochrome P450 (3A4, 2C19, 2J2), one of the most informative proteins corelated with toxicity according to our feature selection paradigm, is well established as a toxic drug target due to its ability to modulate the metabolism of other drugs 25 , involvement in drug-drug interactions 35 and pharmacogenetics 36 . Certain membrane transporters, which often have similar pharmacological role to that of metabolic enzymes in controlling drug ADME 26 , are also implicated in toxicity (Supplementary Table 2). For example, drug modulation of the ATP-binding cassette sub-family results in cardiotoxicity 37 . The bile salt export pump (ABCB11) has known association with hepatotoxicity 38 . Interestingly, the sodium dependent dopamine transporter DAT (SLC6A3), which is targeted by many CNS drugs such as SSRIs and SNRIs, was also correlated with toxicity in our study. This transporter is not known to be related to pharmacokinetics, and thus, it is not regularly screened by for activity during drug development like the drug ADME transporters above. Features of these proteins such as composition of helical regions and number of amino acid contacts also play a large role in the pattern of drug toxicity further explored by our predictive machine learning model. Interestingly, these features are commonly observed in membrane proteins, such as the transporters discussed 39 . Our work therefore suggests additional proteins and structural features of proteins which would be beneficial to include in drug toxicity screening panels. To utilize our new insights on drug toxicity, we developed a screening tool for flagging potentially toxic drugs. We built an ensemble predictor (Fig. 4 A), Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH), which applies classifiers to various chemical and protein features to label a drug with its withdrawal potential. The 10-fold cross-validated ensemble predictor assessed on balanced, holdout test sets scored an accuracy of 92.0%, F1 of 0.919, and a MCC of 0.845 (Table 2). We also analyzed the clinical trial drug landscape and applied the WATCH predictor to these compounds. The indications of clinical trial drugs reflect a slight predominance of new drugs targeting the alimentary and metabolism (16%) system. (Supplementary Fig. 3). Additional indications common in drugs currently in clinical trials include antineoplastic and immune (13%), cardiovascular (12%), and nervous system (13%) drugs. With the exception of antineoplastic drugs, which do not have as many drugs available as these other groups, these are all areas in which more drugs have been withdrawn than are currently available (Fig. 2 ). The preponderance of withdrawn drugs in these categories points to unmet needs that new drugs are being designed to fill. WATCH applied to drugs in clinical trials distinguished a number of drugs that are expected to be withdrawn from the market (Fig. 5 ). Interestingly, Flindokalner targets potassium ion channels 40 , which overlaps with the know toxic targets found through recursive feature elimination, and is indicated for migraines and post-traumatic headaches (NCT03887325). Levosulpiride is indicated for gastric motility disorders. In trials this drug has been observed to cause severe side effects such as acute dystonias and Parkinson like symptoms 41 . Amithiozone has been used as a treatment for tuberculosis, but was replaced with newer, less toxic antibiotics 42 . Currently, Amithiozone is being re-investigated as a therapy for Mycobacterium Avium Complex 43 . AZD1981 is a CRTh2 receptor antagonist indicated as an asthma therapy 44 . Incyclinide is an antineoplastic agent 45 . In summary, many of the drugs flagged by the withdrawn prediction model, have already begun to demonstrate serious side effects in clinical trials. This demonstrates the applicability of the predictor for future drug toxicity investigations. Furthermore, the characterization of withdrawn drugs and WATCH are meant to serve as tools for drug developers in the ever-evolving endeavor to predict the toxicity of candidate compounds. METHODS Drug ATC analysis Using the withdrawal reasons and ATC datasets, we analyzed the main organ systems that drugs target. We manually grouped the non-standardized labels describing reasons for drug withdrawal (Fig. 1 , Supplemental Fig. 2). In addition, to investigate the differences between organ systems of action in withdrawn and not withdrawn drugs, we used the anatomic therapeutic chemical (ATC) code 22 designation, the first letter of which indicates anatomical main group 46 . Frequencies of withdrawn and not withdrawn drugs for each therapeutic indication category were compared using the scipy.stats.fisher_exact test 1.6.0 (Fig. 2 ). Drugs without an ATC code were omitted. The Bonferroni 47 adjusted p value was 0.0036. It is possible for a drug to have more than one reason for withdrawal and organ system of indication. Feature selection Characteristics that most distinguished between withdrawn and not withdrawn drugs were determined with three different feature selection methods on three sets of data features (drug molecular features, drug targets, and drug target features). Specifically, the methods we used were: (i) recursive feature elimination with cross validation (RFECV) in scikit learn 1.1.2. The cross validation splitting strategy (cv) equaled 5 folds, the steps value equaled 1, and the classifier was a random forest model; (ii) Scikit learn 1.1.2. variance threshold with a threshold of 0; and (iii) generic univariate selection for the top 25% of features with a chi squared score function was also applied. These three feature selection methods were applied to each dataset separately 10 times using a random balanced 80% of the data each time and random seeds 0–9. Scores were normalized to be out of 10, where the best features would score a 10, and combined for each feature selection model with equal weights. For visualization of the drug chemical features, the average values of continuous variables and the frequency counts normalized by the total number of drugs in each category for categorical variables were calculated for withdrawn and not withdrawn drugs. For each statistically significantly different chemical feature variable, the log base 2 value of the ratio of the withdrawn drug divided by the not withdrawn drug was plotted (Fig. 3 ). Prediction of compound withdrawal status Drug withdrawal status for existing drugs was predicted using a variety of machine learning approaches. First the fingerprint, drug chemical features, target, and target feature datasets were used independently to generate predictions. Each of the datasets were randomly split into balanced 80-10-10 percent segments. These represent the training, first test, and second test sets respectively. Balancing of the sets ensured an equal number of withdrawn and not withdrawn drugs in each training and testing tranche. Each feature was min-max normalized (Eq. 1) using the minimum and maximum values of the feature in the training set. The Tree-based Pipeline Optimization Tool (TPOT) 48 0.11.7 was used to tune hyperparameters for TPOT neural networks, scikit learn random forest, scikit learn multilayer perceptron, and xgboost extreme gradient boosting. The parameters for TPOT were: population size of 24, offspring size of 12, early stop of 12, scoring using accuracy, cv of 5, and generations of 5. The parameter search space for the models are as follows. Random forest: criterion was entropy or gini; max depth was none and 10 through 500; max features were auto, square root, log2 or none; min samples per leaf were 2 through 15, min sample split was 5 through 15; number of estimators were 150 through 500. Multilayer perceptron: activation was identity, loginst, tanh, or relu; hidden layer sizes were 25 through 400; solver was lbfgs, stochastic gradient descent, or adam. Extreme gradient boosting: max depth was none or 10 through 500; number of estimators was 150 through 500. Following hyperparameter tuning, the top performing model was selected for each data set. Accuracy, area under the receiver operating characteristic curve (AUROC), F1, precision, recall, and Matthews Correlation Coefficient (MCC) were calculated with scikit learn on the first test set. Equation 1 \\(\\:normalization=\\frac{value-\\text{m}\\text{i}\\text{n}\\left(feature\\:values\\right)}{\\text{max}\\left(feature\\:values\\right)-\\text{m}\\text{i}\\text{n}\\left(feature\\:values\\right)}\\) The output of the best overall performing model for each of the 4 datasets was used to generate the training and test sets for the meta classifier, Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH). These were the models that maximized the sum of the accuracy, AUROC, F1, precision, recall, and MCC. The training set consisted of the training set for the previous model in addition to the test set for the first model. The test set was the second holdout test set from the initial data split. Hyperparameter tuning and model selection for the meta classifier proceeded as previously described. The best meta classifier model, WATCH, was a K nearest neighbor classifier with 57 neighbors, and distance weights. This was compared to a meta classifier model that averaged the predictions for the best models on the 4 datasets. Accuracy, area under the receiver operating characteristic curve (AUROC), F1, precision, and recall were calculated with scikit learn on the second, holdout test set. The best performing classifier for the target structural feature dataset was a multilayer perceptron with identity activation, 150 hidden layers, and a stochastic gradient decent solver. The best performing classifier for the proteins was a RBF sampler with 0.05 gamma and a gradient boosting classifier with 0.1 learning rate, 9 max depth, 0.65 maximum features, 14 minimum samples per leaf, 15 minimum samples per split, 100 estimators, and 0.5 subsamples. The best performing classifier for the drug chemical features dataset was an extra trees classifier with bootstrap, gini criterion, 0.65 maximum features, 7 minimum samples per leaf, and 100 estimators stacked with a decision tree with gini criterion, maximum depth of 5, 18 minimum samples per leaf, and a minimum samples per split of 3. The best performing classifier for the drug fingerprint dataset was a random forest classifier with entropy criterion, 64 maximum depth, log2 maximum features, 2 minimum samples per leaf, 5 minimum samples per split, and 188 estimators. (Fig. 4BC) Prediction of toxicity for drugs under clinical investigation We applied WATCH to existing drugs to drugs currently in clinical trials (1216 drugs). The classifier selected for WATCH was the K nearest neighbor meta classifier, which maximized the sum of the accuracy, AUROC, F1, precision, recall, and MCC. WATCH, run 10 times with random states 0–9, was used to classify all drugs found in ChEMBL currently in phase I-III clinical trials (downloaded using chembl-webresource-client 0.10.8 on January 16, 2023). The output for the 10 runs of WATCH for each drug was summed. The closer the resulting value a drug had to 10, the more likely it is to be withdrawn in the future. DATA AVAILABILITY Drug datasets We constructed four drug feature datasets, including drug withdrawal status dataset, drug indication dataset, drug fingerprint dataset, and drug feature dataset. The withdrawal status dataset contains drug withdrawal status and reasons for drug withdrawal collected from ChEMBL 19 using chembl-webresource-client 0.10.8. Of the 230 drugs listed as withdrawn, 120 had reasons for withdrawal found in the database. We manually curated these non-standardized reasons for withdrawals into logical uniform labels (for example “Hepatic toxicity” and “Hepatotoxicity” were combined under the label of Hepatotoxicity). We further grouped each toxicity by the organ system most effected (for example Liver for Hepatotoxicity). The drug indication dataset contains the anatomic therapeutic chemical (ATC) codes, which classifies drugs by site of action and chemical characteristics 35 , for withdrawn (n = 120) and not withdrawn drugs (n = 1511) also determined using ChEMBL 19 . The drug fingerprint dataset contains Morgan fingerprints 49 with radius 2 and 512 bits as vectors generated with Rdkit 2022.09.1 for withdrawn and not withdrawn drugs. The drug feature dataset contains chemical features of withdrawn and not withdrawn drugs found in ChEMBL 19 . The molecular features included in this dataset are: chirality, partition coefficient (alogp (calculated), cx_logp (generated with ACDlabs v12.01)), number of aromatic rings, full molecular weight (including salts), freebase weight (weight of parent compound), and monoisotopic weight (weight of the monoisotopic parent compound), route of administration (oral, parentreral, or topical), status as a prodrug, distribution coefficient (cx_logd (generated using ACDlabs v12.01), the most acidic acid dissociation constant (cx_most_apka), most basic acid dissociation constant (cx_most_bpka), number of hydrogen bond acceptors (hba, hba_lipinski), number of hydrogen bond donners (hbd, hbd_lipinski), number of heavy atoms, identity as an acid, identity as a base, identity as a neutral molecule, identity as a zwitterion, drug likeness (num_lipinski_ro5_violations (number of rule of 5 violations using Lipinski hydrogen bond acceptor and donor definitions), num_ro5_violations (number of rule of 5 violations using hydrogen bond acceptor and donor definitions), qed_weighted (quantitative estimate of drug likeness 24 )), polar surface area (psa), rule of three (ro3_pass 50 ), and number of rotatable bonds (rtb). Drug target datasets We also constructed two drug target feature datasets, including IC 50 values dataset, and drug target features. The drug target dataset contains IC 50 values for each human protein drug target found in ChEMBL 19 . The drug target features were derived from the amino acid sequence and structural models of the proteins in the drug target dataset (Supplementary Table 3). The method for feature generation has been previously described 14 . In brief, structural features, such as protein secondary structure or intrinsically disordered regions, describe characteristics of protein regions. These structural features are determined from either protein sequence, using the methods IUPRED2a 51 , ANCHOR 52 , TMHMM 2.0 53 , PredictProtein 54 , SCOPe 55 ; or from three-dimensional model predicted with AlphaFold2 56 , using the methods Dictionary of Protein Secondary Structure (DSSP) 57 , biopython’s pdb parser 58 , Aggrescan3d 59 . Cutoffs for classifying region identity have been determined in previous work 14 . Features for each protein were normalized by the length of protein and then weighted according to Eq. 2. Where F is the normalization value of the protein feature, f is the unnormalized value of the protein feature, sIC 50 is the set of IC 50 values for the protein tested with all drugs, and pIC 50 is the IC 50 of the protein being normalized. Equation 2 \\(\\:F=\\:f\\frac{\\text{max}\\left({sIC}_{50}\\right)-\\:{pIC}_{50}+1}{\\text{m}\\text{a}\\text{x}\\left(s{IC}_{50}\\right)\\:}\\) Declarations CODE AVAILABILITY A total of 1,296 compounds (204 withdrawn and 1,092 not withdrawn) were found having fingerprints, targets, target features, and drug molecular features. A full list of drugs and their features can be found in the project GitHub repository: https://github.com/schlessinger-lab/withDRAWN . ACKNOWLEGMENTS This work is supported in part by the National Institutes of Health grant F30 HL160179 and T32 HD075735 to N.Z and R01 HD107528 to A.S COMPETING INTERESTS The authors have no conflicts of interest to declare. Author Contribution N.Z. and A.S. conducted the analysis and wrote this manuscript. Data Availability Data and code can be found in the GitHub repository: https://github.com/schlessinger-lab/withDRAWN. References Wysowski, D. K. & Swartz, L. Adverse drug event surveillance and drug withdrawals in the United States, 1969–2002: the importance of reporting suspected reactions. Arch Intern Med 165, 1363–1369 (2005). https://doi.org:10.1001/archinte.165.12.1363 McNaughton, R., Huet, G. & Shakir, S. An investigation into drug products withdrawn from the EU market between 2002 and 2011 for safety reasons and the evidence used to support the decision-making. BMJ Open 4, e004221 (2014). https://doi.org:10.1136/bmjopen-2013-004221 Ishiguro, C., Misu, T., Iwasa, E. & Izawa, T. Analysis of safety-related regulatory actions by Japan's pharmaceutical regulatory agency. Pharmacoepidemiol Drug Saf 26, 1314–1320 (2017). https://doi.org:10.1002/pds.4252 Alves, C., Macedo, A. F. & Marques, F. B. Sources of information used by regulatory agencies on the generation of drug safety alerts. Eur J Clin Pharmacol 69, 2083–2094 (2013). https://doi.org:10.1007/s00228-013-1564-y Eichler, H. G. et al. Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers. Nat Rev Drug Discov 9, 277–291 (2010). https://doi.org:10.1038/nrd3079 Ifeoma Obidike, A. M., Michaela Reddy, Harvey Clewell, Thierry Lave, Melvin Andersen, Ray Greek, Abdelmigid, Nasir Mohamad, Peter Ward, David La, J. Eric McDuffie, Sandra Snook, Elsa Dias, Carina Menezes, Elisabete Valerio, Jose M Fuentes, Jacob John Van Tonder, Mary Gulumian, Vanessa Steenkamp. New Insights into Toxicity and Drug Testing . (InTech, 2013). Lipsky, M. S. & Sharp, L. K. From idea to market: the drug approval process. J Am Board Fam Pract 14, 362–367 (2001). Tropsha, A. Best Practices for QSAR Model Development, Validation, and Exploitation. Mol Inform 29, 476–488 (2010). https://doi.org:10.1002/minf.201000061 Raschi, E., Ceccarini, L., De Ponti, F. & Recanatini, M. hERG-related drug toxicity and models for predicting hERG liability and QT prolongation. Expert Opin Drug Metab Toxicol 5, 1005–1021 (2009). https://doi.org:10.1517/17425250903055070 Vo, A. H., Van Vleet, T. R., Gupta, R. R., Liguori, M. J. & Rao, M. S. An Overview of Machine Learning and Big Data for Drug Toxicity Evaluation. Chem Res Toxicol 33, 20–37 (2020). https://doi.org:10.1021/acs.chemrestox.9b00227 Wang, H., Liu, R., Schyman, P. & Wallqvist, A. Deep Neural Network Models for Predicting Chemically Induced Liver Toxicity Endpoints From Transcriptomic Responses. Front Pharmacol 10, 42 (2019). https://doi.org:10.3389/fphar.2019.00042 Iftkhar, S. et al. cardioToxCSM: A Web Server for Predicting Cardiotoxicity of Small Molecules. J Chem Inf Model 62, 4827–4836 (2022). https://doi.org:10.1021/acs.jcim.2c00822 Banerjee, P., Eckert, A. O., Schrey, A. K. & Preissner, R. ProTox-II: a webserver for the prediction of toxicity of chemicals. Nucleic Acids Res 46, W257-W263 (2018). https://doi.org:10.1093/nar/gky318 Zatorski, N. et al. Structural analysis of genomic and proteomic signatures reveal dynamic expression of intrinsically disordered regions in breast cancer. iScience 27, 1–16 (2024). https://doi.org:10.1016/j.isci.2024.110640 Zhang, Q. C. et al. Structure-based prediction of protein-protein interactions on a genome-wide scale. Nature 490, 556–560 (2012). https://doi.org:10.1038/nature11503 Wang, X. et al. Three-dimensional reconstruction of protein networks provides insight into human genetic disease. Nat Biotechnol 30, 159–164 (2012). https://doi.org:10.1038/nbt.2106 Rahman, R. et al. Protein structure-based gene expression signatures. Proc Natl Acad Sci U S A 118 (2021). https://doi.org:10.1073/pnas.2014866118 Zatorski, N., Stein, D., Rahman, R., Iyengar, R. & Schlessinger, A. Structural signatures: a web server for exploring a database of and generating protein structural features from human cell lines and tissues. Database (Oxford) 2022 (2022). https://doi.org:10.1093/database/baac053 Gaulton, A. et al. The ChEMBL database in 2017. Nucleic Acids Res 45, D945-D954 (2017). https://doi.org:10.1093/nar/gkw1074 Sachana, M., Shafer, T. J. & Terron, A. Toward a Better Testing Paradigm for Developmental Neurotoxicity: OECD Efforts and Regulatory Considerations. Biology (Basel) 10 (2021). https://doi.org:10.3390/biology10020086 Schlessinger, A. et al. Structure-based discovery of prescription drugs that interact with the norepinephrine transporter, NET. Proc Natl Acad Sci U S A 108, 15810–15815 (2011). https://doi.org:10.1073/pnas.1106030108 Hvidberg, E. & Andersen, A. H. [New classification of drugs. The Medical list and the Drug catalogue are introduced in Anatomical–Therapeutic–Chemical classification code (ACT-code) in 1981]. Ugeskr Laeger 142, 396–397 (1980). Darst, B. F., Malecki, K. C. & Engelman, C. D. Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. BMC Genet 19, 65 (2018). https://doi.org:10.1186/s12863-018-0633-8 Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S. & Hopkins, A. L. Quantifying the chemical beauty of drugs. Nat Chem 4, 90–98 (2012). https://doi.org:10.1038/nchem.1243 Bowes, J. et al. Reducing safety-related drug attrition: the use of in vitro pharmacological profiling. Nat Rev Drug Discov 11, 909–922 (2012). https://doi.org:10.1038/nrd3845 International Transporter, C. et al. Membrane transporters in drug development. Nat Rev Drug Discov 9, 215–236 (2010). https://doi.org:10.1038/nrd3028 Zamek-Gliszczynski, M. J. et al. Transporters in Drug Development: International Transporter Consortium Update on Emerging Transporters of Clinical Importance. Clin Pharmacol Ther 112, 485–500 (2022). https://doi.org:10.1002/cpt.2644 Wong, C. H., Siah, K. W. & Lo, A. W. Estimation of clinical trial success rates and related parameters. Biostatistics 20, 273–286 (2019). https://doi.org:10.1093/biostatistics/kxx069 Cook, D. et al. Lessons learned from the fate of AstraZeneca's drug pipeline: a five-dimensional framework. Nat Rev Drug Discov 13, 419–431 (2014). https://doi.org:10.1038/nrd4309 van de Velde, M. E. et al. Genetic Polymorphisms Associated with Vincristine Pharmacokinetics and Vincristine-Induced Peripheral Neuropathy in Pediatric Oncology Patients. Cancers (Basel) 14 (2022). https://doi.org:10.3390/cancers14143510 Cho, H. & Levy, D. Modeling the chemotherapy-induced selection of drug-resistant traits during tumor growth. J Theor Biol 436, 120–134 (2018). https://doi.org:10.1016/j.jtbi.2017.10.005 Kappagoda, S., Singh, U. & Blackburn, B. G. Antiparasitic therapy. Mayo Clin Proc 86, 561–583 (2011). https://doi.org:10.4065/mcp.2011.0203 Fung, M., Anna Thornton, Kathy Mybeck, Jasmanda Hsiao-hui Wu, Ken Hornbuckle, and Edmundo Muniz. Evaluation of the characteristics of safety withdrawal of prescription drugs from worldwide pharmaceutical markets-1960 to 1999. Drug Information Journal 35, 293–317 (2001). Lipinski, C. A., Lombardo, F., Dominy, B. W. & Feeney, P. J. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Adv Drug Deliv Rev 46, 3–26 (2001). https://doi.org:10.1016/s0169-409x(00)00129-0 Guengerich, F. P. Role of cytochrome P450 enzymes in drug-drug interactions. Adv Pharmacol 43, 7–35 (1997). https://doi.org:10.1016/s1054-3589(08)60200-8 Ingelman-Sundberg, M. Pharmacogenetics of cytochrome P450 and its applications in drug therapy: the past, present and future. Trends Pharmacol Sci 25, 193–200 (2004). https://doi.org:10.1016/j.tips.2004.02.007 Couture, L., Nash, J. A. & Turgeon, J. The ATP-binding cassette transporters and their implication in drug disposition: a special look at the heart. Pharmacol Rev 58, 244–258 (2006). https://doi.org:10.1124/pr.58.2.7 Kenna, J. G. et al. Can Bile Salt Export Pump Inhibition Testing in Drug Discovery and Development Reduce Liver Injury Risk? An International Transporter Consortium Perspective. Clin Pharmacol Ther 104, 916–932 (2018). https://doi.org:10.1002/cpt.1222 Schramm, C. A. et al. Knowledge-based potential for positioning membrane-associated structures and assessing residue-specific energetic contributions. Structure 20, 924–935 (2012). https://doi.org:10.1016/j.str.2012.03.016 Sheppard, A. M., Chen, G. D. & Salvi, R. Potassium ion channel openers, Maxipost and Retigabine, protect against peripheral salicylate ototoxicity in rats. Hear Res 327, 1–8 (2015). https://doi.org:10.1016/j.heares.2015.04.007 Joe, J. Levosulpiride-Induced Neurological Adverse Effects: A Prospective Study from a Tertiary Care Center. Ann Indian Acad Neurol 23, 174–176 (2020). https://doi.org:10.4103/aian.AIAN_449_18 Escovitz, W. E. Disturbance in carbohydrate metabolism associated with amithiozone therapy. Am Rev Tuberc 66, 373–377 (1952). Peloquin, C. A. Controversies in the management of Mycobacterium avium complex infection in AIDS patients. Ann Pharmacother 27, 928–937 (1993). https://doi.org:10.1177/106002809302700722 Kuna, P., Bjermer, L. & Tornling, G. Two Phase II randomized trials on the CRTh2 antagonist AZD1981 in adults with asthma. Drug Des Devel Ther 10, 2759–2770 (2016). https://doi.org:10.2147/DDDT.S105142 Alaseem, A. et al. Matrix Metalloproteinases: A challenging paradigm of cancer management. Semin Cancer Biol 56, 100–115 (2019). https://doi.org:10.1016/j.semcancer.2017.11.008 Pahor, M. et al. Drug data coding and analysis in epidemiologic studies. Eur J Epidemiol 10, 405–411 (1994). https://doi.org:10.1007/BF01719664 Bland, J. M. & Altman, D. G. Multiple significance tests: the Bonferroni method. BMJ 310, 170 (1995). https://doi.org:10.1136/bmj.310.6973.170 Le, T. T., Fu, W. & Moore, J. H. Scaling tree-based automated machine learning to biomedical big data with a feature set selector. Bioinformatics 36, 250–256 (2020). https://doi.org:10.1093/bioinformatics/btz470 Morgan, H. L. The Generation of a Unique Machine Description for Chemical Structures-A Technique Developed at Chemical Abstracts Service. Journal of Chemical Documentation 5, 107–113 (1965). https://doi.org:10.1021/c160017a018 Congreve, M., Carr, R., Murray, C. & Jhoti, H. A 'rule of three' for fragment-based lead discovery? Drug Discov Today 8, 876–877 (2003). https://doi.org:10.1016/s1359-6446(03)02831-9 Meszaros, B., Erdos, G. & Dosztanyi, Z. IUPred2A: context-dependent prediction of protein disorder as a function of redox state and protein binding. Nucleic Acids Res 46, W329-W337 (2018). https://doi.org:10.1093/nar/gky384 Meszaros, B., Simon, I. & Dosztanyi, Z. Prediction of protein binding regions in disordered proteins. PLoS Comput Biol 5, e1000376 (2009). https://doi.org:10.1371/journal.pcbi.1000376 Krogh, A., Larsson, B., von Heijne, G. & Sonnhammer, E. L. Predicting transmembrane protein topology with a hidden Markov model: application to complete genomes. J Mol Biol 305, 567–580 (2001). https://doi.org:10.1006/jmbi.2000.4315 Bernhofer, M. et al. PredictProtein - Predicting Protein Structure and Function for 29 Years. Nucleic Acids Res 49, W535-W540 (2021). https://doi.org:10.1093/nar/gkab354 Fox, N. K., Brenner, S. E. & Chandonia, J. M. SCOPe: Structural Classification of Proteins–extended, integrating SCOP and ASTRAL data and classification of new structures. Nucleic Acids Res 42, D304-309 (2014). https://doi.org:10.1093/nar/gkt1240 Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). https://doi.org:10.1038/s41586-021-03819-2 Kabsch, W. & Sander, C. Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features. Biopolymers 22, 2577–2637 (1983). https://doi.org:10.1002/bip.360221211 Cock, P. J. et al. Biopython: freely available Python tools for computational molecular biology and bioinformatics. Bioinformatics 25, 1422–1423 (2009). https://doi.org:10.1093/bioinformatics/btp163 Kuriata, A. et al. Aggrescan3D (A3D) 2.0: prediction and engineering of protein solubility. Nucleic Acids Res 47, W300-W307 (2019). https://doi.org:10.1093/nar/gkz321 Additional Declarations No competing interests reported. Supplementary Files SupportingInformationFinal.docx SUPPLEMENTAL INFORMATION TITLES AND LEGENDS Supplementary Figure 1. Countries of drug withdrawal used for analysis and labeling in the predictor. A large proportion of drugs in this study have been withdrawn by the USFDA. A single drug can be withdrawn from multiple countries. Supplementary Figure 2. Reasons for Drug withdrawals. A. Hepatotoxicity withdrawal reasons are largely unannotated more specifically. B. Cardiovascular toxicity withdrawal reasons primarily center around arrhythmias. C. Neuropsychiatric toxicity withdrawal reasons are vast and fall into a large number of categories. Supplementary Figure 3. Indications of drugs in clinical trials based on ATC category. The largest group of drugs under investigation are indicated for alimentary and metabolism while the fewest are antiparasitic. Supplementary Table 1. Ranking of all drug chemical features in the model. Feature names are taken from the ChEMBL database as accessed with the chembl_webresource_client. Supplementary Table 2. Ranking of all proteins in the model. Supplementary Table 3. Ranking of all protein structural features in the model. Supplementary Table 4. Drugs currently in clinical trials ranked by the number of times the predictor assigned them withdrawal status. The least likely to be withdrawn have rank 0 and the most likely to be withdrawn have rank 10. Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 04 Aug, 2025 Reviews received at journal 03 Aug, 2025 Reviews received at journal 15 Jul, 2025 Reviewers agreed at journal 11 Jul, 2025 Reviewers agreed at journal 08 Jul, 2025 Reviewers invited by journal 08 Jul, 2025 Editor assigned by journal 05 Jul, 2025 Submission checks completed at journal 04 Jul, 2025 First submitted to journal 28 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-6999821\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":false,\"archivedVersions\":[],\"articleType\":\"Article\",\"associatedPublications\":[],\"authors\":[{\"id\":482598293,\"identity\":\"518c6add-a500-4893-abf1-75ff46baeec7\",\"order_by\":0,\"name\":\"Nicole Zatorski\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIie3Pv2oDIRzA8Z8IZpF2/S1NXsHjhrakkFfxONApe4ZQDgKXLXvo0FfoIwQEuwhdHS9LZrvdFGroH2jhzrWDXxBU/MBPgFzuP0bj6n5OB5zCgX3fjxB52bBPUqYJ/CZQNSlyO6HHTq5gcf28se+9u9M7r6oOVvOqGSD3G1YK6YCiZfUT97jce2UEOD1IhOEMqzaOZXlJIeDyxesWSWsS5Ax8FgnpA2rh9bYn5xRpAEUkEAeTwisLpBkjl79YFIVVNeUOi7071SitLgfJmzl2Yf2wmBpjSG8fZ1evqghhPb8ZIl/hn7Mcf57L5XK5RB9JJFcP1MPPCgAAAABJRU5ErkJggg==\",\"orcid\":\"\",\"institution\":\"Duke University Hospital\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Nicole\",\"middleName\":\"\",\"lastName\":\"Zatorski\",\"suffix\":\"\"},{\"id\":482598296,\"identity\":\"8ae8c61c-26d2-47c3-9028-a6d783b4f9c0\",\"order_by\":1,\"name\":\"Avner Schlessinger\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Icahn School of Medicine at Mount Sinai\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Avner\",\"middleName\":\"\",\"lastName\":\"Schlessinger\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-06-28 21:38:11\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-6999821/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-6999821/v1\",\"draftVersion\":[],\"editorialEvents\":[],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":86416631,\"identity\":\"25cc7672-2247-43fc-a2bf-8dacb101021b\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:51:25\",\"extension\":\"png\",\"order_by\":1,\"title\":\"Figure 1\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":24749,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eCauses of drug withdrawals listed by organ system of impact. \\u003c/strong\\u003eThree broad toxicity types, including cardiovascular, hepatic, and neuropsychiatric comprise a majority of reasons for drug withdrawals. Data was compiled from the ChEMBL database (Gaulton, et al., 2017).\\u003cstrong\\u003e \\u003c/strong\\u003eA single drug can have more than one potential cause for withdrawal.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"1.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/45a07984f660f1d4215b257f.png\"},{\"id\":86416633,\"identity\":\"1c807caa-99be-4f5e-b201-a6604115c90e\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:51:25\",\"extension\":\"png\",\"order_by\":2,\"title\":\"Figure 2\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":17552,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eIndication of withdrawn drugs compared to indications of approved drugs. \\u003c/strong\\u003eIndication are defined based on the first letter (Level 1) of the ATC code. Percent of compounds with that ATC first letter designation compared to the total number of compounds in that category is shown on the horizontal axis. Pairwise comparisons between frequencies of ATC indications performed with a fisher exact test show the following indications have differences between withdrawn and not withdrawn drugs: anti-infective, antineoplastic/immune, antiparasitic, hormones, musculo-skeletal and nervous system. A p-value lower than 0.05 is indicated by a *, and a p-value lower than the Bonferroni corrected significance level is indicated by a **.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"2.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/2f8b6caa59196a7c196ff010.png\"},{\"id\":86417558,\"identity\":\"ff8b806c-04e2-481c-8a29-32a3a052adb3\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:59:25\",\"extension\":\"png\",\"order_by\":3,\"title\":\"Figure 3\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":14448,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eDrug chemical features of withdrawn drugs compared to not withdrawn drugs.\\u003c/strong\\u003e The average values of continuous variables and the frequency counts normalized by the total number of drugs in each category for categorical variables were calculated for withdrawn and not withdrawn drugs. For each statistically significantly different chemical feature variable, the log base 2 value of the ratio of the withdrawn drug divided by the not withdrawn drug was plotted. Features with positive values are larger in withdrawn drugs than not withdrawn drugs. All features shown have a p-value smaller than 0.05. Feature names correspond to those found in ChEMBL chembl_webresource_client. Features that are over represented in withdrawn drugs include lipophilic characteristics (cx_logd, alogp and cx_logp, which correspond to logD and logP respectively) as well as number of aromatic rings. Features that are under represented in withdrawn drugs include the number of atoms participating in hydrogen bonds (hydrogen bond acceptors and donors- hba, hbd, hba_lipinski, hbd_lipinski), the molecular weight (mw_freebase, mw_monoisotopic, full_mwt), polar surface area (psa), status as a prodrug, and routes of administration (topical, parenteral, and oral administration).\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"3.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/de1cfd1891001eedfa3c4031.png\"},{\"id\":86416638,\"identity\":\"99bf6976-0ce5-457b-aae4-a826f5a8e639\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:51:25\",\"extension\":\"png\",\"order_by\":4,\"title\":\"Figure 4\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":101062,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eModel predicting drug withdrawal performance on representative test set\\u003c/strong\\u003e. Top-performing classifiers are shown for each data set, and are as follows: multilayer perceptron for drug target features, RBF sampler for drug targets, extra trees classifier stacked with a decision tree for drug chemical features, random forest for drug fingerprints, and K-nearest neighbor for the ensemble. A.\\u003cstrong\\u003e \\u003c/strong\\u003eModel architecture for the withdrawn drug predictor. Each data set was divided into training, test, and validation sets. The training set was used to select and tune the hyperparameters for separate models. The test set was used to evaluate model performance. Predictions from these models were combined to form a new data set which served as the training data for the meta predictor. The ensemble predictor was trained on the pooled predictions from the individual predictors using the original training set and the test set combined. The model was evaluated using validation set. B. Receiver Operating Characteristic (ROC) curve showing predictive model true positive rate compared to false positive rate for each of the models based on different input data as well as the overall ensemble predictor. C. Precision-recall curves for each of the models based on different input data as well as the overall ensemble predictor. Using the two different evaluation measures, the ensemble-based predictor preformed the best of all models.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"4.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/1cf0ff715b355d77b1c12fec.png\"},{\"id\":86416636,\"identity\":\"c784e931-fef7-4735-bf86-258664fbfbdb\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:51:25\",\"extension\":\"png\",\"order_by\":5,\"title\":\"Figure 5\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":55376,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003ePrediction of withdrawal of drugs in clinical trials.\\u003c/strong\\u003e The predictor was trained on 10 different balanced random subsets of existing withdrawn and not withdrawn drug data. These independently trained models were applied to new drugs currently in clinical trials to assess future withdrawal status of these candidate compounds. This figure shows structures of compounds that are currently in clinical trials and are predicted to be withdrawn by the predictor in 10 out of 10 prediction iterations. These include three compounds with serious known toxicities and toxic targets.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"5.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/cb12adf1ba947b561e4ecdea.png\"},{\"id\":86418441,\"identity\":\"ebec1e22-6161-447c-962f-1913919d3c57\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 12:15:26\",\"extension\":\"pdf\",\"order_by\":0,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":1161426,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"manuscript.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/de9110eb-dae0-4aa9-9390-1386b7b5710d.pdf\"},{\"id\":86416639,\"identity\":\"632c59e5-ee9e-4b40-b490-939e84176d4b\",\"added_by\":\"auto\",\"created_at\":\"2025-07-10 11:51:25\",\"extension\":\"docx\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"supplement\",\"size\":368219,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eSUPPLEMENTAL INFORMATION TITLES AND LEGENDS\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Figure 1. Countries of drug withdrawal used for analysis and labeling in the predictor.\\u003c/strong\\u003e A large proportion of drugs in this study have been withdrawn by the USFDA. A single drug can be withdrawn from multiple countries.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Figure 2. Reasons for Drug withdrawals.\\u003c/strong\\u003e A. Hepatotoxicity withdrawal reasons are largely unannotated more specifically. B. Cardiovascular toxicity withdrawal reasons primarily center around arrhythmias. C. Neuropsychiatric toxicity withdrawal reasons are vast and fall into a large number of categories.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Figure 3. Indications of drugs in clinical trials based on ATC category.\\u003c/strong\\u003e The largest group of drugs under investigation are indicated for alimentary and metabolism while the fewest are antiparasitic.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Table 1. Ranking of all drug chemical features in the model.\\u003c/strong\\u003e Feature names are taken from the ChEMBL database as accessed with the chembl_webresource_client.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Table 2. Ranking of all proteins in the model.\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Table 3. Ranking of all protein structural features in the model.\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eSupplementary Table 4. Drugs currently in clinical trials ranked by the number of times the predictor assigned them withdrawal status. \\u003c/strong\\u003eThe least likely to be withdrawn have rank 0 and the most likely to be withdrawn have rank 10.\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"SupportingInformationFinal.docx\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6999821/v1/861b838b3b19680c84379f98.docx\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Capturing Unanticipated Drug Toxicities Using an Ensemble Machine Learning Approach\",\"fulltext\":[{\"header\":\"INTRODUCTION\",\"content\":\"\\u003cp\\u003eUnanticipated toxicities can emerge when drugs are incorporated into standard of care so numerous regulatory bodies monitor the safety of drugs for side effects after the end of their official clinical testing phase\\u003csup\\u003e\\u003cspan additionalcitationids=\\\"CR2 CR3\\\" citationid=\\\"CR1\\\" class=\\\"CitationRef\\\"\\u003e1\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR4\\\" class=\\\"CitationRef\\\"\\u003e4\\u003c/span\\u003e\\u003c/sup\\u003e. If unexpected side effects are observed, these agencies, such as the United States Federal Drug Administration (FDA) or the European Medicines Agency (EMA), can withdraw the drug from the market\\u003csup\\u003e\\u003cspan additionalcitationids=\\\"CR2 CR3\\\" citationid=\\\"CR1\\\" class=\\\"CitationRef\\\"\\u003e1\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR4\\\" class=\\\"CitationRef\\\"\\u003e4\\u003c/span\\u003e\\u003c/sup\\u003e. In addition to the impact unexpected toxicities have on patients, the decision to discontinue a drug further creates economic burden that prevents recoupment of research and development investment that increase costs of healthcare for patients\\u003csup\\u003e\\u003cspan citationid=\\\"CR5\\\" class=\\\"CitationRef\\\"\\u003e5\\u003c/span\\u003e\\u003c/sup\\u003e.\\u003c/p\\u003e\\u003cp\\u003eDue to the difficulties resulting from unexpected toxicities and subsequent drug withdrawals, many approaches have been used in an effort to predict toxicity earlier in the development pipeline\\u003csup\\u003e\\u003cspan citationid=\\\"CR6\\\" class=\\\"CitationRef\\\"\\u003e6\\u003c/span\\u003e\\u003c/sup\\u003e. Currently, the industry standard relies on laboratory and animal testing followed by Phase 1 trials in healthy volunteers to assess toxic side effects\\u003csup\\u003e\\u003cspan citationid=\\\"CR7\\\" class=\\\"CitationRef\\\"\\u003e7\\u003c/span\\u003e\\u003c/sup\\u003e. Traditional computation toxicity prediction approaches have centered around the use of drug Quantitative Structure-Activity Relationships (QSAR)\\u003csup\\u003e\\u003cspan citationid=\\\"CR8\\\" class=\\\"CitationRef\\\"\\u003e8\\u003c/span\\u003e\\u003c/sup\\u003e, which typically involve statistical or machine learning models that characterize the relationship between chemical properties of a drug and a phenotypes associated with toxicity. These features are used, for example, to predict a drug\\u0026rsquo;s likelihood of inhibiting a particular protein, such as the hERG channel in the heart, which can precipitate arrhythmias when modulated by a drug\\u003csup\\u003e\\u003cspan citationid=\\\"CR9\\\" class=\\\"CitationRef\\\"\\u003e9\\u003c/span\\u003e\\u003c/sup\\u003e. While this system is fairly robust, it fails to capture the long-term use toxicities and very rare side effects uncovered in post marketing surveillance.\\u003c/p\\u003e\\u003cp\\u003eIn addition, emerging big data analysis and machine learning techniques can be applied preclinically, during the drug development process, to provide a supplemental approach to toxicity prediction and eliminate potentially toxic drug candidates\\u003csup\\u003e\\u003cspan citationid=\\\"CR10\\\" class=\\\"CitationRef\\\"\\u003e10\\u003c/span\\u003e\\u003c/sup\\u003e. For example, deep neural network models applied to whole-genome DNA microarray data from the Open Toxicogenomics Project-Genomics Assisted Toxicity Evaluation System and DrugMatrix have predicted three liver toxicity endpoints with a range of 56\\u0026ndash;89% accuracy\\u003csup\\u003e11\\u003c/sup\\u003e. Cardio ToxCSM, is a webserver that predicts six cardiac toxicity outcomes from chemical drug structure representations and molecular descriptors achieving areas under the receiver operating characteristic curve (AUROC) ranging from 0.63 to 0.92\\u003csup\\u003e12\\u003c/sup\\u003e. ProTox-II provides predictions for 33 toxicity endpoints based on drug chemical data and 15 assayed targets\\u003csup\\u003e\\u003cspan citationid=\\\"CR13\\\" class=\\\"CitationRef\\\"\\u003e13\\u003c/span\\u003e\\u003c/sup\\u003e. These existing predictors, which are not yet used in established drug development pipelines, focus on detecting organ system or pathway toxicities rather than labeling drugs based on their likelihood of resulting in any side effects that necessitate withdrawal.\\u003c/p\\u003e\\u003cp\\u003eStructural features of protein drug targets have great potential for bolstering toxicity predictive models. Because these features describe the underlying structures of proteins, they are more readily able to capture similarities in different proteins\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e. These structural similarities, due to the interconnectedness of structure and function, therefore allow structural features to capture functional similarities and therefore similarities in drug effects. Structural features have been used, along with machine learning approaches, to predict protein-protein interactions\\u003csup\\u003e\\u003cspan citationid=\\\"CR15\\\" class=\\\"CitationRef\\\"\\u003e15\\u003c/span\\u003e\\u003c/sup\\u003e and to construct genetic disease protein networks\\u003csup\\u003e\\u003cspan citationid=\\\"CR16\\\" class=\\\"CitationRef\\\"\\u003e16\\u003c/span\\u003e\\u003c/sup\\u003e. Protein structural features also capture patterns in normal human tissues\\u003csup\\u003e\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e\\u003c/sup\\u003e, cancer\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR18\\\" class=\\\"CitationRef\\\"\\u003e18\\u003c/span\\u003e\\u003c/sup\\u003e, and drug perturbations\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e\\u003c/sup\\u003e from gene expression and proteomics data. These features therefore, combined with current machine learning techniques, hold great potential for revealing unanticipated drug toxicities.\\u003c/p\\u003e\\u003cp\\u003eHere we leverage data from 948 protein targets and their structural features with drug structural information to predict drug toxicity as defined by the compound\\u0026rsquo;s withdrawal status. Drug withdrawal based on the recommendation of global drug safety regulatory agencies is a metric for drug toxicity that captures both extremely rare and severe drug side effects that have not been anticipated by traditional screening approaches. We begin with an examination of the different characteristics between withdrawn drugs and those still deemed safe to use. Indeed, through rigorous feature analysis, we identify drug and drug target features correlated with withdrawal drug status which can be used to inform future drug development. We then develop a predictor, named Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH), to determine the likelihood of a drug having unanticipated toxicity leading to regulatory withdrawal. Furthermore, we discuss how WATCH can be applied as a screening tool to anticipate toxicities that may emerge in drugs under investigation by applying the predictor to drugs in clinical trials.\\u003c/p\\u003e\"},{\"header\":\"RESULTS\",\"content\":\"\\u003cdiv id=\\\"Sec3\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eDrug withdrawals categorized by organ system\\u003c/h2\\u003e\\u003cp\\u003eThe 120 drugs, found in the ChEMBL database\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e, that have been withdrawn by regulatory agencies worldwide have targeted a variety of organ systems (Supplementary Fig.\\u0026nbsp;1). Interestingly, side effects affecting three organ systems make up a majority of reasons for drug withdrawal from the market. Cardiovascular (19.5%), hepatic (19.2%), and neuropsychiatric toxicity (13.2%) are the three largest contributors to drug withdrawal (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig1\\\" class=\\\"InternalRef\\\"\\u003e1\\u003c/span\\u003e). Cardiovascular reasons for withdrawal are largely due to arrhythmias and conditions that predispose to arrhythmias: arrhythmias (12%), QT prolongation (6%), ventricular tachycardia (6%), torsade de pointes (6%), ventricular arrhythmias (5%), and repolarization abnormalities (3%) (Supplementary Fig.\\u0026nbsp;2A). This is supported by the focus of early cardiotoxicity screens, which primarily investigate compound interactions with arrhythmia inducing channels such as hERG\\u003csup\\u003e\\u003cspan citationid=\\\"CR9\\\" class=\\\"CitationRef\\\"\\u003e9\\u003c/span\\u003e\\u003c/sup\\u003e. Besides general hepatotoxicity, liver failure is the second largest listed reason for hepatotoxicity (8%) (Supplementary Fig.\\u0026nbsp;2B). In contrast to hepatotoxicity and cardiotoxicity, neuropsychiatric toxicity has more variety of side effects that range from abuse (21%) to central nervous system stimulation (2%) (Supplementary Fig.\\u0026nbsp;2C). This reflects the wide range of observed neuropsychiatric toxicities and the difficulty in the development of \\u003cem\\u003ein vivo\\u003c/em\\u003e screening tests which can be applied prior to lengthy animal studies\\u003csup\\u003e\\u003cspan citationid=\\\"CR20\\\" class=\\\"CitationRef\\\"\\u003e20\\u003c/span\\u003e\\u003c/sup\\u003e. Neuropsychiatric drugs are well known for their polypharmacology, where individual drugs bind many proteins, which contributes to increased frequency and severity of toxicity\\u003csup\\u003e\\u003cspan citationid=\\\"CR21\\\" class=\\\"CitationRef\\\"\\u003e21\\u003c/span\\u003e\\u003c/sup\\u003e.\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\\u003c/div\\u003e\\n\\u003ch3\\u003eComparison between withdrawn and not withdrawn drugs by indication\\u003c/h3\\u003e\\n\\u003cp\\u003eA fisher exact test analysis of drug Anatomical Therapeutic Chemical (ATC) codes\\u003csup\\u003e\\u003cspan citationid=\\\"CR22\\\" class=\\\"CitationRef\\\"\\u003e22\\u003c/span\\u003e\\u003c/sup\\u003e, a hierarchical classification system of medicines maintained by the World Health Organization (WHO), reveals differences in target organ system and disease indication between drugs that are in use and those that have been withdrawn (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e). Withdrawn drugs have different frequencies of therapeutic indications from drugs that are not withdrawn for various treatment types. For example, neither the antineoplastic nor antiparasitic categories have any withdrawn drugs. Antineoplastics account for 10% of not withdrawn drugs (p\\u0026thinsp;=\\u0026thinsp;1.66e-5) and antiparasitic drugs comprise 3% of not withdrawn drugs (p\\u0026thinsp;=\\u0026thinsp;0.0448) (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e). For other ATC indications, more drugs have been withdrawn than not withdrawn. Drugs that are indicated for diseases of the alimentary and metabolic systems (22% of withdrawn and 9% of not withdrawn p\\u0026thinsp;=\\u0026thinsp;3.63e-5), cardiovascular system (19% of withdrawn and 12% of not withdrawn p\\u0026thinsp;=\\u0026thinsp;0.0294), dermatologic system (12% of withdrawn and 6% of not withdrawn p\\u0026thinsp;=\\u0026thinsp;0.0331), musculo-skeletal system (22% of withdrawn and 5% of not withdrawn p\\u0026thinsp;=\\u0026thinsp;4.40e-10), and nervous system (45% of withdrawn and 18% of not withdrawn p\\u0026thinsp;=\\u0026thinsp;1.90e-10) have a higher frequency of withdrawals. A sub analysis of drugs actively in clinical trials reveals that the primary indications of focus are: alimentary and metabolism (16%), antineoplastic and immune (13%), cardiovascular (12%), and nervous system (13%).\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003eFeatures of withdrawn drugs and their protein targets\\u003c/h3\\u003e\\n\\u003cp\\u003eTo determine the characteristics that mark a drug removed from use due to toxicity, we investigated drug structural and pharmacological features through recursive feature elimination with cross validation (RFECV)\\u003csup\\u003e\\u003cspan citationid=\\\"CR23\\\" class=\\\"CitationRef\\\"\\u003e23\\u003c/span\\u003e\\u003c/sup\\u003e using a random forest model, variance threshold, and generic univariate selection (Methods). Feature selection of drug chemical features demonstrates that route of administration, formulation as a prodrug, acidity (as measured by apKa), quantitative estimate of drug-likeness \\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e\\u003c/sup\\u003e, drug identity as a base, number of hydrogen bond acceptors (hba), molecular weight (MW), polar surface area (tPSA), lipophilicity (i.e. logP, logD), and frequency of heavy atoms are all highly distinguishing features between withdrawn and not withdrawn drugs (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e, Supplementary Table\\u0026nbsp;1). Consistent with these results, underrepresented features in withdrawn drugs include molecular weight, frequency of heavy atoms, polar surface area, and quantitative estimate of drug-likeness (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e).\\u003c/p\\u003e\\u003cp\\u003eWe next applied the feature selection approach to the protein targets, including the protein name, as well as protein structural and functional features, such as secondary structure elements, disordered regions, and coiled-coils. These proteins include intended and unintended targets of the drug. The most informative drug targets include proteins involved in drug absorption, disposition, metabolism, and excretion (ADME) that are screened in drug development as part of drug toxicity target panels, such as the metabolic enzyme Cytochrome P450 (3A4, 2C19, 2J2)\\u003csup\\u003e\\u003cspan citationid=\\\"CR25\\\" class=\\\"CitationRef\\\"\\u003e25\\u003c/span\\u003e\\u003c/sup\\u003e; and the membrane transporters \\u003csup\\u003e\\u003cspan citationid=\\\"CR26\\\" class=\\\"CitationRef\\\"\\u003e26\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR27\\\" class=\\\"CitationRef\\\"\\u003e27\\u003c/span\\u003e\\u003c/sup\\u003e the ATP-binding cassette sub-family C (members 2\\u0026ndash;4), the bile salt\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\\u003cp\\u003e\\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Taba\\\" border=\\\"1\\\"\\u003e\\u003ccolgroup cols=\\\"5\\\"\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c5\\\" colnum=\\\"5\\\"\\u003e\\u003c/div\\u003e\\u003cthead\\u003e\\u003ctr\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eProtein Target\\u003csup\\u003eA\\u003c/sup\\u003e\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eGene Name\\u003csup\\u003eB\\u003c/sup\\u003e\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eProtein Target Name\\u003csup\\u003eC\\u003c/sup\\u003e\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003eBiological Function\\u003csup\\u003eD\\u003c/sup\\u003e\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eDrugs with lowest IC\\u003csub\\u003e50\\u003c/sub\\u003e standard values\\u003csup\\u003eE\\u003c/sup\\u003e\\u003c/p\\u003e\\u003c/th\\u003e\\u003c/tr\\u003e\\u003c/thead\\u003e\\u003ctbody\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eO15438\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eABCC3\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eATP-binding cassette sub-family C member 3\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports drugs across cell membranes\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eCyclosporin A, amg-009, olmesartan medoxomil\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eO95342\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eABCB11\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eATP-binding cassette sub-family B member 11\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports bile salts across hepatocyte membrane\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eAlisporivir, BDBM172715, Pioglitazone\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eO15439\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eABCC4\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eMultidrug resistance-associated protein 4\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports compounds out of cells\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eEstradiol 3,17-disulfate, Tranilast, Sulfasalazine\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eP08684\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eCYP3A4\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eCytochrome P450 3A4\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003emediates metabolism of steroid hormones\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eBDBM144637, BDBM50448963, BDBM144636\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eQ92887\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eABCC2\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eATP-binding cassette sub-family C member 2\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports drugs across cell membranes\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eLeukotrien C4, Bilirubin monoglucuronide, Bilirubin bisglucuronide\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eO15245\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eSLC22A1\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eSolute carrier family 22 member 1\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports organic cations\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003echlorhexidine, SKF550, Decynium-22\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eQ96FL8\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eSLC47A1\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eMultidrug and toxin extrusion protein 1\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003etransports cationic compounds\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eondansetron, imatinib, ritonavir\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eP33261\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eCYP2C19\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eCytochrome P450 2C19\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003emediates metabolism of fatty acids\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eSulconazole, loratadine, Miconazole\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eQ01959\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eSLC6A3\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eSodium-dependent dopamine transporter\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003emediates dopamine transport\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003ehydroperoxycadiforin, Rhenium complex, BAY-390\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eP51589\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eCYP2J2\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eCytochrome P450 2J2\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003emediates metabolism of fatty acids\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003eChlorzoxazone, Danazol, BDBM235143\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003c/tbody\\u003e\\u003c/colgroup\\u003e\\u003c/table\\u003e\\u003c/div\\u003e\\u003c/p\\u003e\\u003cp\\u003eexport pump (ABCB11), SLC family 22 member 1 (SLC22A1, OCT1), multidrug and toxin extrusion protein 1 (SLC47A1, MATE1), and the ATP-dependent translocase ABCB1 (Table 1, Supplementary Table 3). The feature selection paradigm also uncovers proteins not regularly screened by for activity during drug development, such as the sodium dependent dopamine transporter DAT (SLC6A3). Feature analysis of the structural characteristics of the proteins inhibited by drugs indicate an importance in composition of helical regions, turn regions, and number of amino acid contacts (Supplementary Table 3).\\u003c/p\\u003e\\u003cp\\u003e\\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable border=\\\"1\\\"\\u003e\\u003c/table\\u003e\\u003c/div\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003ePrediction of drug withdrawal status\\u003c/h3\\u003e\\n\\u003cp\\u003eWe hypothesized that these differences between withdrawn and not withdrawn drugs can be exploited to predict if a drug is likely to be withdrawn by regulatory agencies due to unanticipated toxicity using a machine learning. We developed various classifiers trained on protein targets, protein target structural features, as well as molecular fingerprints of the drug, and other structural features of drugs. Models were selected and optimized for each dataset separately. Top models were determined by maximizing the sum of model accuracy, F1, Precision, Recall, and MCC. The various datasets, which each contain different types of features, individually display a wide range of performances (Fig.\\u0026nbsp;4BC). Of the individual predictors, the classifier trained on inhibited protein features, a multilayer perceptron, performed the best with an accuracy of 84.3%, F1 of 0.867, and an MCC of 0.725 (Table\\u0026nbsp;2). The classifier trained on drug features, a stacked extra trees and decision tree, performed with an accuracy of 77.0% (Table\\u0026nbsp;2). The classifier trained on molecular fingerprints, classified with random forest, and inhibited proteins, classified with an RBF sampler and gradient boosting, performed with accuracies of 68.3% and 62.8% respectively (Table\\u0026nbsp;2).\\u003c/p\\u003e\\u003cp\\u003eThe predictions from the individually trained classifiers were combined using an ensemble predictor, the Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH) (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig4\\\" class=\\\"InternalRef\\\"\\u003e4\\u003c/span\\u003eA). To minimize overfitting, holdout data was used in the training of WATCH (Methods). Application of this ensemble learning approach improved performance of drug prediction of withdrawal status compared to individual models trained on chemical fingerprints, chemical features, inhibited proteins, and features of inhibited proteins respectively (Fig.\\u0026nbsp;4BC).\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\\u003cp\\u003eOverall accuracy of this K nearest neighbor classifier on the balanced test data set was 92.0%, F1 was 0.919, and MCC was 0.845 (Table\\u0026nbsp;2). For comparison, averaging the individual model predictions achieved accuracy of 85.5% and a MCC of 0.720 (Table\\u0026nbsp;2). Therefore, WATCH demonstrated improved performance compared to the best individual classifier and the simple averaging of all the predicted probabilities from each individual model. Thus, using a combination of feature types derived from drug chemical structure as well as protein targets, WATCH was best able to predict withdrawal status of a compound of all tested models.\\u003c/p\\u003e\\u003cp\\u003e\\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"No\\\" id=\\\"Tabb\\\" border=\\\"1\\\"\\u003e\\u003ccolgroup cols=\\\"7\\\"\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c5\\\" colnum=\\\"5\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c6\\\" colnum=\\\"6\\\"\\u003e\\u003c/div\\u003e\\u003cdiv align=\\\"char\\\" char=\\\".\\\" class=\\\"colspec\\\" colname=\\\"c7\\\" colnum=\\\"7\\\"\\u003e\\u003c/div\\u003e\\u003cthead\\u003e\\u003ctr\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eData\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eModel\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003eAccuracy\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003eF1\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003ePrecision\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003eRecall\\u003c/p\\u003e\\u003c/th\\u003e\\u003cth align=\\\"left\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003eMCC\\u003c/p\\u003e\\u003c/th\\u003e\\u003c/tr\\u003e\\u003c/thead\\u003e\\u003ctbody\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eEnsemble of all data sets\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eK nearest neighbors\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.920\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.919\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.929\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e0.915\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.845\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eAveraged prediction of all models on each data set\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eAverage\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.855\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.859\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.839\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e0.890\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.720\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eFeatures of protein targets\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eMultilayer perceptron\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.843\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.867\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.769\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e1.000\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.725\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eProtein targets\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eRBF sampler and gradient boosting\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.628\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.704\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.589\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e0.880\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.294\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eMolecular fingerprints\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eRandom forest\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.683\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.690\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.680\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e0.710\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.370\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003ctr\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u003cp\\u003eDrug features\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u003cp\\u003eStacked extra trees and decision tree\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c3\\\"\\u003e\\u003cp\\u003e0.770\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c4\\\"\\u003e\\u003cp\\u003e0.766\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c5\\\"\\u003e\\u003cp\\u003e0.850\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c6\\\"\\u003e\\u003cp\\u003e0.750\\u003c/p\\u003e\\u003c/td\\u003e\\u003ctd align=\\\"char\\\" char=\\\".\\\" colname=\\\"c7\\\"\\u003e\\u003cp\\u003e0.581\\u003c/p\\u003e\\u003c/td\\u003e\\u003c/tr\\u003e\\u003c/tbody\\u003e\\u003c/colgroup\\u003e\\u003c/table\\u003e\\u003c/div\\u003e\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003eDrug withdrawal status predictor applied to drugs in clinical trials\\u003c/h3\\u003e\\n\\u003cp\\u003eOnce WATCH was trained and validated on drug data collected it was tasked with predicting the likelihood that drugs in investigative trials at that time which would ultimately be withdrawn by regulatory agencies due to toxicity. Certain drugs in phase 1\\u0026ndash;3 clinical trials\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e were predicted to belong to the withdrawn drug class (Supplementary Table\\u0026nbsp;4). Following ten iterations of prediction, a majority of compounds were not predicted to be withdrawn (4 withdrawal predictions or less) (Supplementary Table. 4). Just over 25% of drugs were labeled as likely to be withdrawn in 50% of iterations. There were 5 drugs however that were predicted to be withdrawn in each of the ten iterations: Flindokalner, Levosulpiride, Amithiozone, AZD1981, and Incyclinide (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig5\\\" class=\\\"InternalRef\\\"\\u003e5\\u003c/span\\u003e).\\u003c/p\\u003e\\u003cp\\u003e\\u003c/p\\u003e\"},{\"header\":\"DISCUSSION\",\"content\":\"\\u003cp\\u003eDrug withdrawal from the market is disruptive to patient care and drug development. Regulatory agencies decide to withdraw a drug after serious and unexpected toxicities in patients are observed. These rare or late manifesting toxicities have not been detected during the normal rigorous drug design process. Here we characterize withdrawn drugs using various features and apply this knowledge in combination with an ensemble predictor to anticipate drug withdrawal status earlier in drug development.\\u003c/p\\u003e\\u003cp\\u003eOur study provides an overview for landscape of withdrawn drugs. Side-effects associated with withdrawn drugs (derived from ChEMBL) primarily affect three organ systems including the liver, the cardiovascular system, and the neuropsychiatric system (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig1\\\" class=\\\"InternalRef\\\"\\u003e1\\u003c/span\\u003e). This is consistent with multiple studies investigating drug toxicity\\u003csup\\u003e\\u003cspan citationid=\\\"CR28\\\" class=\\\"CitationRef\\\"\\u003e28\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR29\\\" class=\\\"CitationRef\\\"\\u003e29\\u003c/span\\u003e\\u003c/sup\\u003e. Moreover, comprehensive analysis of ATC codes (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e), enabled comparisons of trends in withdrawn and non-withdrawn drugs. In the case of antineoplastic drugs, underlying disease severity and resistance development may influence withdrawal status. For example, although vincristine is associated with serious neuropathy, it is still an important part of the regime used to treat childhood cancers such as acute lymphoblastic leukemia, Hodgkin lymphoma, rhabdomyosarcoma, nephroblastoma, medulloblastoma, and low-grade glioma\\u003csup\\u003e\\u003cspan citationid=\\\"CR30\\\" class=\\\"CitationRef\\\"\\u003e30\\u003c/span\\u003e\\u003c/sup\\u003e. Similarly, the potential for multidrug resistance increases the need for numerous approved antineoplastic therapies in order to allow for multidrug therapies\\u003csup\\u003e\\u003cspan citationid=\\\"CR31\\\" class=\\\"CitationRef\\\"\\u003e31\\u003c/span\\u003e\\u003c/sup\\u003e. Antiparasitic drugs also have no withdrawn treatments. However, in contrast to antineoplastic drugs, the shorter treatment course of certain antiparasitic drugs- less than 90 days\\u003csup\\u003e\\u003cspan citationid=\\\"CR32\\\" class=\\\"CitationRef\\\"\\u003e32\\u003c/span\\u003e\\u003c/sup\\u003e- is hypothesized to play a role in this trend. The high number of withdrawn drugs in other categories could potentially result from the number of other existing treatments for diseases in these categories combined with a different assessment of a toxicity to benefit ratio\\u003csup\\u003e\\u003cspan citationid=\\\"CR33\\\" class=\\\"CitationRef\\\"\\u003e33\\u003c/span\\u003e\\u003c/sup\\u003e. Taken together, this analysis suggests the possible underlying influence of treatment course and disease severity on withdrawal status, as well as demonstrates current priorities in research and development strategies.\\u003c/p\\u003e\\u003cp\\u003eTo gain more insight into what makes a drug toxic, we characterized drugs based on their chemical and pharmacological features. Feature selection using a combination of three methods distinguished important characteristics of globally withdrawn drugs that can be employed in future drug development. Interestingly, the top four chemical features overrepresented in withdrawn drugs as compared to drugs still in use are related to drug lipophilicity. In particular, logP and logD could be informative because they capture drug distribution throughout the body, with higher concentrations of drug in more locations increasing the potential for binding off-target increasing side effects. Consistent with these results, underrepresented features in withdrawn drugs include molecular weight, frequency of heavy atoms, polar surface area, and quantitative estimate of drug-likeness, which also affect the drug\\u0026rsquo;s ability to cross lipid membrane bilayers, translating to drug oral bioavailability\\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR34\\\" class=\\\"CitationRef\\\"\\u003e34\\u003c/span\\u003e\\u003c/sup\\u003e. Route of administration, another set of features extracted by this analysis, is vital for the understanding of the speed and level of systemic reach of a compound upon use. Indeed, chemical features highly correlated with drug withdrawal including route of administration and pKa reflect known pharmacological ADME design trends (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e). These features can be used in the chemical design process and provide additional confidence in our approach as they overlap with existing drug chemical structure rules\\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR34\\\" class=\\\"CitationRef\\\"\\u003e34\\u003c/span\\u003e\\u003c/sup\\u003e. In the future, drugs with properties that have values in the withdrawn range may become candidate for more thorough screening or may be altered chemically during early stages of development.\\u003c/p\\u003e\\u003cp\\u003eSimilarly, our feature selection approach highlighted the importance of certain drug proteins targets, some of which have known and experimentally validated toxicity. Cytochrome P450 (3A4, 2C19, 2J2), one of the most informative proteins corelated with toxicity according to our feature selection paradigm, is well established as a toxic drug target due to its ability to modulate the metabolism of other drugs\\u003csup\\u003e\\u003cspan citationid=\\\"CR25\\\" class=\\\"CitationRef\\\"\\u003e25\\u003c/span\\u003e\\u003c/sup\\u003e, involvement in drug-drug interactions\\u003csup\\u003e\\u003cspan citationid=\\\"CR35\\\" class=\\\"CitationRef\\\"\\u003e35\\u003c/span\\u003e\\u003c/sup\\u003e and pharmacogenetics\\u003csup\\u003e\\u003cspan citationid=\\\"CR36\\\" class=\\\"CitationRef\\\"\\u003e36\\u003c/span\\u003e\\u003c/sup\\u003e. Certain membrane transporters, which often have similar pharmacological role to that of metabolic enzymes in controlling drug ADME\\u003csup\\u003e\\u003cspan citationid=\\\"CR26\\\" class=\\\"CitationRef\\\"\\u003e26\\u003c/span\\u003e\\u003c/sup\\u003e, are also implicated in toxicity (Supplementary Table\\u0026nbsp;2). For example, drug modulation of the ATP-binding cassette sub-family results in cardiotoxicity\\u003csup\\u003e\\u003cspan citationid=\\\"CR37\\\" class=\\\"CitationRef\\\"\\u003e37\\u003c/span\\u003e\\u003c/sup\\u003e. The bile salt export pump (ABCB11) has known association with hepatotoxicity\\u003csup\\u003e\\u003cspan citationid=\\\"CR38\\\" class=\\\"CitationRef\\\"\\u003e38\\u003c/span\\u003e\\u003c/sup\\u003e. Interestingly, the sodium dependent dopamine transporter DAT (SLC6A3), which is targeted by many CNS drugs such as SSRIs and SNRIs, was also correlated with toxicity in our study. This transporter is not known to be related to pharmacokinetics, and thus, it is not regularly screened by for activity during drug development like the drug ADME transporters above. Features of these proteins such as composition of helical regions and number of amino acid contacts also play a large role in the pattern of drug toxicity further explored by our predictive machine learning model. Interestingly, these features are commonly observed in membrane proteins, such as the transporters discussed\\u003csup\\u003e\\u003cspan citationid=\\\"CR39\\\" class=\\\"CitationRef\\\"\\u003e39\\u003c/span\\u003e\\u003c/sup\\u003e. Our work therefore suggests additional proteins and structural features of proteins which would be beneficial to include in drug toxicity screening panels.\\u003c/p\\u003e\\u003cp\\u003eTo utilize our new insights on drug toxicity, we developed a screening tool for flagging potentially toxic drugs. We built an ensemble predictor (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig4\\\" class=\\\"InternalRef\\\"\\u003e4\\u003c/span\\u003eA), Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH), which applies classifiers to various chemical and protein features to label a drug with its withdrawal potential. The 10-fold cross-validated ensemble predictor assessed on balanced, holdout test sets scored an accuracy of 92.0%, F1 of 0.919, and a MCC of 0.845 (Table\\u0026nbsp;2).\\u003c/p\\u003e\\u003cp\\u003eWe also analyzed the clinical trial drug landscape and applied the WATCH predictor to these compounds. The indications of clinical trial drugs reflect a slight predominance of new drugs targeting the alimentary and metabolism (16%) system. (Supplementary Fig.\\u0026nbsp;3). Additional indications common in drugs currently in clinical trials include antineoplastic and immune (13%), cardiovascular (12%), and nervous system (13%) drugs. With the exception of antineoplastic drugs, which do not have as many drugs available as these other groups, these are all areas in which more drugs have been withdrawn than are currently available (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e). The preponderance of withdrawn drugs in these categories points to unmet needs that new drugs are being designed to fill. WATCH applied to drugs in clinical trials distinguished a number of drugs that are expected to be withdrawn from the market (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig5\\\" class=\\\"InternalRef\\\"\\u003e5\\u003c/span\\u003e). Interestingly, Flindokalner targets potassium ion channels\\u003csup\\u003e\\u003cspan citationid=\\\"CR40\\\" class=\\\"CitationRef\\\"\\u003e40\\u003c/span\\u003e\\u003c/sup\\u003e, which overlaps with the know toxic targets found through recursive feature elimination, and is indicated for migraines and post-traumatic headaches (NCT03887325). Levosulpiride is indicated for gastric motility disorders. In trials this drug has been observed to cause severe side effects such as acute dystonias and Parkinson like symptoms\\u003csup\\u003e\\u003cspan citationid=\\\"CR41\\\" class=\\\"CitationRef\\\"\\u003e41\\u003c/span\\u003e\\u003c/sup\\u003e. Amithiozone has been used as a treatment for tuberculosis, but was replaced with newer, less toxic antibiotics\\u003csup\\u003e\\u003cspan citationid=\\\"CR42\\\" class=\\\"CitationRef\\\"\\u003e42\\u003c/span\\u003e\\u003c/sup\\u003e. Currently, Amithiozone is being re-investigated as a therapy for Mycobacterium Avium Complex\\u003csup\\u003e\\u003cspan citationid=\\\"CR43\\\" class=\\\"CitationRef\\\"\\u003e43\\u003c/span\\u003e\\u003c/sup\\u003e. AZD1981 is a CRTh2 receptor antagonist indicated as an asthma therapy\\u003csup\\u003e\\u003cspan citationid=\\\"CR44\\\" class=\\\"CitationRef\\\"\\u003e44\\u003c/span\\u003e\\u003c/sup\\u003e. Incyclinide is an antineoplastic agent\\u003csup\\u003e\\u003cspan citationid=\\\"CR45\\\" class=\\\"CitationRef\\\"\\u003e45\\u003c/span\\u003e\\u003c/sup\\u003e. In summary, many of the drugs flagged by the withdrawn prediction model, have already begun to demonstrate serious side effects in clinical trials. This demonstrates the applicability of the predictor for future drug toxicity investigations. Furthermore, the characterization of withdrawn drugs and WATCH are meant to serve as tools for drug developers in the ever-evolving endeavor to predict the toxicity of candidate compounds.\\u003c/p\\u003e\"},{\"header\":\"METHODS\",\"content\":\"\\u003cdiv id=\\\"Sec10\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eDrug ATC analysis\\u003c/h2\\u003e\\u003cp\\u003eUsing the withdrawal reasons and ATC datasets, we analyzed the main organ systems that drugs target. We manually grouped the non-standardized labels describing reasons for drug withdrawal (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig1\\\" class=\\\"InternalRef\\\"\\u003e1\\u003c/span\\u003e, Supplemental Fig.\\u0026nbsp;2). In addition, to investigate the differences between organ systems of action in withdrawn and not withdrawn drugs, we used the anatomic therapeutic chemical (ATC) code\\u003csup\\u003e\\u003cspan citationid=\\\"CR22\\\" class=\\\"CitationRef\\\"\\u003e22\\u003c/span\\u003e\\u003c/sup\\u003e designation, the first letter of which indicates anatomical main group\\u003csup\\u003e\\u003cspan citationid=\\\"CR46\\\" class=\\\"CitationRef\\\"\\u003e46\\u003c/span\\u003e\\u003c/sup\\u003e. Frequencies of withdrawn and not withdrawn drugs for each therapeutic indication category were compared using the scipy.stats.fisher_exact test 1.6.0 (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e). Drugs without an ATC code were omitted. The Bonferroni\\u003csup\\u003e\\u003cspan citationid=\\\"CR47\\\" class=\\\"CitationRef\\\"\\u003e47\\u003c/span\\u003e\\u003c/sup\\u003e adjusted p value was 0.0036. It is possible for a drug to have more than one reason for withdrawal and organ system of indication.\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec11\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eFeature selection\\u003c/h2\\u003e\\u003cp\\u003eCharacteristics that most distinguished between withdrawn and not withdrawn drugs were determined with three different feature selection methods on three sets of data features (drug molecular features, drug targets, and drug target features). Specifically, the methods we used were: (i) recursive feature elimination with cross validation (RFECV) in scikit learn 1.1.2. The cross validation splitting strategy (cv) equaled 5 folds, the steps value equaled 1, and the classifier was a random forest model; (ii) Scikit learn 1.1.2. variance threshold with a threshold of 0; and (iii) generic univariate selection for the top 25% of features with a chi squared score function was also applied. These three feature selection methods were applied to each dataset separately 10 times using a random balanced 80% of the data each time and random seeds 0\\u0026ndash;9. Scores were normalized to be out of 10, where the best features would score a 10, and combined for each feature selection model with equal weights.\\u003c/p\\u003e\\u003cp\\u003eFor visualization of the drug chemical features, the average values of continuous variables and the frequency counts normalized by the total number of drugs in each category for categorical variables were calculated for withdrawn and not withdrawn drugs. For each statistically significantly different chemical feature variable, the log base 2 value of the ratio of the withdrawn drug divided by the not withdrawn drug was plotted (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e).\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec12\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003ePrediction of compound withdrawal status\\u003c/h2\\u003e\\u003cp\\u003eDrug withdrawal status for existing drugs was predicted using a variety of machine learning approaches. First the fingerprint, drug chemical features, target, and target feature datasets were used independently to generate predictions. Each of the datasets were randomly split into balanced 80-10-10 percent segments. These represent the training, first test, and second test sets respectively. Balancing of the sets ensured an equal number of withdrawn and not withdrawn drugs in each training and testing tranche. Each feature was min-max normalized (Eq.\\u0026nbsp;1) using the minimum and maximum values of the feature in the training set. The Tree-based Pipeline Optimization Tool (TPOT)\\u003csup\\u003e\\u003cspan citationid=\\\"CR48\\\" class=\\\"CitationRef\\\"\\u003e48\\u003c/span\\u003e\\u003c/sup\\u003e 0.11.7 was used to tune hyperparameters for TPOT neural networks, scikit learn random forest, scikit learn multilayer perceptron, and xgboost extreme gradient boosting. The parameters for TPOT were: population size of 24, offspring size of 12, early stop of 12, scoring using accuracy, cv of 5, and generations of 5. The parameter search space for the models are as follows. Random forest: criterion was entropy or gini; max depth was none and 10 through 500; max features were auto, square root, log2 or none; min samples per leaf were 2 through 15, min sample split was 5 through 15; number of estimators were 150 through 500. Multilayer perceptron: activation was identity, loginst, tanh, or relu; hidden layer sizes were 25 through 400; solver was lbfgs, stochastic gradient descent, or adam. Extreme gradient boosting: max depth was none or 10 through 500; number of estimators was 150 through 500. Following hyperparameter tuning, the top performing model was selected for each data set.\\u003c/p\\u003e\\u003cp\\u003eAccuracy, area under the receiver operating characteristic curve (AUROC), F1, precision, recall, and Matthews Correlation Coefficient (MCC) were calculated with scikit learn on the first test set.\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec13\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003e\\u003cb\\u003eEquation 1\\u003c/b\\u003e \\u003cspan class=\\\"InlineEquation\\\"\\u003e\\u003cspan class=\\\"mathinline\\\"\\u003e\\\\(\\\\:normalization=\\\\frac{value-\\\\text{m}\\\\text{i}\\\\text{n}\\\\left(feature\\\\:values\\\\right)}{\\\\text{max}\\\\left(feature\\\\:values\\\\right)-\\\\text{m}\\\\text{i}\\\\text{n}\\\\left(feature\\\\:values\\\\right)}\\\\)\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/h2\\u003e\\u003cp\\u003eThe output of the best overall performing model for each of the 4 datasets was used to generate the training and test sets for the meta classifier, Withdrawal Anticipation and Toxicity Characterization of Hits (WATCH). These were the models that maximized the sum of the accuracy, AUROC, F1, precision, recall, and MCC. The training set consisted of the training set for the previous model in addition to the test set for the first model. The test set was the second holdout test set from the initial data split. Hyperparameter tuning and model selection for the meta classifier proceeded as previously described. The best meta classifier model, WATCH, was a K nearest neighbor classifier with 57 neighbors, and distance weights. This was compared to a meta classifier model that averaged the predictions for the best models on the 4 datasets. Accuracy, area under the receiver operating characteristic curve (AUROC), F1, precision, and recall were calculated with scikit learn on the second, holdout test set.\\u003c/p\\u003e\\u003cp\\u003eThe best performing classifier for the target structural feature dataset was a multilayer perceptron with identity activation, 150 hidden layers, and a stochastic gradient decent solver. The best performing classifier for the proteins was a RBF sampler with 0.05 gamma and a gradient boosting classifier with 0.1 learning rate, 9 max depth, 0.65 maximum features, 14 minimum samples per leaf, 15 minimum samples per split, 100 estimators, and 0.5 subsamples. The best performing classifier for the drug chemical features dataset was an extra trees classifier with bootstrap, gini criterion, 0.65 maximum features, 7 minimum samples per leaf, and 100 estimators stacked with a decision tree with gini criterion, maximum depth of 5, 18 minimum samples per leaf, and a minimum samples per split of 3. The best performing classifier for the drug fingerprint dataset was a random forest classifier with entropy criterion, 64 maximum depth, log2 maximum features, 2 minimum samples per leaf, 5 minimum samples per split, and 188 estimators. (Fig.\\u0026nbsp;4BC)\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec14\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003ePrediction of toxicity for drugs under clinical investigation\\u003c/h2\\u003e\\u003cp\\u003eWe applied WATCH to existing drugs to drugs currently in clinical trials (1216 drugs). The classifier selected for WATCH was the K nearest neighbor meta classifier, which maximized the sum of the accuracy, AUROC, F1, precision, recall, and MCC. WATCH, run 10 times with random states 0\\u0026ndash;9, was used to classify all drugs found in ChEMBL currently in phase I-III clinical trials (downloaded using chembl-webresource-client 0.10.8 on January 16, 2023). The output for the 10 runs of WATCH for each drug was summed. The closer the resulting value a drug had to 10, the more likely it is to be withdrawn in the future.\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec15\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eDATA AVAILABILITY\\u003c/h2\\u003e\\u003cdiv id=\\\"Sec16\\\" class=\\\"Section3\\\"\\u003e\\u003ch2\\u003eDrug datasets\\u003c/h2\\u003e\\u003cp\\u003eWe constructed four drug feature datasets, including drug withdrawal status dataset, drug indication dataset, drug fingerprint dataset, and drug feature dataset. The withdrawal status dataset contains drug withdrawal status and reasons for drug withdrawal collected from ChEMBL\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e using chembl-webresource-client 0.10.8. Of the 230 drugs listed as withdrawn, 120 had reasons for withdrawal found in the database. We manually curated these non-standardized reasons for withdrawals into logical uniform labels (for example \\u0026ldquo;Hepatic toxicity\\u0026rdquo; and \\u0026ldquo;Hepatotoxicity\\u0026rdquo; were combined under the label of Hepatotoxicity). We further grouped each toxicity by the organ system most effected (for example Liver for Hepatotoxicity).\\u003c/p\\u003e\\u003cp\\u003eThe drug indication dataset contains the anatomic therapeutic chemical (ATC) codes, which classifies drugs by site of action and chemical characteristics\\u003csup\\u003e\\u003cspan citationid=\\\"CR35\\\" class=\\\"CitationRef\\\"\\u003e35\\u003c/span\\u003e\\u003c/sup\\u003e, for withdrawn (n\\u0026thinsp;=\\u0026thinsp;120) and not withdrawn drugs (n\\u0026thinsp;=\\u0026thinsp;1511) also determined using ChEMBL\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e.\\u003c/p\\u003e\\u003cp\\u003eThe drug fingerprint dataset contains Morgan fingerprints\\u003csup\\u003e\\u003cspan citationid=\\\"CR49\\\" class=\\\"CitationRef\\\"\\u003e49\\u003c/span\\u003e\\u003c/sup\\u003e with radius 2 and 512 bits as vectors generated with Rdkit 2022.09.1 for withdrawn and not withdrawn drugs.\\u003c/p\\u003e\\u003cp\\u003eThe drug feature dataset contains chemical features of withdrawn and not withdrawn drugs found in ChEMBL\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e. The molecular features included in this dataset are: chirality, partition coefficient (alogp (calculated), cx_logp (generated with ACDlabs v12.01)), number of aromatic rings, full molecular weight (including salts), freebase weight (weight of parent compound), and monoisotopic weight (weight of the monoisotopic parent compound), route of administration (oral, parentreral, or topical), status as a prodrug, distribution coefficient (cx_logd (generated using ACDlabs v12.01), the most acidic acid dissociation constant (cx_most_apka), most basic acid dissociation constant (cx_most_bpka), number of hydrogen bond acceptors (hba, hba_lipinski), number of hydrogen bond donners (hbd, hbd_lipinski), number of heavy atoms, identity as an acid, identity as a base, identity as a neutral molecule, identity as a zwitterion, drug likeness (num_lipinski_ro5_violations (number of rule of 5 violations using Lipinski hydrogen bond acceptor and donor definitions), num_ro5_violations (number of rule of 5 violations using hydrogen bond acceptor and donor definitions), qed_weighted (quantitative estimate of drug likeness\\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e\\u003c/sup\\u003e)), polar surface area (psa), rule of three (ro3_pass\\u003csup\\u003e\\u003cspan citationid=\\\"CR50\\\" class=\\\"CitationRef\\\"\\u003e50\\u003c/span\\u003e\\u003c/sup\\u003e), and number of rotatable bonds (rtb).\\u003c/p\\u003e\\u003c/div\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec17\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eDrug target datasets\\u003c/h2\\u003e\\u003cp\\u003eWe also constructed two drug target feature datasets, including IC\\u003csub\\u003e50\\u003c/sub\\u003e values dataset, and drug target features. The drug target dataset contains IC\\u003csub\\u003e50\\u003c/sub\\u003e values for each human protein drug target found in ChEMBL\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e.\\u003c/p\\u003e\\u003cp\\u003eThe drug target features were derived from the amino acid sequence and structural models of the proteins in the drug target dataset (Supplementary Table\\u0026nbsp;3). The method for feature generation has been previously described\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e. In brief, structural features, such as protein secondary structure or intrinsically disordered regions, describe characteristics of protein regions. These structural features are determined from either protein sequence, using the methods IUPRED2a\\u003csup\\u003e\\u003cspan citationid=\\\"CR51\\\" class=\\\"CitationRef\\\"\\u003e51\\u003c/span\\u003e\\u003c/sup\\u003e, ANCHOR\\u003csup\\u003e\\u003cspan citationid=\\\"CR52\\\" class=\\\"CitationRef\\\"\\u003e52\\u003c/span\\u003e\\u003c/sup\\u003e, TMHMM 2.0\\u003csup\\u003e53\\u003c/sup\\u003e, PredictProtein\\u003csup\\u003e\\u003cspan citationid=\\\"CR54\\\" class=\\\"CitationRef\\\"\\u003e54\\u003c/span\\u003e\\u003c/sup\\u003e, SCOPe\\u003csup\\u003e\\u003cspan citationid=\\\"CR55\\\" class=\\\"CitationRef\\\"\\u003e55\\u003c/span\\u003e\\u003c/sup\\u003e; or from three-dimensional model predicted with AlphaFold2\\u003csup\\u003e56\\u003c/sup\\u003e, using the methods Dictionary of Protein Secondary Structure (DSSP)\\u003csup\\u003e\\u003cspan citationid=\\\"CR57\\\" class=\\\"CitationRef\\\"\\u003e57\\u003c/span\\u003e\\u003c/sup\\u003e, biopython\\u0026rsquo;s pdb parser\\u003csup\\u003e\\u003cspan citationid=\\\"CR58\\\" class=\\\"CitationRef\\\"\\u003e58\\u003c/span\\u003e\\u003c/sup\\u003e, Aggrescan3d\\u003csup\\u003e\\u003cspan citationid=\\\"CR59\\\" class=\\\"CitationRef\\\"\\u003e59\\u003c/span\\u003e\\u003c/sup\\u003e. Cutoffs for classifying region identity have been determined in previous work\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e. Features for each protein were normalized by the length of protein and then weighted according to Eq.\\u0026nbsp;2. Where F is the normalization value of the protein feature, f is the unnormalized value of the protein feature, sIC\\u003csub\\u003e50\\u003c/sub\\u003e is the set of IC\\u003csub\\u003e50\\u003c/sub\\u003e values for the protein tested with all drugs, and pIC\\u003csub\\u003e50\\u003c/sub\\u003e is the IC\\u003csub\\u003e50\\u003c/sub\\u003e of the protein being normalized.\\u003c/p\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec18\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003e\\u003cb\\u003eEquation 2\\u003c/b\\u003e \\u003cspan class=\\\"InlineEquation\\\"\\u003e\\u003cspan class=\\\"mathinline\\\"\\u003e\\\\(\\\\:F=\\\\:f\\\\frac{\\\\text{max}\\\\left({sIC}_{50}\\\\right)-\\\\:{pIC}_{50}+1}{\\\\text{m}\\\\text{a}\\\\text{x}\\\\left(s{IC}_{50}\\\\right)\\\\:}\\\\)\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/h2\\u003e\"},{\"header\":\"Declarations\",\"content\":\"\\u003cdiv id=\\\"Sec19\\\" class=\\\"Section3\\\"\\u003e\\u003ch2\\u003eCODE AVAILABILITY\\u003c/h2\\u003e\\u003cp\\u003eA total of 1,296 compounds (204 withdrawn and 1,092 not withdrawn) were found having fingerprints, targets, target features, and drug molecular features. A full list of drugs and their features can be found in the project GitHub repository: \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://github.com/schlessinger-lab/withDRAWN\\u003c/span\\u003e\\u003cspan address=\\\"https://github.com/schlessinger-lab/withDRAWN\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e.\\u003c/p\\u003e\\u003c/div\\u003e\\u003c/div\\u003e\\u003cdiv id=\\\"Sec20\\\" class=\\\"Section2\\\"\\u003e\\u003ch2\\u003eACKNOWLEGMENTS\\u003c/h2\\u003e\\u003cp\\u003eThis work is supported in part by the National Institutes of Health grant F30 HL160179 and T32 HD075735 to N.Z and R01 HD107528 to A.S\\u003c/p\\u003e\\u003c/div\\u003e\\u003cp\\u003e\\u003ch2\\u003eCOMPETING INTERESTS\\u003c/h2\\u003e\\u003cp\\u003eThe authors have no conflicts of interest to declare.\\u003c/p\\u003e\\u003c/p\\u003e\\u003ch2\\u003eAuthor Contribution\\u003c/h2\\u003e\\u003cp\\u003eN.Z. and A.S. conducted the analysis and wrote this manuscript.\\u003c/p\\u003e\\u003ch2\\u003eData Availability\\u003c/h2\\u003e\\u003cp\\u003eData and code can be found in the GitHub repository: https://github.com/schlessinger-lab/withDRAWN.\\u003c/p\\u003e\"},{\"header\":\"References\",\"content\":\"\\u003col\\u003e\\u003cli\\u003e\\u003cspan\\u003eWysowski, D. K. \\u0026amp; Swartz, L. Adverse drug event surveillance and drug withdrawals in the United States, 1969\\u0026ndash;2002: the importance of reporting suspected reactions. \\u003cem\\u003eArch Intern Med\\u003c/em\\u003e 165, 1363\\u0026ndash;1369 (2005). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1001/archinte.165.12.1363\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1001/archinte.165.12.1363\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eMcNaughton, R., Huet, G. \\u0026amp; Shakir, S. An investigation into drug products withdrawn from the EU market between 2002 and 2011 for safety reasons and the evidence used to support the decision-making. \\u003cem\\u003eBMJ Open\\u003c/em\\u003e 4, e004221 (2014). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1136/bmjopen-2013-004221\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1136/bmjopen-2013-004221\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eIshiguro, C., Misu, T., Iwasa, E. \\u0026amp; Izawa, T. Analysis of safety-related regulatory actions by Japan's pharmaceutical regulatory agency. \\u003cem\\u003ePharmacoepidemiol Drug Saf\\u003c/em\\u003e 26, 1314\\u0026ndash;1320 (2017). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1002/pds.4252\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1002/pds.4252\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eAlves, C., Macedo, A. F. \\u0026amp; Marques, F. B. Sources of information used by regulatory agencies on the generation of drug safety alerts. \\u003cem\\u003eEur J Clin Pharmacol\\u003c/em\\u003e 69, 2083\\u0026ndash;2094 (2013). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1007/s00228-013-1564-y\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1007/s00228-013-1564-y\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eEichler, H. G. \\u003cem\\u003eet al.\\u003c/em\\u003e Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers. \\u003cem\\u003eNat Rev Drug Discov\\u003c/em\\u003e 9, 277\\u0026ndash;291 (2010). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nrd3079\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nrd3079\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eIfeoma Obidike, A. M., Michaela Reddy, Harvey Clewell, Thierry Lave, Melvin Andersen, Ray Greek, Abdelmigid, Nasir Mohamad, Peter Ward, David La, J. Eric McDuffie, Sandra Snook, Elsa Dias, Carina Menezes, Elisabete Valerio, Jose M Fuentes, Jacob John Van Tonder, Mary Gulumian, Vanessa Steenkamp. \\u003cem\\u003eNew Insights into Toxicity and Drug Testing\\u003c/em\\u003e. (InTech, 2013).\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eLipsky, M. S. \\u0026amp; Sharp, L. K. From idea to market: the drug approval process. \\u003cem\\u003eJ Am Board Fam Pract\\u003c/em\\u003e 14, 362\\u0026ndash;367 (2001).\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eTropsha, A. Best Practices for QSAR Model Development, Validation, and Exploitation. \\u003cem\\u003eMol Inform\\u003c/em\\u003e 29, 476\\u0026ndash;488 (2010). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1002/minf.201000061\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1002/minf.201000061\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eRaschi, E., Ceccarini, L., De Ponti, F. \\u0026amp; Recanatini, M. hERG-related drug toxicity and models for predicting hERG liability and QT prolongation. \\u003cem\\u003eExpert Opin Drug Metab Toxicol\\u003c/em\\u003e 5, 1005\\u0026ndash;1021 (2009). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1517/17425250903055070\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1517/17425250903055070\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eVo, A. H., Van Vleet, T. R., Gupta, R. R., Liguori, M. J. \\u0026amp; Rao, M. S. An Overview of Machine Learning and Big Data for Drug Toxicity Evaluation. \\u003cem\\u003eChem Res Toxicol\\u003c/em\\u003e 33, 20\\u0026ndash;37 (2020). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1021/acs.chemrestox.9b00227\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1021/acs.chemrestox.9b00227\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eWang, H., Liu, R., Schyman, P. \\u0026amp; Wallqvist, A. Deep Neural Network Models for Predicting Chemically Induced Liver Toxicity Endpoints From Transcriptomic Responses. \\u003cem\\u003eFront Pharmacol\\u003c/em\\u003e 10, 42 (2019). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.3389/fphar.2019.00042\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.3389/fphar.2019.00042\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eIftkhar, S. \\u003cem\\u003eet al.\\u003c/em\\u003e cardioToxCSM: A Web Server for Predicting Cardiotoxicity of Small Molecules. \\u003cem\\u003eJ Chem Inf Model\\u003c/em\\u003e 62, 4827\\u0026ndash;4836 (2022). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1021/acs.jcim.2c00822\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1021/acs.jcim.2c00822\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eBanerjee, P., Eckert, A. O., Schrey, A. K. \\u0026amp; Preissner, R. ProTox-II: a webserver for the prediction of toxicity of chemicals. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 46, W257-W263 (2018). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gky318\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gky318\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eZatorski, N. \\u003cem\\u003eet al.\\u003c/em\\u003e Structural analysis of genomic and proteomic signatures reveal dynamic expression of intrinsically disordered regions in breast cancer. \\u003cem\\u003eiScience\\u003c/em\\u003e 27, 1\\u0026ndash;16 (2024). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.isci.2024.110640\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.isci.2024.110640\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eZhang, Q. C. \\u003cem\\u003eet al.\\u003c/em\\u003e Structure-based prediction of protein-protein interactions on a genome-wide scale. \\u003cem\\u003eNature\\u003c/em\\u003e 490, 556\\u0026ndash;560 (2012). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nature11503\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nature11503\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eWang, X. \\u003cem\\u003eet al.\\u003c/em\\u003e Three-dimensional reconstruction of protein networks provides insight into human genetic disease. \\u003cem\\u003eNat Biotechnol\\u003c/em\\u003e 30, 159\\u0026ndash;164 (2012). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nbt.2106\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nbt.2106\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eRahman, R. \\u003cem\\u003eet al.\\u003c/em\\u003e Protein structure-based gene expression signatures. \\u003cem\\u003eProc Natl Acad Sci U S A\\u003c/em\\u003e 118 (2021). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1073/pnas.2014866118\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1073/pnas.2014866118\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eZatorski, N., Stein, D., Rahman, R., Iyengar, R. \\u0026amp; Schlessinger, A. Structural signatures: a web server for exploring a database of and generating protein structural features from human cell lines and tissues. \\u003cem\\u003eDatabase (Oxford)\\u003c/em\\u003e 2022 (2022). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/database/baac053\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/database/baac053\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eGaulton, A. \\u003cem\\u003eet al.\\u003c/em\\u003e The ChEMBL database in 2017. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 45, D945-D954 (2017). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gkw1074\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gkw1074\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eSachana, M., Shafer, T. J. \\u0026amp; Terron, A. Toward a Better Testing Paradigm for Developmental Neurotoxicity: OECD Efforts and Regulatory Considerations. \\u003cem\\u003eBiology (Basel)\\u003c/em\\u003e 10 (2021). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.3390/biology10020086\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.3390/biology10020086\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eSchlessinger, A. \\u003cem\\u003eet al.\\u003c/em\\u003e Structure-based discovery of prescription drugs that interact with the norepinephrine transporter, NET. \\u003cem\\u003eProc Natl Acad Sci U S A\\u003c/em\\u003e 108, 15810\\u0026ndash;15815 (2011). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1073/pnas.1106030108\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1073/pnas.1106030108\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eHvidberg, E. \\u0026amp; Andersen, A. H. [New classification of drugs. The Medical list and the Drug catalogue are introduced in Anatomical\\u0026ndash;Therapeutic\\u0026ndash;Chemical classification code (ACT-code) in 1981]. \\u003cem\\u003eUgeskr Laeger\\u003c/em\\u003e 142, 396\\u0026ndash;397 (1980).\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eDarst, B. F., Malecki, K. C. \\u0026amp; Engelman, C. D. Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. \\u003cem\\u003eBMC Genet\\u003c/em\\u003e 19, 65 (2018). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1186/s12863-018-0633-8\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1186/s12863-018-0633-8\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eBickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S. \\u0026amp; Hopkins, A. L. Quantifying the chemical beauty of drugs. \\u003cem\\u003eNat Chem\\u003c/em\\u003e 4, 90\\u0026ndash;98 (2012). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nchem.1243\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nchem.1243\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eBowes, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Reducing safety-related drug attrition: the use of in vitro pharmacological profiling. \\u003cem\\u003eNat Rev Drug Discov\\u003c/em\\u003e 11, 909\\u0026ndash;922 (2012). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nrd3845\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nrd3845\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eInternational Transporter, C. \\u003cem\\u003eet al.\\u003c/em\\u003e Membrane transporters in drug development. \\u003cem\\u003eNat Rev Drug Discov\\u003c/em\\u003e 9, 215\\u0026ndash;236 (2010). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nrd3028\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nrd3028\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eZamek-Gliszczynski, M. J. \\u003cem\\u003eet al.\\u003c/em\\u003e Transporters in Drug Development: International Transporter Consortium Update on Emerging Transporters of Clinical Importance. \\u003cem\\u003eClin Pharmacol Ther\\u003c/em\\u003e 112, 485\\u0026ndash;500 (2022). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1002/cpt.2644\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1002/cpt.2644\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eWong, C. H., Siah, K. W. \\u0026amp; Lo, A. W. Estimation of clinical trial success rates and related parameters. \\u003cem\\u003eBiostatistics\\u003c/em\\u003e 20, 273\\u0026ndash;286 (2019). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/biostatistics/kxx069\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/biostatistics/kxx069\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eCook, D. \\u003cem\\u003eet al.\\u003c/em\\u003e Lessons learned from the fate of AstraZeneca's drug pipeline: a five-dimensional framework. \\u003cem\\u003eNat Rev Drug Discov\\u003c/em\\u003e 13, 419\\u0026ndash;431 (2014). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/nrd4309\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/nrd4309\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003evan de Velde, M. E. \\u003cem\\u003eet al.\\u003c/em\\u003e Genetic Polymorphisms Associated with Vincristine Pharmacokinetics and Vincristine-Induced Peripheral Neuropathy in Pediatric Oncology Patients. \\u003cem\\u003eCancers (Basel)\\u003c/em\\u003e 14 (2022). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.3390/cancers14143510\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.3390/cancers14143510\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eCho, H. \\u0026amp; Levy, D. Modeling the chemotherapy-induced selection of drug-resistant traits during tumor growth. \\u003cem\\u003eJ Theor Biol\\u003c/em\\u003e 436, 120\\u0026ndash;134 (2018). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.jtbi.2017.10.005\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.jtbi.2017.10.005\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKappagoda, S., Singh, U. \\u0026amp; Blackburn, B. G. Antiparasitic therapy. \\u003cem\\u003eMayo Clin Proc\\u003c/em\\u003e 86, 561\\u0026ndash;583 (2011). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.4065/mcp.2011.0203\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.4065/mcp.2011.0203\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eFung, M., Anna Thornton, Kathy Mybeck, Jasmanda Hsiao-hui Wu, Ken Hornbuckle, and Edmundo Muniz. Evaluation of the characteristics of safety withdrawal of prescription drugs from worldwide pharmaceutical markets-1960 to 1999. \\u003cem\\u003eDrug Information Journal\\u003c/em\\u003e 35, 293\\u0026ndash;317 (2001).\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eLipinski, C. A., Lombardo, F., Dominy, B. W. \\u0026amp; Feeney, P. J. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. \\u003cem\\u003eAdv Drug Deliv Rev\\u003c/em\\u003e 46, 3\\u0026ndash;26 (2001). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/s0169-409x(00)00129-0\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/s0169-409x(00)00129-0\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eGuengerich, F. P. Role of cytochrome P450 enzymes in drug-drug interactions. \\u003cem\\u003eAdv Pharmacol\\u003c/em\\u003e 43, 7\\u0026ndash;35 (1997). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/s1054-3589(08)60200-8\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/s1054-3589(08)60200-8\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eIngelman-Sundberg, M. Pharmacogenetics of cytochrome P450 and its applications in drug therapy: the past, present and future. \\u003cem\\u003eTrends Pharmacol Sci\\u003c/em\\u003e 25, 193\\u0026ndash;200 (2004). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.tips.2004.02.007\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.tips.2004.02.007\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eCouture, L., Nash, J. A. \\u0026amp; Turgeon, J. The ATP-binding cassette transporters and their implication in drug disposition: a special look at the heart. \\u003cem\\u003ePharmacol Rev\\u003c/em\\u003e 58, 244\\u0026ndash;258 (2006). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1124/pr.58.2.7\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1124/pr.58.2.7\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKenna, J. G. \\u003cem\\u003eet al.\\u003c/em\\u003e Can Bile Salt Export Pump Inhibition Testing in Drug Discovery and Development Reduce Liver Injury Risk? An International Transporter Consortium Perspective. \\u003cem\\u003eClin Pharmacol Ther\\u003c/em\\u003e 104, 916\\u0026ndash;932 (2018). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1002/cpt.1222\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1002/cpt.1222\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eSchramm, C. A. \\u003cem\\u003eet al.\\u003c/em\\u003e Knowledge-based potential for positioning membrane-associated structures and assessing residue-specific energetic contributions. \\u003cem\\u003eStructure\\u003c/em\\u003e 20, 924\\u0026ndash;935 (2012). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.str.2012.03.016\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.str.2012.03.016\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eSheppard, A. M., Chen, G. D. \\u0026amp; Salvi, R. Potassium ion channel openers, Maxipost and Retigabine, protect against peripheral salicylate ototoxicity in rats. \\u003cem\\u003eHear Res\\u003c/em\\u003e 327, 1\\u0026ndash;8 (2015). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.heares.2015.04.007\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.heares.2015.04.007\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eJoe, J. Levosulpiride-Induced Neurological Adverse Effects: A Prospective Study from a Tertiary Care Center. \\u003cem\\u003eAnn Indian Acad Neurol\\u003c/em\\u003e 23, 174\\u0026ndash;176 (2020). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.4103/aian.AIAN_449_18\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.4103/aian.AIAN_449_18\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eEscovitz, W. E. Disturbance in carbohydrate metabolism associated with amithiozone therapy. \\u003cem\\u003eAm Rev Tuberc\\u003c/em\\u003e 66, 373\\u0026ndash;377 (1952).\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003ePeloquin, C. A. Controversies in the management of Mycobacterium avium complex infection in AIDS patients. \\u003cem\\u003eAnn Pharmacother\\u003c/em\\u003e 27, 928\\u0026ndash;937 (1993). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1177/106002809302700722\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1177/106002809302700722\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKuna, P., Bjermer, L. \\u0026amp; Tornling, G. Two Phase II randomized trials on the CRTh2 antagonist AZD1981 in adults with asthma. \\u003cem\\u003eDrug Des Devel Ther\\u003c/em\\u003e 10, 2759\\u0026ndash;2770 (2016). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.2147/DDDT.S105142\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.2147/DDDT.S105142\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eAlaseem, A. \\u003cem\\u003eet al.\\u003c/em\\u003e Matrix Metalloproteinases: A challenging paradigm of cancer management. \\u003cem\\u003eSemin Cancer Biol\\u003c/em\\u003e 56, 100\\u0026ndash;115 (2019). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/j.semcancer.2017.11.008\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/j.semcancer.2017.11.008\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003ePahor, M. \\u003cem\\u003eet al.\\u003c/em\\u003e Drug data coding and analysis in epidemiologic studies. \\u003cem\\u003eEur J Epidemiol\\u003c/em\\u003e 10, 405\\u0026ndash;411 (1994). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1007/BF01719664\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1007/BF01719664\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eBland, J. M. \\u0026amp; Altman, D. G. Multiple significance tests: the Bonferroni method. \\u003cem\\u003eBMJ\\u003c/em\\u003e 310, 170 (1995). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1136/bmj.310.6973.170\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1136/bmj.310.6973.170\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eLe, T. T., Fu, W. \\u0026amp; Moore, J. H. Scaling tree-based automated machine learning to biomedical big data with a feature set selector. \\u003cem\\u003eBioinformatics\\u003c/em\\u003e 36, 250\\u0026ndash;256 (2020). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/bioinformatics/btz470\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/bioinformatics/btz470\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eMorgan, H. L. The Generation of a Unique Machine Description for Chemical Structures-A Technique Developed at Chemical Abstracts Service. \\u003cem\\u003eJournal of Chemical Documentation\\u003c/em\\u003e 5, 107\\u0026ndash;113 (1965). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1021/c160017a018\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1021/c160017a018\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eCongreve, M., Carr, R., Murray, C. \\u0026amp; Jhoti, H. A 'rule of three' for fragment-based lead discovery? \\u003cem\\u003eDrug Discov Today\\u003c/em\\u003e 8, 876\\u0026ndash;877 (2003). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1016/s1359-6446(03)02831-9\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1016/s1359-6446(03)02831-9\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eMeszaros, B., Erdos, G. \\u0026amp; Dosztanyi, Z. IUPred2A: context-dependent prediction of protein disorder as a function of redox state and protein binding. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 46, W329-W337 (2018). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gky384\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gky384\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eMeszaros, B., Simon, I. \\u0026amp; Dosztanyi, Z. Prediction of protein binding regions in disordered proteins. \\u003cem\\u003ePLoS Comput Biol\\u003c/em\\u003e 5, e1000376 (2009). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1371/journal.pcbi.1000376\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1371/journal.pcbi.1000376\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKrogh, A., Larsson, B., von Heijne, G. \\u0026amp; Sonnhammer, E. L. Predicting transmembrane protein topology with a hidden Markov model: application to complete genomes. \\u003cem\\u003eJ Mol Biol\\u003c/em\\u003e 305, 567\\u0026ndash;580 (2001). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1006/jmbi.2000.4315\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1006/jmbi.2000.4315\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eBernhofer, M. \\u003cem\\u003eet al.\\u003c/em\\u003e PredictProtein - Predicting Protein Structure and Function for 29 Years. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 49, W535-W540 (2021). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gkab354\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gkab354\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eFox, N. K., Brenner, S. E. \\u0026amp; Chandonia, J. M. SCOPe: Structural Classification of Proteins\\u0026ndash;extended, integrating SCOP and ASTRAL data and classification of new structures. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 42, D304-309 (2014). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gkt1240\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gkt1240\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eJumper, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Highly accurate protein structure prediction with AlphaFold. \\u003cem\\u003eNature\\u003c/em\\u003e 596, 583\\u0026ndash;589 (2021). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1038/s41586-021-03819-2\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1038/s41586-021-03819-2\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKabsch, W. \\u0026amp; Sander, C. Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features. \\u003cem\\u003eBiopolymers\\u003c/em\\u003e 22, 2577\\u0026ndash;2637 (1983). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1002/bip.360221211\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1002/bip.360221211\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eCock, P. J. \\u003cem\\u003eet al.\\u003c/em\\u003e Biopython: freely available Python tools for computational molecular biology and bioinformatics. \\u003cem\\u003eBioinformatics\\u003c/em\\u003e 25, 1422\\u0026ndash;1423 (2009). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/bioinformatics/btp163\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/bioinformatics/btp163\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003cli\\u003e\\u003cspan\\u003eKuriata, A. \\u003cem\\u003eet al.\\u003c/em\\u003e Aggrescan3D (A3D) 2.0: prediction and engineering of protein solubility. \\u003cem\\u003eNucleic Acids Res\\u003c/em\\u003e 47, W300-W307 (2019). \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://doi.org:10.1093/nar/gkz321\\u003c/span\\u003e\\u003cspan address=\\\"https://doi.org:10.1093/nar/gkz321\\\" targettype=\\\"DOI\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/span\\u003e\\u003c/li\\u003e\\u003c/ol\\u003e\"}],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":true,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":false,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":false,\"isAuthorSuppliedPdf\":false,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":false,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"npj-drug-discovery\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"\",\"sideBox\":\"Learn more about [npj Drug Discovery](https://www.nature.com/npjdrugdiscov/)\",\"snPcode\":\"44386\",\"submissionUrl\":\"https://submission.springernature.com/new-submission/44386/3\",\"title\":\"npj Drug Discovery\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"NPJ\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true},\"keywords\":\"\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-6999821/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-6999821/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"\\u003cp\\u003eDespite rigorous safety evaluations during development, numerous drugs have been withdrawn from the market due to serious toxicities. Here we investigate the features found in drugs with these unanticipated toxicities and apply a machine learning approach to predict if a drug is likely to be withdrawn due to intolerable side effects without the need for human trial data. Our best preforming classifier was an ensemble predictor trained on protein targets, protein structure features, chemical fingerprints, and chemical features that achieved 92% accuracy and 0.845 Matthews Correlation Coefficient with 10-fold holdout test set cross validation. Analysis of features predictive of unanticipated toxicity revealed both known factors such as inhibition of cytochrome P450 as well as yet uninvestigated factors including the inhibition of bile salt export pumps. This predictor and subsequent feature analysis pave the way for the larger role of computational methods in screening potential candidates during drug development.\\u003c/p\\u003e\",\"manuscriptTitle\":\"Capturing Unanticipated Drug Toxicities Using an Ensemble Machine Learning Approach\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-07-10 11:51:20\",\"doi\":\"10.21203/rs.3.rs-6999821/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0},{\"type\":\"decision\",\"content\":\"Revision requested\",\"date\":\"2025-08-04T23:19:06+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-08-03T12:48:01+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-07-15T19:00:33+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"184672081892761752064323370463905869971\",\"date\":\"2025-07-11T05:20:29+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"115940246718519058327966345966039094583\",\"date\":\"2025-07-08T14:16:57+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewersInvited\",\"content\":\"\",\"date\":\"2025-07-08T13:55:12+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorAssigned\",\"content\":\"\",\"date\":\"2025-07-05T13:41:24+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"checksComplete\",\"content\":\"\",\"date\":\"2025-07-04T09:52:03+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"submitted\",\"content\":\"npj Drug Discovery\",\"date\":\"2025-06-28T21:36:57+00:00\",\"index\":\"\",\"fulltext\":\"\"}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"npj-drug-discovery\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"\",\"sideBox\":\"Learn more about [npj Drug Discovery](https://www.nature.com/npjdrugdiscov/)\",\"snPcode\":\"44386\",\"submissionUrl\":\"https://submission.springernature.com/new-submission/44386/3\",\"title\":\"npj Drug Discovery\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"NPJ\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"4d77757c-cc78-4aad-b590-2ee3889120cc\",\"owner\":[],\"postedDate\":\"July 10th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"in-revision\",\"subjectAreas\":[{\"id\":51241207,\"name\":\"Physical sciences/Chemistry\"},{\"id\":51241208,\"name\":\"Biological sciences/Computational biology and bioinformatics\"},{\"id\":51241209,\"name\":\"Biological sciences/Drug discovery\"}],\"tags\":[],\"updatedAt\":\"2025-08-04T23:23:20+00:00\",\"versionOfRecord\":[],\"versionCreatedAt\":\"2025-07-10 11:51:20\",\"video\":\"\",\"vorDoi\":\"\",\"vorDoiUrl\":\"\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-6999821\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-6999821\",\"identity\":\"rs-6999821\",\"version\":[\"v1\"]},\"buildId\":\"XKTyCvWXoU3ODBz1xrDgd\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}