Synergistic drug combinations prediction by integrating pharmacological data.

Let D = { d 1 , d 2 , ⋯ , d n } denotes the drug set, T = { t 1 , t 2 , ⋯ , t r } be the target set, E = { e 1 , e 2 , ⋯ , e q } be the enzyme set and A = { a 1 , a 2 , ⋯ , a p } be the ATC set. For each drug pair ( d i , d j ) , y i j is defined to indicate their synergistic relationship, y i j = 1 if ( d i , d j ) is an approved synergistic combination, and y i j = − 1 if not. For each drug pair ( d i , d j ) , we constructed its feature vector F i j = ( F i j T , F i j E , F i j A ) as follows, 1. Target pairs of ( d i , d j ) , F i j T : (i) Denote the targets of d i as T d i = { t i } and the target pairs of ( d i , d j ) as { ( t s , t k ) } , t s ∈ T d i , t k ∈ T d j . (ii) Collect the target pairs of all synergistic drug combinations, written T s y n = { ( t p , t q ) } . In other words, each pair ( t p , t q ) in T s y n is a target pair of at least one synergistic drug combination and the target pairs in T s y n were sorted order by target ID. (iii) Construct a 0–1 row vector F i j T and | F i j T | = | T s y n | , where the k -th components of F i j T is 1 if the k -th target pair is one target pair of ( d i , d j ) . Otherwise, the k -th components of F i j T is 0. 2. Enzyme pairs of ( d i , d j ) , F i j E : The enzyme pairs of all synergistic drug combinations could be obtained according to drug-enzyme associations, recorded as E s y n . Sort the enzyme pairs in E s y n by enzyme ID and construct a 0–1 row vector F i j E . The components of F i j E is set in a similar way as described above for the construction of F i j T . 3. ATC pairs of ( d i , d j ) , F i j A : The ATC pairs of all synergistic drug combinations could be obtained according to drug-ATC associations, recorded as A s y n . Sort the ATC pairs in A s y n by ATC ID and construct a 0–1 row vector F i j A . The components of F i j A is set in a similar way as described above for the construction of F i j T . 4. Feature vector of ( d i , d j ) , F i j : Concatenate three features together, and the feature vector of ( d i , d j ) was denoted as F i j = ( f i j k ) k = ( F i j T , F i j E , F i j A ) . Here, C k ∈ { T ( a r g e t ) , E ( n z y m e ) , A ( T C ) } is referred to as a filed of feature f i j k and each components f i j k is referred to as a feature. It is worth noting that the features constructed by our method are very sparse due to incomplete pharmacological data and the fact that most drugs do not have the same targets, enzymes or ATC. Target pairs of ( d i , d j ) , F i j T : (i) Denote the targets of d i as T d i = { t i } and the target pairs of ( d i , d j ) as { ( t s , t k ) } , t s ∈ T d i , t k ∈ T d j . (ii) Collect the target pairs of all synergistic drug combinations, written T s y n = { ( t p , t q ) } . In other words, each pair ( t p , t q ) in T s y n is a target pair of at least one synergistic drug combination and the target pairs in T s y n were sorted order by target ID. (iii) Construct a 0–1 row vector F i j T and | F i j T | = | T s y n | , where the k -th components of F i j T is 1 if the k -th target pair is one target pair of ( d i , d j ) . Otherwise, the k -th components of F i j T is 0. Enzyme pairs of ( d i , d j ) , F i j E : The enzyme pairs of all synergistic drug combinations could be obtained according to drug-enzyme associations, recorded as E s y n . Sort the enzyme pairs in E s y n by enzyme ID and construct a 0–1 row vector F i j E . The components of F i j E is set in a similar way as described above for the construction of F i j T . ATC pairs of ( d i , d j ) , F i j A : The ATC pairs of all synergistic drug combinations could be obtained according to drug-ATC associations, recorded as A s y n . Sort the ATC pairs in A s y n by ATC ID and construct a 0–1 row vector F i j A . The components of F i j A is set in a similar way as described above for the construction of F i j T . Feature vector of ( d i , d j ) , F i j : Concatenate three features together, and the feature vector of ( d i , d j ) was denoted as F i j = ( f i j k ) k = ( F i j T , F i j E , F i j A ) . Here, C k ∈ { T ( a r g e t ) , E ( n z y m e ) , A ( T C ) } is referred to as a filed of feature f i j k and each components f i j k is referred to as a feature. It is worth noting that the features constructed by our method are very sparse due to incomplete pharmacological data and the fact that most drugs do not have the same targets, enzymes or ATC. Degree-2-polynomial mappings (Poly2) has been recorded as an effective approach to learn information implicit in feature conjunctions [ 7 , 34 ]. Based on this, we can simplify synergistic mechanisms as combinations of pharmacological characteristics, and for each drug pair ( d i , d j ) , the combinations of features can be represented by a linear function (1) ϕ ( w , F i j ) = ∑ k ∑ l w k , l f i j k f i j l + ∑ k w k f i j k w k , l and w k denoted the contribution values of feature interaction ( f i j k , f i j l ) and a single feature f i j k to the synergistic relationship between d i and d j , respectively. ϕ ( w , F i j ) is modeled to be an indicator to measure the strength of synergy between d i and d j , so ϕ ( w , F i j ) should be learned approximating to the real label y i j . Unfortunately, the above function could not reflect the real interactions between different features accurately. Take feature interactions ( f i j k , f i j l 1 ) and ( f i j k , f i j l 2 ) for example, f i j l 1 may affect the interaction between f i j k and f i j l 2 . However, w k , l 1 is independent from w k , l 2 in the above function and they could not capture this mutually dependent relationship. So, we improved ϕ by implementing FFM, which replaces all interactions by mapping similar features to be embedded near each other into several latent spaces according to fields it belongs to. Thus, dot product between latent vectors would capture interactions related to similar features and interactions not recorded in the available dataset are also allowed to caught. Although higher order of interactions could be estimated, we only considered first and second order due to high computational complexity. In our dataset, each feature can be represented by three latent vectors v T , v E , v A and the contribution value of feature interaction ( f i j k , f i j l ) can be represented by product of their corresponding latent vectors, writren by 〈 v k , C l , v l , C k 〉 . The latent vectors would help to extract the potential interactions between two drugs and ϕ ( w , F i j ) can be rewritten as (2) ϕ ( v , w , F i j ) = ∑ k ∑ l 〈 v k , C l , v l , C k 〉 f i j k f i j l + ∑ k w k f i j k For example, the feature vector of a drug combination is f = ( f 1 , f 2 , f 3 ) , and f 1 ∈ T ( a r g e t ) , f 2 ∈ E ( n z y m e ) , f 3 ∈ A ( T C ) , then the interaction of features can be represented as follows, (3) 〈 v f 1 , E , v f 2 , T 〉 f 1 f 2 + 〈 v f 1 , A , v f 3 , T 〉 f 1 f 3 + 〈 v f 2 , A , v f 3 , E 〉 f 2 f 3 + w 1 f 1 + w 2 f 2 + w 3 f 3 Indicator ϕ ( v , w , F i j ) should be learned to approximate the real synergistic relationship y i j . Logistic loss function is chosen to learn feature interactions, which is a widely used loss function to penalize incorrect predictions in classification problem. In addiction, l 2 -regularization terms are introduced to prevent overfitting of SyFFM and generalize SyFFM to other drugs not included in the available dataset. Thus, the problem of synergistic drug combinations prediction would be converted to learn v and w by solving the following optimization problem, (4) min v , w ∑ i , j log ( 1 + exp ( − y i j ϕ ( v , w , F i j ) ) ) + λ 2 | | v | | 2 2 + λ 2 | | w | | 2 2 We implemented stochastic gradient methods to derive the optimal solution of this optimization problem. Let ( v * , w * ) be the optimum solution, and the synergistic score between d i and d j was defined as ϕ ( v * , w * , F i j ) . Given a threshold θ , ( d i , d j ) was predicted as a synergistic combination if its synergistic score was larger than θ , and θ was selected by cross validations. The code of field-aware factorization machines can be got at https://www.csie.ntu.edu.tw/∼cjlin/libffm/ contributed by the Machine Learning Group at National Taiwan University.

SyFFM was implemented on three datasets respectively, DCDB data (including 946 approved pairwise synergistic drug combinations, 759 drugs from DCDB [ 40 ]), NData (68 approved pairwise synergistic combinations and 92 drugs from paper [ 28 ]) and FDA data (184 approved pairwise synergistic combinations and 238 drugs from FDA until November 2010). In DCDB, there are 287661 possible pairs of 759 drugs and 3‰  were approved to be synergistic combinations. Similarly, there are 1% pairs in NData and 6‰  pairs in FDA data were approved to be synergistic combinations. In each dataset, we regarded these approved synergistic combinations as ’positive samples', and the ’negative samples’ were obtained by sampling from the random pairs of all drugs. We set different ratios of positive-to-negative samples (1:1, 1:2, 1:3, 1:4, 1:5) and analysis the impact of ratios on performance of SyFFM. Firstly, 5-fold cross validation was implemented to evaluate our method. To guarantee the nonrandom of the prediction results, we implemented our method on 50 different negative samples and the optimal parameters were adjusted by cross validation. We got average AUC (DCDB) = 0.925 (0.87 ∼ 0.93), AUC (NData) = 0.876 (0.84 ∼ 0.92), AUC (FDA) = 0.896(0.86 ∼ 0.92) ( Fig. 1 ). Secondly, the synergistic scores obtained by our method were sorted in descending order, for DCDB data, SyFFM correctly retrieved all associations in the top-400 records, 697 in the top-700 records and 791 in the top-800 records. For NData, SyFFM correctly retrieved 40, 59, 67 in the top-40, 60, 80 records. For FDA data, SyFFM correctly retrieved 48 93, 135 in the top-50, 100, 150 records. ( Fig. 1 ). Fig. 1 The results of SyFFM on pairwise synergistic drug combinations. Fig. 1 The results of SyFFM on pairwise synergistic drug combinations. Synergistic combinations in FDA were updated after November 2010 and the predictive ability of SyFFM for potential synergistic drug combinations is evaluated by implemented on FDA data (until November 2010). 11 potential synergistic combinations are predicted by SyFFM and 9 are confirmed by literature, which is shown in Table 1 . Table 1 The 11 potential synergistic drug combinations predicted by SyFFM and 9 are confirmed by literature. Table 1 Drug 1 Drug 2 Evidence of synergy combinations ampicillin sodium tazobactam PMID: 24041466 esomeprazole magnesium sodium bicarbonate None trandolapril hydrochlorothiazide PMID: 7527107 dienogest ethinyl estradiol PMID: 20394455 eprosartan mesylate amlodipine besylate None tenofovir disoproxil fumarate lamivudine PMID: 13130407 niacin atorvastatin calcium PMID: 10095800 trandolapril amlodipine besylate None norelgestromin Ethinyl Estradiol PMID: 11849630 Ethinyl estradiol ethynodiol diacetate PMID: 2046084 desogestrel Ethinyl estradiol PMID: 9673846 The 11 potential synergistic drug combinations predicted by SyFFM and 9 are confirmed by literature. To test the impact of different proportions of positive and negative samples on the performance of SyFFM, we randomly selected different numbers of negative samples from all non-approved drug pairs. AUC, AUPR and F1 score are used and the result is shown in Table 2 , which means that the performance of SyFFM is getting worse as the number of negative samples increased. According to the above analysis, we set the ratio of positive-to-negative to be 1:1. Table 2 The performance of SyFFM of different ratios of positive-to-negative samples. Table 2 Ratio of positive-to-negative AUC AUPR F1 score 1:1 0.925 0.934 0.7611 1:2 0.8773 0.8287 0.7405 1:3 0.8318 0.7629 0.7162 1:4 0.7435 0.6959 0.6963 1:5 0.7377 0.6976 0.6759 The performance of SyFFM of different ratios of positive-to-negative samples. We also evaluated the performance of SyFFM on 295 approved three-drug synergistic combinations collected from DCDB, including 368 drugs. Each three-drug combination was regarded as three pairwise drug combination, for example, three-drug combination ( d i , d j , d k ) was regarded as ( d i , d j ) , ( d i , d k ) and ( d j , d k ) . If ( d i , d j , d k ) is a synergistic combination, then three pairwise combinations are treated as synergistic combinations (we record this hypothesis as H ). In this way, we converted three-drug synergistic combinations prediction to pairwise synergistic drug combinations prediction, and the three-drug combination is predicted as synergistic if and only if all the corresponding three pairwise drug combinations are predicted as synergistic combinations. By decomposing entire three-drug combinations of 368 drugs into pairwise combinations, we implemented SyFFM and the pairwise combinations were predicted. Then three-drug combinations could be predicted by screening the corresponding pairwise combinations. We got AUC of 0.946 in terms of 5-fold cross validation on the entire pairwise combinations ( Fig. 2 (blue curve)) and AUC of 0.944 on the entire three-drug combinations ( Fig. 2 (yellow curve)). Fig. 2 The results of SyFFM on three-drug synergistic combinations. Fig. 2 The results of SyFFM on three-drug synergistic combinations. In order to verify the influences of features defined in our method, we used target pairs F i j T , enzyme pairs F i j E or ATC pairs F i j A to be the feature vector of drug pair ( d i , d j ) . Then, field-aware factorization machines were implemented on pairwise drug combinations using the new feature vectors respectively. The average AUCs achieved using new feature vectors were 0.706, 0.679 and 0.855, respectively, which was shown in Fig. 3 . The performance of our method was not satisfying in terms of AUCs under 5-fold cross validation, which indicates that interactions between features from different fileds play important roles in synergistic mechanisms. Fig. 3 The results of SyFFM using three new feature vectors. Fig. 3 The results of SyFFM using three new feature vectors. The coefficients in function ϕ indicate the contribution values of the corresponding features or feature interactions. Incorporating biological knowledge, we analyzed coefficients obtained on the prediction of pairwise drug combinations. We sorted coefficients of single features (i.e. the first order coefficients w ) in descending way and found that the features corresponding to top-10 coefficients are all target pairs. The result is shown in Table 3 . Table 3 Features and approved synergistic drug combinations corresponding to top-10 coefficients. Table 3 Target 1 Target 2 Approved synergistic combination Disease Progesterone receptor Estrogen receptor Estrogens, conjugated; Medroxyprogesterone acetate Symptoms of menopause  [ 18 ] Ethinyl Estradiol; Norethindrone Contraception  [ 1 ] Angiotensin-converting enzyme (ACE) Type-1 angiotensin II receptor (AR) Lisinopril; Losartan Kidney Diseases  [ 17 ] DDP-4 PPAR- γ Rosiglitazone; Saxagliptin Diabetes Mellitus, Type 2  [ 23 , 32 , 33 ] Alogliptin; Pioglitazone Type 2 Diabetes Mellitus  [ 14 , 48 ] Glucocorticoid receptor Beta-2 adrenergic receptor Fluticasone Propionate; Formoterol Asthma, Bronchial  [ 5 , 13 , 42 ] Budesonide; Formoterol Chronic Obstructive Pulmonary Disease; Asthma; Bronchiectasis  [ 46 , 59 ] 5′-AMPK DDP-4 Alogliptin benzoate; Metformin Hydrochloride Type 2 Diabetes Mellitus  [ 2 ] Metformin; Sitagliptin Type 2 diabetes who do not use daily insulin injections; Diabetes Mellitus, Type 2  [ 57 ] DNA Ribonucleoside-diphosphate reductase subunit M2 Fluorouracil; SC144 Colorectal cancer  [ 40 ] Renin AR Aliskiren; Irbesartan Hypertension  [ 40 ] Progesterone receptor GnRH Leuprorelin acetate; Acetynone acetonidone For initial management of the painful symptoms of endometriosis and for management of recurrence of symptoms  [ 40 ] Follicle-stimulating hormone receptor GnRH-R Buserelin; Pregnyl Ovarian Hyperstimulation  [ 25 ] Angiotensin-converting enzyme Voltage-dependent calcium channel gamma-1 subunit None approved in DCDB Features and approved synergistic drug combinations corresponding to top-10 coefficients. It is shown from the table that these target pairs are keys to the mechanism of synergy. Since a target is involved in different biological pathways [ 9 , 19 ], the multiple interactions between different targets may be applied in treatment of other diseases. For example, the activation of progesterone receptor can also inhibit the growth of breast tumors by changing the synthesis of estrogen receptor chromatin  [ 49 , 53 ]; both ACE and AR play an important roles in the regulation of plasma, and the American Diabetes Association also suggested using ACE inhibitors or AR inhibitors to treat diabetes  [ 22 , 47 ]; sitagliptin, inhibitors of DDP-4, can regulate the expression of PPAR and may have a protective effect on myocardial cells  [ 35 ]; both 5′-AMPK and DDP-4 are targets of polycystic ovary syndrome  [ 55 ]. These findings show that our model finds important features of synergism and may help to explain the synergistic mechanism between two drugs. Using DCDB data, we compared the prediction performance between SyFFM and Bai's method proposed in Ref. [ 4 ]. Bai's method is a synergistic drug combination prediction model by applying a novel improved naïve Bayesian algorithm on various types of features (targets, side effects, pathways, enzymes, transporters). The comparison result of leave-one-out cross validation is shown in Table 4 . Table 4 Comparison result of SyFFM and Bai's method using leave-one-out cross validation. Table 4 Method Precision Recall F1 score Accuracy SyFFM 0.9983 0.4573 0.9213 0.9255 Bai's method with targets 0.9008 0.5000 0.6431 0.7034 Bai's method with side effects 0.6957 0.6400 0.6667 0.6800 Bai's method with pathways 0.6133 0.6216 0.6174 0.6238 Bai's method with enzymes 0.6018 0.8095 0.6904 0.6115 Bai's method with transporters 0.4815 0.7500 0.5865 0.5339 Comparison result of SyFFM and Bai's method using leave-one-out cross validation.

By representing drug combinations as combinations of their pharmacological characteristics, we proposed a computational synergistic drug combination prediction approach, namely SyFFM. The features of drug combinations are constructed by integrating pharmacologic data such as drug targets and enzymes, and the synergistic scores between drugs are obtained by implementing field-aware factorization machines. SyFFM achieves good performances by predicting pairwise synergistic drug combinations and three-drug synergistic combinations, and plenty of prediction results are validated by various databases and literature. We demonstrated that SyFFM provides biological explanations for the synergistic principles underlying combinatorial therapy and offers a new method for studies on synergistic mechanisms. However, several potential limitations exist in SyFFM. Firstly, the performance of model is limited due to the incompletion data about synergistic drug combinations. Secondly, SyFFM relies on the feature patterns enriched in synergistic combinations and other feature construction approaches may be more preferable to be adopted. It is worth noting that the strict hypothesis H would introduce unreasonable bias for n -drug combinations with n increases and the computation complexity would present to be an exponential growth. So we only test SyFFM on pairwise and three-drug combinations, and the application of SyFFM to multi-drug synergistic combinations still has to be improved further. How to build a better synergistic drug combinations prediction model by utilizing diverse knowledge on drugs remains be an challenge for future research.

Synergistic drug combinations consist of two or more active pharmaceutical ingredients (APIs), which are developed to target multiple diseases or conditions, such as Exforge (amlodipine/valsartan). Accumulated studies have shown that synergistic drug combinations are widely used in the treatment of cancer, AIDs and other complex diseases [ 12 , 16 ]. Christie M. Ballantyne et al. found that the therapeutic effect of the combination of ezetimibe and simvastatin (Vytorin) was significant for treatment of hypercholesterolemic patients [ 3 ]. Zhong et al. have found that Homoharringtonine combining with arsenic trioxide could induce apoptosis of RPMI 8226 cell line [ 63 ]. Drug combinations could improve medication compliance by reducing the pill burden of patients. Moreover, drug combinatorial therapy has better efficacy and smaller side effect than single drugs because of the fact that the development and progression of systemic diseases often involve in complex biological processes. Investments in synergistic drug combinations have emerged and become an increasingly important strategy because of the high failure rate in new drug development pipeline and high costs in marketing new drugs [ 15 ]. It also has been argued that synergistic drug combinations played vital roles in drug development as a new paradigm of drug discovery [ 24 ]. Despite of more attentions have been paid to research on synergistic drug combinations, there are still a lack of effective prediction methods. Most synergistic combinations were found using exhaustive methods, such as high-throughput screening or pure experimental tests, which are time-consuming and costly. Therefore, a number of computational approaches for synergistic combinations prediction have been emerged with the growth of biological datasets [ 36 ]. For example, Zhao et al. proposed a computational method to predict novel combinations by integrating molecular and pharmacological data, but its performance is limited by the feature constructed in this model [ 62 ]. Ligeti et al. presented a prediction model by applying a propagation algorithm on a protein-protein association network [ 38 ], but the construction of network is affected by incompleteness of biological data. Sun et al. constructed support vector machines and naïve Bayesian classifiers on Hadoop for drug combination prediction, but it cannot explain synergistic mechanisms [ 51 ]. Chen et al. collected synergistic antifungal drug combinations and construct the first Antifungal Synergistic Drug Combination Databases (ASDCD) [ 10 ] and developed a semi-supervised algorithm implemented on drug-target interactions and drug chemical structures data [ 11 ], but its predictive performance is evaluated only on antifungal drugs. Liu constructed features of drug pairs using some feature selection techniques and predicted potential synergistic drug combinations by applying random forest algorithm [ 41 ]. Based on drug target network and drug induced gene expression profiles [ 26 ], Li built a prediction model named by SyDRa using random forest algorithm [ 37 ], which is a simple and effective computational model. However, it is a cancer-specific model and it was not tested on other diseases. Besides, most of these quantitative and qualitative prediction methods are proposed to pairwise synergistic combinations and cannot be applied to predict multi-drug synergistic combinations. In this paper, by considering synergistic mechanisms as combinations of their pharmacological characteristics [ 8 ], we proposed a new model, namely SyFFM, to infer potential pairwise synergistic combinations and three-drug synergistic combinations. Drug targets, metabolic enzymes and ATC (Anatomical Therapeutic Chemical Classification System) were collected for constructing features of drug combinations and FFM (Field-aware Factorization Machines) [ 30 ] were applied to calculate the synergistic scores. FFM models pairwise interactions via inner products of respective feature latent vectors, and it has good performance under huge sparsity. Using factorized parameters, FFM provides a new perspective to investigate interactions between drugs. SyFFM was evaluated on various datasets, and the good predictive performance suggests that it would help to identify novel drug combinations effectively. Analysis on the parameters of SyFFM demonstrates that the new model can provide insights to uncover the synergistic mechanisms.