Predicting clinical pregnancy using clinical features and machine learning algorithms in in vitro fertilization.

Infertility is a global health issue that affects individuals, families, and society. The prevalence of infertility has increased in recent years. In the United States, the prevalence of infertility is approximately 8.8% (National Center for Health Statistics, 2020). In the United Kingdom, the prevalence of infertility was 12.5% in women and 10.1% in men [ 1 ]. In 2010, in 190 countries and territories worldwide, women aged 20–44 faced the possibility of pregnancy, and 1.9% were unable to experience a live birth [ 2 ]. In Taiwan, the female fertility rate decreased from 7.04 children per woman in 1951 to 1.165 in 2014. As a result, Taiwan has one of the lowest fertility rates worldwide [ 3 ]. Therefore, infertility has become a major healthcare issue in Taiwan. Since the advent of assisted reproductive technologies (ARTs) in 1984, the use of in vitro fertilization (IVF) cycles has increased worldwide. Clinical pregnancy is defined as the pregnancy that lasted 6 weeks (or 42 days) after the onset of the last menstrual period and confirmed by human chorionic gonadotropin (hCG) assay [ 4 ]. Achieving high clinical pregnancy rates has been a major goal for both physicians and patients. Pregnancy rates after IVF treatment are approximately 30–70%, depending on the age of the female patient and the different types of interventions employed. Factors that contribute to successful pregnancies have been widely explored. Factors that affect the success of ARTs include age, body mass index (BMI), uterine and ovarian factors, the type of stimulation protocol used, the stimulation dose, and the use of fresh or frozen protocols. ART lies in the interaction of these variables to achieve higher numbers of oocytes and higher pregnancy rates. There are no fixed protocols for individual patient conditions. While reducing or removing adverse factors for pregnancy, prediction models based on existing large-scale databases have been established to boost our understanding of IVF procedures and improve pregnancy rates [ 5 ]. The development of prediction models can help physicians provide personalized treatment for each infertility couple. Artificial intelligence (AI) is a technology suitable for application in ART fields for several reasons. ARTs require highly individualized protocols. Parameters affecting the success of ARTs may be discovered during the process of AI calculations. Moreover, by predicting the best pregnancy rates, individualized, tailored protocols can be used for infertile couples. This study aimed to utilize machine-learning algorithms to predict clinical pregnancy rates. Machine learning is a computational method that focuses on how computers learn from data. It intersects with statistics, with the goal of discerning relationships from data and computer science, and emphasizes efficient computing algorithms [ 6 ]. Algorithms use AI and statistics to find patterns in a dataset. Data patterns are then used to make predictions [ 7 ]. Several previous studies proposed predicting pregnancy using IVF based on traditional statistical and machine-learning methods. Logistic regression is a popular statistical method that develops prediction models. To construct a prediction model based on logistic regression, Ottosen et al . used 1,675 records containing IVF and intracytoplasmic sperm injection (ICSI) treatment cycles. Their variables included embryo quality, patient age, duration of infertility (months), BMI, basal follicle-stimulating hormone (FSH), treatment type, indication for treatment, number of oocytes retrieved, fertilization rate, and scores of the best and second-best embryos. Embryo quality, patient age, and basal FSH levels showed statistically significant effects on pregnancy. A receiver operating characteristics (ROC) curve was incorporated to evaluate the predictive performance, and the area under the ROC curve (AUC) values were 0.64 and 0.68 for singleton and twin pregnancy models, respectively [ 8 ]. Hansen et al . used a logistic regression to analyze conception, clinical pregnancies, and live births in a dataset that included 19 medical and socioeconomic variables of 900 couples with unexplained infertility. The AUC values of the prediction models for conception, clinical pregnancies, and live births were 0.66, 0.64, and 0.65, respectively [ 9 ]. Meijerink et al . developed a prediction model based on multivariate logistic regression using a dataset containing 289 couples with 553 testicular sperm extraction (TESE)-ICSI cycles. The dataset included types of infertility, duration of infertility (months), female age, parity, average menstrual cycle length (days), uterine abnormalities, antral follicle count before stimulation, alcohol use and smoking status for men and women, BMI at baseline for men and women, male age, male luteinizing hormone (LH), male inhibin levels, male FSH, total testicular volume, and a suspected primary diagnosis of azoospermia before sperm retrieval. The AUCs of the logistic regression prediction model were 0.62 and 0.67 for the validation data [ 10 ]. Compared to logistic regressions, machine learning algorithms are more sensitive and more specific screening techniques, for which the assumptions and restrictions of traditional regressions are relaxed. Machine learning algorithms have become increasingly common for learning from data to develop more reliable predictions [ 11 ]. In reproductive science, machine learning algorithms have also been incorporated in several studies. Blank et al . used a dataset collected by the Department of Reproductive Medicine, Ghent University Hospital, Belgium, containing 1,052 patients who underwent single-embryo transfer (SET) using fresh day-5 blastocysts. The dataset contained 32 variables, including continuous variables (male and female age and anti-Müllerian hormone [AMH]), categorical variables (stimulation protocols), and discrete variables (number of oocytes). To predict implantation after blastocyst transfer in IVF, a random forest algorithm showed better predictive performance than a logistic regression in terms of the AUC (0.74 for the random forest and 0.66 for the logistic regression) [ 12 ]. Qiu et al . applied variables, including age, AMH, duration of infertility, BMI, previous live births, previous miscarriages, previous abortions, and type of infertility (classified into tubal, anovulatory, male factor, and unexplained factors) with 7,188 records of women who were undergoing their first IVF treatment. To compare predictive performances based on the AUC, machine learning algorithms, including support vector machines, random forest, and extreme gradient boosting (XGBoost), outperformed traditional logistic regression in personalized predictions of live births prior to the first IVF treatment. Moreover, XGBoost and random forest algorithms achieved higher AUCs of approximately 0.73 compared to the other algorithms [ 13 ]. In addition to models for predicting pregnancy in ARTs, machine learning algorithms have also been incorporated to evaluate fetal health status. Akbulut et al . proposed an e-Health application based on machine learning algorithms to predict fetal anomaly status by referring to maternal and clinical data. Several binary prediction models were trained with a clinical dataset consisting of 96 pregnant women, and the highest accuracy achieved was 89.5% in a test set with a random forest model. In addition, the proposed model was applied to a real-life test of 16 users and obtained an accuracy of 87.5% [ 14 ]. This study aimed to incorporate machine learning algorithms to analyze independent variables influencing clinical pregnancy outcomes using ART, which may also provide useful information for clinicians and infertile couples.

The workflow of this study is shown in Fig 1 . The original dataset included 24,730 records. We excluded patients who were simultaneously administered fresh and freeze-thawed embryos, those without implantation, and those with frozen embryos. In addition, we randomly sampled patients with the same features to avoid bias in elderly patients to improve prediction accuracy. In total, 7,362 (approximately 42.58%) missing values of the ovarian stimulation protocol were imputed based on the missForest package in the R environment. With the exception of the natural cycle and others, all other missing values were imputed. In addition, the dataset was divided into two parts, and machine learning algorithms were compared based on the predictive performance of test dataset in terms of AUC. Finally, the effect of each analytic variable on clinical pregnancy was depicted. In total, 17,288 women were analyzed in this study, and 37.88% of women (aged 36.07±3.85 years) achieved a clinical pregnancy. The chi-squared test was used to examine the independence between categorical variables. As shown in Table 2 , all variables, except for artificially assisted reproductive methods and sources of sperm and oocytes, showed statistical significance in clinical pregnancy. Student’s t -test was used to examine whether the means of continuous variables showed statistically significant differences ( Table 3 ). * p < 0.05, ** p < 0.01, *** p < 0.001 by a chi-squared test for categorical variables. IVF/ET, in vitro fertilization/embryo transfer; ZIFT/TET, zygote intrafallopian transfer/tubal embryo transfer; ICSI, intracytoplasmic sperm injection; PGS, preimplantation genetic screening; PGD, preimplantation genetic diagnosis. Abbreviations: M, mean; SD, standard deviation; IVF, in vitro fertilization *** p < 0.001 by Student’s t-test for continuous variables. The original dataset was randomly separated into training and test datasets. According to the results generated based on the random forest model, the best “mtry” equals 3 with the minimum OOB error and maximum AUC (shown as S1, S2 Figs in S1 File ). In the training set, the AUC was used as the main evaluation measure to assess the clinical pregnancy performance of the prediction model. Four measures were used to assess the training dataset (accuracy: 62.20% vs. 83.39%; sensitivity: 62.05% vs. 83.02%; specificity: 62.36% vs. 83.76%; AUC: 0.9197 vs. 0.6783) and the test dataset (accuracy: 62.78% vs. 64.78%; sensitivity: 61.81% vs. 66.58%; specificity: 63.12% vs. 64.16%; and AUC: 0.6766 vs. 0.7208) in the random forest vs. logistic regression prediction models. Table 4 shows that the random forest model outperformed the logistic regression model in terms of accuracy, sensitivity, specificity, and AUC. In Fig 2 , the ROC curve of the random forest is closer to the upper left corner of the diagram. Therefore, the random forest model was chosen as the best model for predicting clinical pregnancy outcomes in our study. AUC, area under the receiver operating characteristics curve. To further validate the prediction model, we collected 3,352 new samples from 2019 to 2020 as the external dataset to evaluate the performance of our prediction model. The training and test data sets were combined together to construct a prediction model, and the external data set was used as an independent test set to evaluate the true predictive performance. As shown in Table 5 , the predictive performance of the external dataset is similar to the original test dataset (accuracy: 64.78% vs. 62.98%; sensitivity: 66.58% vs. 68.55%; specificity: 64.16% vs. 60.97%; and AUC: 0.7208 vs. 0.7123). This demonstrated that the proposed prediction model did not show overfitting of predictive performance, and thus it can be generalized to predict the clinical pregnancy rate of other patients. In the random forest model, the function varImpPlot() can generate a partial dependency plot of the importance of variables by measuring the mean decrease in accuracy (MDA). The results showed that the ovarian stimulation protocol, the total number of frozen embryos, and female age were the most important variables in the analysis. We discuss how variables affect the probability of clinical pregnancy. Important variables affecting clinical pregnancy outcomes were further analyzed to determine the relationships between predictors and clinical pregnancy. These results can help us understand the relationship between the marginal effects of predictors and clinical pregnancy ( Fig 3 ). The correlations between analytic variables and clinical pregnancy depicted by partial dependency plots were shown in Fig 4 . The total number of frozen and transferred embryos was positively correlated with clinical pregnancy outcomes, whereas the female age and duration of infertility were negatively correlated. The total number of frozen embryos showed a non-linear relationship with clinical pregnancy outcomes. As shown in Fig 4a , the marginal effect positively increased in eight frozen embryos and began to gradually decrease. The female age showed an obvious negative effect. Women < 36 years old exhibited constant conditions. After 36 years of age, there was a negative propensity for clinical pregnancy. After 40 years of age, the probability of a clinical pregnancy dramatically declined ( Fig 4b ). As shown in Fig 4c , the propensity of clinical pregnancy was positively correlated with the number of embryos transferred in the same IVF cycle, and the transfer of three and four embryos had little effect on clinical pregnancy compared to one or two, which showed a greater propensity of becoming pregnant. The duration of infertility was negatively correlated with clinical pregnancy. After 1 year of infertility, the propensity decreased dramatically ( Fig 4d ). Compared to other causes of infertility, other female factors and either of the two factors were the main factors that decreased the probability of a clinical pregnancy ( Fig 4e ). The use of frozen embryos was positively correlated with clinical pregnancy, while the transfer of fresh embryo was negatively correlated ( Fig 4f ). Long and ultra-long protocols were two effective treatments compared to the other protocols ( Fig 4g ). Continuous variables are presented as line plots, and categorical are presented as bar plots.

Our analysis showed that RF outperformed logistic regression for predicting clinical pregnancy outcomes. In our results, the AUC values of the test dataset with the logistic regression and random forest models were 0.6766 and 0.7208, respectively. The performance of logistic regression was similar to those of previous studies, and the random forest model outperformed them all.

Taipei Medical University Hospital (TMUH) is one of the leading hospitals in reproductive medicine in Taiwan, with more than 2,000 ART cycles conducted annually. Every IVF cycle was registered in a national health bureaucracy database. More than 100,000 IVF cycles were registered in the database. This study was approved by the Institutional Review Board (IRB) of TMUH (TMU-Joint Institutional Review Board N201908012). The IRB waived the requirement to obtain informed consent. In total, 24,730 patients underwent IVF/ICSI cycles with clinical pregnancy outcomes in the original data from the Health Promotion Administration, Ministry of Health, and Welfare. We used deidentified personal data and performed the study in accordance with the Declaration of Helsinki. As shown in Table 1 , the clinical data of women aged 21–55 years who underwent IVF/ICSI cycles at TMUH were analyzed. Patients who underwent embryo transfer with both fresh and freeze-thawed embryos simultaneously in the same cycle of IVF treatment were excluded because the embryo origin could not be defined when assessing pregnancy outcomes. Moreover, we collect 3,352 new samples from 2020–2021 as an independent test set for external validation of our proposed model. In total, 7,362 records of missing values in ovarian stimulation protocols were imputed by the missForest function in the random forest algorithm. This random forest-based function can impute missing values and perform better compared to other methods of imputation, especially in datasets that consist of different types of variables and complex interactions or non-linear relationships [ 15 ]. The imputation of missing values plays an important role in the data analysis. Moreover, we used the functional strata of package sampling for random sampling to divide the original dataset into 50% as the training dataset for model development and 50% as the test dataset for model validation. Statistical analyses were conducted using the RStudio (2009–2018 RStudio) software version 1.1.463 and SAS version 9.4 (SAS Institute, Cary, NC, USA). Means and standard deviations (SDs) were calculated for continuous variables, and frequencies and percentages were computed for categorical variables. Baseline characteristics between participants with and without a clinical pregnancy were compared using a chi-squared test for categorical variables and Student’s t -test for continuous variables. To predict pregnancy outcomes after IVF, we compared the performances of prediction models built using logistic regression or random forest algorithms. Previous studies used logistic regressions to build prediction models and observed their results using odds ratios (ORs). We incorporated the random forest algorithm to develop predictive models because advantages of the random forest include that it converges due to the law of large numbers and shows no overfitting without pruning in its predictions [ 16 ]. This study aimed to accurately predict clinical pregnancies and to depict the partial effect of each variable on the outcome. The random forest method is a combination of tree predictors, in which all trees are independently built by random vector sampling and have the same distribution in the forest. The generation error converges to a limit as trees in forests become larger. The classifier error in a forest depends on the strength and correlation between the trees [ 16 ]. Breiman proposed the random forest model in 2001, which can be created using the randomForest package, and is also available in the R environment [ 17 ]. The random forest model was constructed using the randomForest package [ 17 ]. We built the model using 1,000 trees, and three variables were sampled randomly in each tree. The importance of the variables in the model was measured by the mean decrease accuracy (MDA), which was calculated by how much accuracy was reduced when each variable was left out. To depict the influence of different variables, the random forest model can generate the importance of variables by observing the out-of-bag (OOB) error of a specific variable, as the other predictors remain stable [ 17 ]. The importance of a variable depicts the rank of that variable in influencing pregnancy predictions when building a prediction model. After comparing the importance of variables, we analyzed how the variables affected the probability of a clinical pregnancy. The use of partial dependence plots is a good way to provide insights into each machine learning model, as they depict how each variable influences the prediction when all other variables are simultaneously averaged [ 18 ]. partialPlot is a function in the randomForest package, which depicts the marginal effects of variables on the class probability for classification. Plots show the relative logit distributions of class probabilities from the model. Positive values on the y -axis indicate that the values of the independent variables are more likely to be a positive class. In contrast, negative values are less likely to be a positive class. Zero indicates the absence of an average influence on the class probability. One of the most important variables in the random forest algorithm is the mtry parameter, which is defined as the number of predictors sampled for splitting at each node. In our experiments, two approaches were incorporated for hyperparameter tuning of the mtry variable. The function “tuneRF” of the package “randomForest” and function “train” of package “caret” are two methods in tuning hyperparameter “mtry” for random forest algorithm. We used “tuneRF” to find the “mtry” with the minimum OOB error for 10 times and applied “train” to find the highest AUC with ten-fold cross-validations. Four measures, including accuracy, sensitivity, specificity, and AUC, were used to evaluate the predictive model. Accuracy, sensitivity, and specificity were generated from a function confusion matrix, and the AUC was generated from the ROCR package in the R environment. The accuracy, sensitivity, and specificity of the clinical pregnancy prediction model were calculated. Sensitivity and specificity points were plotted in the ROC curve, where the x -axis denotes “1-specificity,” whereas the y -axis represents “sensitivity.” An ROC curve with good predictive performance showed an AUC close to 1.

(DOCX) Click here for additional data file.