Advanced KPI Framework for IVF Pregnancy Prediction Models in IVF protocols

Advanced KPI Framework for IVF Pregnancy Prediction Models in IVF protocols | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Advanced KPI Framework for IVF Pregnancy Prediction Models in IVF protocols Sergei Sergeev, Iuliia Diakova This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4445375/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 27 Nov, 2024 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract The utilization of neural networks in assisted reproductive technology is essential due to their capability to process complex and multidimensional data inherent in IVF procedures, offering opportunities for clinical outcome prediction, personalized treatment implementation, and overall advancement in fertility treatment. The aim of this study was to develop a novel approach to IVF laboratory data analysis, employing deep neural networks to predict the likelihood of clinical pregnancy occurrence within an individual protocol, integrating both key performance indicators and clinical data. We conducted a retrospective analysis spanning 11 years, encompassing 8732 protocols, to extract the most relevant features to our goal and train the model. Internal validation was performed on 1600 preimplantation genetic testing for aneuploidy embryo transfers, while external was conducted across two independent clinics (over 10,000 cases). Leveraging recurrent neural networks, our model demonstrates high accuracy in predicting the likelihood of clinical pregnancy within specific IVF protocols (AUC: 0.68–0.86; Test accuracy: 0.78, F1 Score: 0.71, Sensitivity: 0.62; Specificity: 0.86) comparable to time-lapse system but with a simpler approach. Our model facilitates both retrospective analysis of outcomes and prospective evaluation of clinical pregnancy chances, thus presenting a promising avenue for quality management programs and promotes their realization in medical centers. Biological sciences/Computational biology and bioinformatics Health sciences/Health care Health sciences/Medical research Deep neural networks IVF laboratory data analysis Clinical pregnancy prediction Personalized treatment ART quality management Introduction In recent decades, neural networks and machine learning models (ML) have become key tools in various fields, including assisted reproductive technology (ART). They serve as a foundation for creating algorithms capable of extracting complex dependencies from data, and making decisions based on these dependencies. In some cases, neural networks can identify a broader spectrum of associations than other statistical methods, thanks to their ability to recognize highly nonlinear associations among input parameters [ 1 ]. Most ML models applied in the field of in vitro fertilization (IVF) are based on regression and logistic regression algorithms to identify relationships between the target variable (outcome metric of clinic success) and input parameters [ 2 ]. However, most models used for that evaluation are based on patient data and their previous IVF protocols, often failing to track the patterns in changes of quality laboratory indicators relevant to the final transfer outcome [ 3 ]. The importance of the laboratory stage in IVF is explained in detail in consensus resolutions that outline quality control (QC) measures and the evaluation of key performance indicators (KPIs), in accordance with international standards.[ 4 ]. Nevertheless, it is hard to disagree that laboratory and clinic efforts have to be aimed in fact not towards achieving outstanding KPI values, but towards application and integration of quality monitoring to increase the number of protocols resulting in embryo implantation and birth of a healthy child. Modern infertility treatment methods involve a personalized approach to individual patients, their treatment protocols, and specific embryos within them [ 5 ]. To facilitate this, many clinics have introduced medical information systems and time-lapse (TL) imaging equipment to monitor the fate of individual fertilized oocytes. Applying artificial intelligence strategies to TL, a tool for additional embryo ranking has been developed, estimating the implantation odds [ 6 , 7 ]. However, this approach entails the acquisition of additional expensive equipment: TL incubators and software, as well as allocation of additional human and time resources. Assessing the feasibility of such an approach is beyond the scope of this study, but in practice, it may not always be fully achievable in the laboratory setting. For extrapolation to patients from different clinics in various regions and countries, and for successful operation, such algorithms require additional validation in a specific laboratory with individual patient population characteristics [ 8 ]. However, there is very little open access data for validation assessments performing in such decision-making systems. Another deal is that the majority of the studies on the TL imaging effectiveness are conducted on private databases, which do not allow an independent quality assessment of the models offered as commercial solutions. To simplify the presentation of the necessary information for subsequent predicting, alternative methods have been developed, including principal component analysis for models of embryo selection based on previous IVF protocols and embryo morphology before transfer [ 9 ]. But this method still requires embryo images, which are often impossible to achieve in some clinics. However, most laboratory KPIs, used for quality control and competence level maintenance, are overlooked by these algorithms [ 10 , 11 ]. Taking into account the limitations of existing systems, the aim of this study was to develop a different approach to laboratory data analysis with deep neural networks (DNN) to predict the chances of clinical pregnancy occurrence in a specific protocol based on a combination of IVF laboratory KPIs with the sum of their rankings (KPIScore [ 12 ]) and clinical data. The major reason to select DNN was that factors influencing the outcome of embryo transfer are not always obvious, which is confirmed by abundance of “unexplained infertility” diagnoses. It is evident that manipulations with the embryo in laboratory conditions ( in vitro ) and its individual development cannot be adequately described by conventional linear mathematical models and requires comprehensive approaches to enhance our understanding. Results The developed neural network makes predictions based on 19 parameters: 13 of them are recorded in the laboratory database, and 6 are mathematically calculated in the script code (Supplementary 1). Using XGBoost we justify correlation of values with pregnancy occurrence for selected parameters, considering their significance in predicting the embryo transfer outcome. Model training and performance A total of 8732 complete protocols were available for primary analysis. After selecting the protocols containing all the necessary information for neural network training, 3858 cases of embryo transfer with known outcomes have been used to form the training (80%) and validation (20%) set, and 4874 protocols without known transfer outcome were used for model pre-training process with partial teacher involvement for the purpose of propria weights forming. It was the first stage in the DNN development. The developed model exhibits a significantly higher accuracy (0.67; AUC 0.63, SD = 0.058; U-Statistic 230.0, p-value 0.002) compared to the Gradient Boosting, Random Forest, and Decision Tree models. A correlation analysis between model accuracy and various age groups of patients (up to 29; 30–34; 35–39; over 40 years) was conducted, considering different methods of fertilization (IVF and ICSI). No significant (p > 0.05) differences were observed. In cross-validation, Average Accuracy 0.68 (SD = 0.05), Maximum Accuracy 0.83 were observed. For model propria training and fitting, we added transfer outcomes from cryo protocols with known implantation data to the previous dataset (totally all 8732 protocols have been used). We split that data to train (70%), validation (20%) and test (10%) sets. The stratified random sampling approach was used to ensure that all split data sets have the same distribution of pregnancy “positive” and “negative” classes. On this data, a high stability of model was observed: Accuracy 0.74; Sensitivity 0.6; Specificity 0.66; PPV 0.46; NPV 0.78; FPR 0.33; FNR 0.4; Overall Accuracy (test) 0.71, AUC 0.68 (SD = 0.054). Validation and fine-tuning Concordance validation between actual and predicted values of pregnancy occurrence using DNN was carried out on 1600 IVF protocols with embryos after PGT-A at the validation dataset. Accuracy of 0.7685 with an 86% sensitivity for pregnancy class (the proportion of actual positive results detected by the model out of all actual positive results) and F1-score of 0.71 have been obtained. We compared our neural network model to Logistic Regression, Gradient Boosting, Random Forest, and AdaBoost using leave one out (LOO) cross-validation for performance evaluation in a single IVF protocol. Our research demonstrates that the neural network has performed favorably across all considered metrics, notably standing out with a high MCC (0.412) that indicates its ability for precise and balanced classification (Table 1 ). Table 1 Leave one out cross-validation for different ML models with performance metrics Model AUC CA F1 Precision Recall MCC DNN 0.798 0.735 0.732 0.730 0.735 0.412 Logistic Regression 0.787 0.720 0.711 0.712 0.720 0.366 Gradient Boosting 0.779 0.712 0.708 0.706 0.712 0.359 Random Forest 0.764 0.706 0.702 0.700 0.706 0.346 AdaBoost 0.739 0.694 0.692 0.690 0.694 0.325 DNN - Deep learning neural network; AUC - Area Under the ROC Curve; CA - Classification Accuracy; F1: The harmonic mean of precision and recall; MCC - Matthews Correlation Coefficient After training the DNN, we utilized the probabilities obtained and applied logistic regression for calibration: Precision 0.837 and Recall 0.66. On 5-fold cross validation the DNN model demonstrated high specificity (0.864) and ability to correctly predict clinical pregnancy (Average Accuracy 0.781, SD = 0.04, Maximum Accuracy 0.875) (Table 2 ). Table 2 DNN model performance after tuning on 5 fold cross-validation Folds AUC Sensitivity Specificity PPV NPV FPR FNR Accuracy 1 0,849 0,714 0,833 0,769 0,789 0,167 0,286 0,781 2 0,888 0,583 0,850 0,700 0,773 0,150 0,417 0,750 3 0,895 0,400 0,909 0,667 0,769 0,091 0,600 0,750 4 0,823 0,700 0,773 0,583 0,850 0,227 0,300 0,750 5 0,864 0,700 0,955 0,875 0,875 0,045 0,300 0,875 Mean 86,37% 61,95% 86,39% 71,88% 81,13% 13,61% 38,05% 78,13% SD 0,024 0,109 0,057 0,090 0,039 0,057 0,109 0,044 AUC - Area Under the ROC Curve; PPV – Positive predictive value; NPV – Negative predictive value; FPR – False positive rate; FNR – False negative rate. The model then was fine-tuned based on the database from 2018 to 2023 (3500 protocols with known implantation data) for all embryos with oocyte source specification (autological or donor). During this period, a comprehensive quality control system was being implemented in the laboratory, and there was no variability in approaches to oocyte pick-up, embryo culture and transfer. Average Accuracy 0.855 (SD = 0.027), Maximum Accuracy 0.914, Minimum Accuracy 0.786, AUC 0.86 (SD = 0.06). In the validation process for the pregnancy occurrence prediction accuracy, a quarterly analysis for 2022–2023 was conducted, comparing the model's predictions with the actual clinical report and statistics. The average difference between the calculated predictions and reports was 2.56%, not significant (U-Statistics = 20.0, p value = 0.62). External validation For external validation, we used a distinct patient population with significantly different data distribution (KPIScore t-statistics − 9.05378, p-value < 0.0001) from independent 2 centers from different countries (Russia#1 with the majority of poor-prognosis patients (2013–2023 years), 6240 protocols; and Georgia #2 with good prognosis patients and the majority of cycles with donor oocytes (2022–2024 years) 3888 protocols). From the protocols provided by clinics #1 and #2, those were selected executed by the current team of embryologists (2 embryologists from clinic #1 and 4 from clinic #2). Criteria for protocol exclusion: incomplete data, errors in information provision, and procedures performed by multiple embryologists. Protocols meeting the Vienna consensus criteria were chosen to prepare the dataset for individual staff KPI analysis. Quarterly analysis (for years 2013–2023 in clinic #1 and 2022–2024 in clinic #2) of the input laboratory parameters showed stability in MII oocyte rate (med 87%, Q1-Q3 0.855–0.893) above the Maribor consensus competency threshold (≥ 74%, target value ≥ 90%), and its invariance across years. Statistically significant differences in laboratory KPIs between IVF and ICSI procedures were not found for fertilization rate (U statistic 4687.0, p-value 0.2945), blastocyst development rate (U statistic 6096.5, p-value 0.0528), TGBDR (U statistic 4920.5, p-value 0.90306), and availability of oocytes for fertilization (U statistic 5404.0, p-value 0.8479), allowing the use of these datasets as reference examples for validation the training process of our model in a specific IVF center. The mean AUC 0.73 was reached during that external validation. Explanation of the Model results The importance of explaining and interpreting the results of neural network models used in reproductive medicine is critical for ensuring transparency and trust both for patients and medical professionals. Understanding which factors have influence on clinical pregnancy probability is particularly important to ensure the most effective treatment and to improve outcomes. We chose linear regression to analyze probabilities predicted by the neural network in order to find the simplest approach to increase its interpretability. The dependent variable was predicted probability of pregnancy, and the independent variables were parameters used in the model. The mean squared error value was 0.00267. This suggests that the model's predictions are close to the actual values of pregnancy probability. The coefficient of determination (R^2) was found to be 0.937, indicating that approximately 93.7% of the variance in pregnancy probability is explained by the variables in the model, confirming the model's high predictive potential. We utilized SHAP, ICE and LIME algorithms to interpret the results of DNN model. Combining these methods, we were able to determine the average minimum conditions of the IVF laboratory stage for pregnancy occurrence in a specific protocol. For our DNN predictions the following threshold values have been obtained: KPIScore = 15, Number of follicles = 4, OCC = 3, MII = 2, 2pN = 2, Number of D3 embryos = 2, Number of D5 embryos = 1, Good quality blastocysts = 1, Transferred embryos = 1 and Patient Age = 36. With the LIME algorithm we observed that for positive pregnancy result it is necessary to reach KPIScore = 17, OCC retrieval rate = 0.83, MII rate = 0.78, Fertilization rate = 0.64, Blastocyst development rate = 0.44, TGBDR = 0.35. It is remarkable that those values are consistent with Vienna and Maribor consensus opinion for good IVF practice. Discussion The first neural network prediction model in IVF based on the clinical data was introduced in 1997 [ 13 ]. We simulated it in this study but unfortunately, its accuracy and reproducibility were insufficient to predict the likelihood of clinical pregnancy occurrence in IVF protocol (loss 0.6658, accuracy 0.5735). This led to analysis of additional publications and studies involving modeling of raw protocol data and its embryological component in outcome prediction [ 14 – 17 ]. The aim was to select optimal input parameters to predict the embryo implantation. Analysis of linear dependencies for those parameters makes it quite challenging to trace their influence on the embryo transfer outcome. That's why we employed a DNN in our work instead of other ML algorithms. A comparison between the developed model and other ML approaches described in the literature has been conducted on our data [ 18 ]: The DNN exhibits a significantly higher (U-Statistic 230.0, p-value 0.002) accuracy (0.67; AUC 0.63, SD = 0.058). That means a high difference in data from multiple IVF clinics and importance of multicenter validation for all ML models before practical implementation. We observed comparative results with the van Loendersloot's model capacity (AUC 0.64, CI 0.61–0.67) [ 19 ]. That result demonstrated that our DNN was robust on the pretraining step. After a complete training process, the DNN (AUC 0.68, SD = 0.054), in comparison with AI models described in the literature, has revealed advantages in terms of implantation probability prediction (AUC 0.573 [ 20 ]; 0.629 [ 21 ]; 0.543 [ 22 ]; 0.638 [ 23 ]; 0.64 [ 24 ]). After fine tuning we achieved DNN Average Accuracy 0.855, SD = 0.027. We compared our DNN model with three fine-tuned pre-trained convolutional neural network (CNN) models for pregnancy prediction utilizing the ImageNet system: VGG16, ResNet50, DenseNet121. The best performance was obtained in the ResNet50 architecture in reconstructed image data, with an AUC 0.741, Accuracy 0.682, Sensitivity 0.7114, Specificity 0.669. Two other CNN showed comparative but slightly lower results [ 25 ]. Only sensitivity was higher than in our DNN model because of actual blastocyst images in the CNN models. Therefore, those embryos that did not develop to the blastocyst stage were excluded. During validation, our model exhibited AUC = 0.67–0.75 (U- Statistic 145.0, p-value 0.471), comparable to the AI KIDScore™ (for known implantation data AUC 0.66, CI 0.60–0.75 across different clinics). Additionally, the model after fitting demonstrated a curve similar to the KIDScore™ (0.86 vs 0.89) for all embryos. This convincingly illustrates performance in predicting the pregnancy occurrence frequency for the developed model equivalent to commercially utilized TL systems [ 26 – 29 ], also fitted with additional clinical data (AUC 0.72–0.78) [ 24 , 30 ]. Possible discrepancies in quality metrics between our DNN model and traditional TL systems can be explained by different features used as the variables for outcome prediction. Our neural network allows looking at the formation path of necessary features for assessing the chances of clinical pregnancy occurrence using ML from a different perspective - introducing laboratory KPI to the analysis. Meanwhile, TL models utilize morphokinetic data of individual embryo development for forecasting. Traditional embryo quality assessment combined with KPI consideration enables reasonably accurate predictions, similar to constant video monitoring in TL. However, embryo developmental kinetics data utilization involves analyzing a large number of images, which introduces noise into the primary data and its final assessment, unlike calculated KPIs or static images. The accuracy of 78.13% and sensitivity of 62% of the model in clinical pregnancy prediction are just as good as in the Irvine Scientific Life Whisperer model [ 31 ] (accuracy 76.85%, sensitivity 70.1%, specificity 60.5%, combined accuracy 64.3%); and the FiTTE model [ 32 ] (accuracy 65.2%, AUC 0.71) and has a significantly (p < 0.01) better AUC (0.67–0.75) than in traditionally used predictive models (AUC 0.6202–0.6367) [ 33 , 34 ] on external validation data. Finally, our model validated on PGT-A protocols demonstrated higher performance than IDAScore V2 [ 35 ] for the same patient category (AUC 0.654) and similar performance to the Eeva model (AUC 0.698–0.744) [ 36 ]. Predictive ability of the model is comparable to those of GERI AI with overall accuracy of 67.8%, AUC 0.61–0.65 [ 37 ] and of MIRI AI model with AUC 0.69 [ 38 ]. At this point, it is difficult to assert confidently whether our algorithm is better or worse than traditionally used ones, additional multicenter studies are required. However, it enables a more comprehensive evaluation of implantation predictors related to intralaboratory conditions and is easier to interpret within the framework of quality control programs compared to TL assessment. DNN performance metrics are the same that are reported by Alife Health artificial intelligence model with AUC 0.62–0.64 [ 39 ], Fairtility artificial intelligence model (AUC 68–0.70) [ 40 ] and CHLOE-EQ model (AUC 64–0.726) [ 24 ]. Compared to the available ALIFE IVF success rate online calculator, our DNN model provided lower pregnancy probabilities (0.43 vs 0.30) which are much more reliable to actual implantation rate in single embryo transfer protocols (0.33) from the analyzed data set. The results achieved have shown the real importance of laboratory KPIs for precise prediction in individual protocols. This approach gives less optimistic predictions than those based only on clinical data. ALIFE model has been trained on a different patient population, and assured model comparison may be held only after tuning of both models on the same training data. After logistic regression calibration we compared our DNN model to the calibrated Large-scale simulation model reported at the ESHRE Annual Meeting [ 41 ]. It has the same results of the average pregnancy rate for the top-ranked embryos (60% vs 59.4%) with the minimal pregnancy possibility of 18.4% in low grade embryos. These results convincingly demonstrate effectiveness and reproducibility of the DNN model prediction algorithm (mean AUC 0.86) compared with other ML (AUC 0.632) for single embryo transfer [ 42 ], along with ML spatial stream model (AUC 0.76), temporal stream model (AUC 0.77), and ensemble model STEM (AUC 0.82) [ 43 ]. The model interpretation with SHAP, LIME and ICE methods has established the minimum requirements for pregnancy achievement. Concerning prognosis of the treatment cycle outcome, they accord with Bologna Criteria [ 44 ] for poor responders and POSEIDON (Patient-Oriented Strategies Encompassing Individualized Oocyte Number Criteria) [ 45 ]. Also, they demonstrate the same range of essential KPI levels that were proposed in the Vienna and the Maribor consensus. Conclusion Thus, our DNN model provides the ability to determine a theoretically justified actual potential of pregnancy occurrence probability in a specific patient group by using KPIs data from individual protocol. Above all else, application of the model as a quality control tool is valid in terms of accurate determination of competency level threshold values in pregnancy rate for individual clinics with their unique patient subpopulation, allowing to identify specific time frames for audits, and areas of increased interest during quality control assessments. Materials and methods Data collection We used retrospective data with known outcomes in “IVF and Genetic Center”, Moscow, Russia from SQL database in the medical informative system Medwork 3.5 from January 2013 till January 2024 to develop a data set for DNN model: totally 8732 protocols (3856 with fresh embryo transfer). Additional data from 2018–2024 (3500 protocols) and for PGT-A embryos (1600 protocols) was used for model internal validation. For external model validation we used data from 2 independent ART centers: “#1” Moscow, Russia with 6240 protocols and “#2” Tbilisi, Georgia with 3888 protocols. All the protocols with missing data values were discarded from the study. Patient informed consent for that study was not necessary because only retrospective and fully de-identified data from embryo development has been used, fully noninvasive for patients or their embryos (no medical intervention was performed on the subject, and no biological samples from the patient were collected to develop that model). For embryo evaluation, the ESHRE recommendations and Gardner blastocyst grading system was used, in which “good blastocysts” were identified as Bl3BB and higher grade. Our DNN model incorporates KPI data on the base of the Vienna Consensus, showing a correlational dependence with occurrence of pregnancy. We also use fine-tuning KPI for total good blastocyst development rate (TGBDR) in our model according to publications [ 11 ]. TGBDR has positive correlation with all itralaboratory KPIs and negative correlation with patient age in our data. Model also includes the rank sum of laboratory and clinical parameters (KPIScore), used with adaptation to our data (SART embryo grading and antral follicle count on the trigger day) to forecast the chance of pregnancy occurrence [ 12 ]. The positive correlation (+ 0.785) for KPIScore with TGBDR has been observed. Statistical analysis For data set description, statistical analysis of individual KPIs was conducted using the StatTech v. 3.0.6 software. Quantitative indicators with a normal distribution were described using the mean and standard deviation (SD), with 95% confidence intervals. Direction and strength of the correlation between two quantitative variables were evaluated using the Spearman rank correlation coefficient. For statistical approach P-value < 0.05 was used as a significant threshold limit in analysis of clinical features. Comparison of groups based on quantitative indicators was performed with one-way analysis of variance (ANOVA), with post hoc comparisons using the Kruskal-Wallis test for non-normal distribution. Analysis was performed with Information Gain, Gini Coefficient, ANOVA, Chi-square (χ²) and ReliefF to evaluate feature importance in classification tasks (Table 3 ). Table 3 Evaluating feature importance to identify the most informative variables for DNN training Rank Metric Info. gain Gain ratio Gini ANOVA χ² ReliefF 1 KPIScore 0.107 0.054 0.062 1421.400 906.190 0.017 2 Patient age 0.085 0.043 0.051 1045.865 747.026 0.016 3 Transfer day 0.080 0.051 0.051 1036.645 1281.690 0.009 4 Number of oocytes 0.073 0.036 0.045 750.572 721.893 0.009 5 Antral follicle count 0.073 0.036 0.043 648.521 629.716 0.009 6 Day 3 embryos number 0.072 0.036 0.042 614.261 617.259 0.004 7 Total Blastocyst number 0.068 0.036 0.042 629.322 880.509 0.002 8 2pN 0.065 0.033 0.038 557.655 548.234 0.007 9 Blastocyst rate 0.064 0.034 0.040 699.393 731.757 0.004 10 Number of inseminated 0.062 0.031 0.036 505.209 515.078 0.010 11 Good Blastocyst number 0.062 0.037 0.039 475.841 934.627 0.003 12 TGBDR 0.060 0.037 0.038 472.284 800.634 0.009 13 Day 5 embryos number 0.059 0.030 0.036 571.130 643.703 0.001 14 Number of transferred embryos 0.039 0.025 0.025 281.349 254.111 0.002 15 IVF attempt number 0.037 0.020 0.022 313.250 515.722 0.008 16 Number of cryo embryos 0.028 0.015 0.018 236.567 325.789 0.003 17 Fertilization rate 0.026 0.013 0.017 106.370 13.275 0.018 18 Cleavage rate 0.022 0.031 0.013 476.941 49.713 0.001 19 Oocyte retrieval rate 0.018 0.011 0.012 2.625 28.416 0.002 TGBDR - total good quality blastocyst development rate Python 3.6, Scikit-learn 1.4.2 and Sklearn 1.4 were used to implement machine learning models and statistical modeling. A t-test was used to determine whether there was a significant difference between the means of two groups. Mann-Whitney U Test (Wilcoxon Rank Sum Test) was used to compare differences between two groups: model predicted values and real clinical pregnancy rate in different patient groups, staff members and time periods. Model Compilation The neural network model has been developed and executed in the GPU PyCharm 17.0.10 environment using Python 3.6 programming language with the Tensorflow 2.15.0 and Keras library 2.14.0. The proposed neural network model is a sequential Recurrent Neural Network (RNN) consisting of two layers with L2 and L1 regularization to control overfitting by adding penalties to the model's weights. The first layer - SimpleRNN with 32 neurons with Rectified Linear Activation (ReLU). The second layer - SimpleRNN − 16 neurons and ReLU activation with added Dropout layer to prevent model overfitting by randomly excluding neurons during training. The final Dense layer with one neuron and Sigmoid activation, designed for binary classification. Sigmoid is used to predict the probability of a positive class - a positive outcome of the embryo transfer procedure. This DNN model was optimized using the gradient descent method with the Adam optimization algorithm with learning rate 0.0001. Binary cross-entropy was used as the loss function. The model was trained on a data array for 12 epochs (independently passes through the entire data frame, splitting it into train (70%), validation (20%) and test (10%) set) with a batch size of 8. Cross-validation for the model was conducted to evaluate it on 5 different data splits (folds). Calibration of the DNN was performed using the CalibratedClassifierCV from Scikit-learn, which applies logistic regression to align probabilities. A comparative analysis of prediction errors was conducted with Area under the receiver operating characteristic curve (AUC), Accuracy, Positive Predictive Value (PPV), Negative Predictive Value (NPV), False Positive Rate (FPR), False Negative Rate (FNG), Specificity, Sensitivity and Matthews Correlation Coefficient (MCC). Explanation of the Model results In the context of approaches to interpret the DNN results, we have utilized SHAP (SHapley Additive exPlanations), ICE (Individual Conditional Expectation) and LIME (Local Interpretable Model-agnostic Explanations) methods. Declarations Competing interests The authors declare no conflict of interest and no competing interests. Author Contribution S.S. and I.D. designed the study. S.S. conducted studies on the establishment of DNN architecture, implemented machine learning for comparison, and analyzed the data. I.D. interpreted the external validation data and drafted the manuscript. All authors reviewed the manuscript. Data Availability https://github.com/embryossa/DNN.git References Glatstein, I., Chavez-Badiola, A., & Curchoe, C. L. New frontiers in embryo selection. Journal of assisted reproduction and genetics. 40, 223–234. DOI: https://doi.org/10.1007/s10815-022-02708-5 (2023). Alizadehsani, R. et al. Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991–2020). Annals of operations research , 1–42; https://doi.org/10.1007/s10479-021-04006-2 (2021). Fernandez, E. et al. Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classification of reproductive data. Journal of assisted reproduction and genetics. 37, 2359–2376. DOI: https://doi.org/10.1007/s10815-020-01881-9 (2020). ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine. The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reproductive biomedicine online. 35, 494–510. DOI: https://doi.org/10.1016/j.rbmo.2017.06.015 (2017). Medenica, S. et al. The Future Is Coming: Artificial Intelligence in the Treatment of Infertility Could Improve Assisted Reproduction Outcomes-The Value of Regulatory Frameworks. Diagnostics. 12, 2979. DOI: https://doi.org/10.3390/diagnostics12122979 (2022). Uyar, A., Bener, A., & Ciray, H. N. Predictive Modeling of Implantation Outcome in an In Vitro Fertilization Setting: An Application of Machine Learning Methods. Medical decision making: an international journal of the Society for Medical Decision Making. 35, 714–725. DOI: https://doi.org/10.1177/0272989X14535984 (2015). Bamford, T. et al. A comparison of morphokinetic models and morphological selection for prioritizing euploid embryos: a multicentre cohort study. Human reproduction. 39, 53–61. DOI: https://doi.org/10.1093/humrep/dead237 (2024). Blais, I., Koifman, M., Feferkorn, I., Dirnfeld, M., & Lahav-Baratz, S. Improving embryo selection by the development of a laboratory-adapted time-lapse model. F&S science. 2, 176–197. DOI: https://doi.org/10.1016/j.xfss.2021.02.001 (2021). Xi, Q. et al. Individualized embryo selection strategy developed by stacking machine learning model for better in vitro fertilization outcomes: an application study. Reproductive biology and endocrinology. 19, 53. DOI: https://doi.org/10.1186/s12958-021-00734-z (2021). Ratna, M. B., Bhattacharya, S., Abdulrahim, B., & McLernon, D. J. A systematic review of the quality of clinical prediction models in in vitro fertilisation. Human reproduction. 35, 100–116. DOI: https://doi.org/10.1093/humrep/dez258 (2020). Zacà, C. et al. Fine-tuning IVF laboratory key performance indicators of the Vienna consensus according to female age. Journal of assisted reproduction and genetics. 39, 945–952. DOI: https://doi.org/10.1007/s10815-022-02468-2 (2022). Franco, J. G. et al. Key performance indicators score (KPIs-score) based on clinical and laboratorial parameters can establish benchmarks for internal quality control in an ART program. JBRA assisted reproduction. 21, 61–66. DOI: https://doi.org/10.5935/1518-0557.20170016 (2017). Kaufmann, S. J., Eastaugh, J. L., Snowden, S., Smye, S. W., & Sharma, V. The application of neural networks in predicting the outcome of in-vitro fertilization. Human reproduction. 12, 1454–1457. DOI: https://doi.org/10.1093/humrep/12.7.1454 (1997). Liu, H. et al. Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study. eLife. 12, e83662. DOI: https://doi.org/10.7554/eLife.83662 (2023). Blank, C. et al. Prediction of implantation after blastocyst transfer in in vitro fertilization: a machine-learning perspective. Fertility and sterility. 111, 318–326. DOI: https://doi.org/10.1016/j.fertnstert.2018.10.030 (2019). Raef, B., Maleki, M., & Ferdousi, R. Computational prediction of implantation outcome after embryo transfer. Health informatics journal. 26, 1810–1826. DOI: https://doi.org/10.1177/1460458219892138 (2020). Bormann, C. L. et al. Performance of a deep learning based neural network in the selection of human blastocysts for implantation. eLife . 9, e55301. DOI: https://doi.org/10.7554/eLife.55301 (2020). Li, L., Cui, X., Yang, J., Wu, X., & Zhao, G. Using feature optimization and LightGBM algorithm to predict the clinical pregnancy outcomes after in vitro fertilization. Frontiers in endocrinology. 14, 1305473. DOI: https://doi.org/10.3389/fendo.2023.1305473 (2023). Sarais, V. et al. Predicting the success of IVF: external validation of the van Loendersloot's model. Human reproduction. 31, 1245–1252. DOI: https://doi.org/10.1093/humrep/dew069 . (2016). Chamayou, S. et al. The use of morphokinetic parameters to select all embryos with full capacity to implant. Journal of assisted reproduction and genetics. 30, 703–710. DOI: https://doi.org/10.1007/s10815-013-9992-2 (2013). Basile, N. et al. The use of morphokinetics as a predictor of implantation: a multicentric study to define and validate an algorithm for embryo selection. Human reproduction. 30, 276–283. DOI: https://doi.org/10.1093/humrep/deu331 (2015). Dal Canto, M. et al. Faster fertilization and cleavage kinetics reflect competence to achieve a live birth after intracytoplasmic sperm injection, but this association fades with maternal age. Fertility and sterility. 115, 665–672. DOI: https://doi.org/10.1016/j.fertnstert.2020.06.023 (2021). Bori, L. et al. The higher the score, the better the clinical outcome: retrospective evaluation of automatic embryo grading as a support tool for embryo selection in IVF laboratories. Human reproduction. 37, 1148–1160. DOI: https://doi.org/10.1093/humrep/deac066 (2022). Benchaib, M., Labrune, E., Giscard d'Estaing, S., Salle, B., & Lornage, J. Shallow artificial networks with morphokinetic time-lapse parameters coupled to ART data allow to predict live birth. Reproductive medicine and biology. 21, e12486. DOI: https://doi.org/10.1002/rmb2.12486 (2022). Kim, H. M. et al. Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images. Scientific reports. 14, 3240. DOI: https://doi.org/10.1038/s41598-024-52241-x (2024). Fréour, T. et al. External validation of a time-lapse prediction model. Fertility and sterility. 103, 917–922. DOI: https://doi.org/10.1016/j.fertnstert.2014.12.111 (2015). Tran, D., Cooke, S., Illingworth, P. J., & Gardner, D. K. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Human reproduction. 34, 1011–1018. DOI: https://doi.org/10.1093/humrep/dez064 (2019). Reignier, A. et al. Performance of Day 5 KIDScore™ morphokinetic prediction models of implantation and live birth after single blastocyst transfer. Journal of assisted reproduction and genetics. 36, 2279–2285. DOI: https://doi.org/10.1007/s10815-019-01567-x (2019). Berntsen, J., Rimestad, J., Lassen, J. T., Tran, D., & Kragh, M. F. Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences. PloS one. 17, e0262661. DOI: https://doi.org/10.1371/journal.pone.0262661 (2022). Lee, C. I. et al. Associations between the artificial intelligence scoring system and live birth outcomes in preimplantation genetic testing for aneuploidy cycles. Reproductive biology and endocrinology. 22, 12. DOI: https://doi.org/10.1186/s12958-024-01185-y (2024). VerMilyea, M. et al. Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF. Human reproduction. 35, 770–784. DOI: https://doi.org/10.1093/humrep/deaa013 (2020). Enatsu, N. et al. A novel system based on artificial intelligence for predicting blastocyst viability and visualizing the explanation. Reproductive medicine and biology. 21, e12443. DOI: https://doi.org/10.1002/rmb2.12443 (2022). Nelson, S. M., & Lawlor, D. A. Predicting live birth, preterm delivery, and low birth weight in infants born from in vitro fertilisation: a prospective study of 144,018 treatment cycles. PLoS medicine. 8, e1000386. DOI: https://doi.org/10.1371/journal.pmed.1000386 (2011). Ratna, M. B., Bhattacharya, S., & McLernon, D. J. External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment. Human reproduction. 38, 1998–2010. DOI: https://doi.org/10.1093/humrep/dead165 (2023). Ueno, S., Berntsen, J., Okimura, T., & Kato, K. Improved pregnancy prediction performance in an updated deep-learning embryo selection model: a retrospective independent validation study. Reproductive biomedicine online. 48, 103308. DOI: https://doi.org/10.1016/j.rbmo.2023.103308 (2024). Tzukerman, N. et al. Using Unlabeled Information of Embryo Siblings from the Same Cohort Cycle to Enhance In Vitro Fertilization Implantation Prediction. Advanced science. 10, e2207711. DOI: https://doi.org/10.1002/advs.202207711 (2023). Diakiw, S. M. et al. An artificial intelligence model correlated with morphological and genetic features of blastocyst quality improves ranking of viable embryos. Reproductive biomedicine online. 45, 1105–1117. DOI: https://doi.org/10.1016/j.rbmo.2022.07.018 (2022). Duval, A. et al. A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems. Human reproduction. 38, 596–608. DOI: https://doi.org/10.1093/humrep/dead023 (2023). Erlich, I. et al. Pseudo contrastive labeling for predicting IVF embryo developmental potential. Scientific reports. 12, 2488. DOI: https://doi.org/10.1038/s41598-022-06336-y (2022). Loewke, K. et al. Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos. Fertility and sterility. 117, 528–535. DOI: https://doi.org/10.1016/j.fertnstert.2021.11.022 (2022). Cho, J.H. et al. Large-scale simulation of pregnancy rate improvements using an AI model for embryo ranking. 38th Hybrid Annual Meeting of the ESHRE (2022). Sayed, S. et al. Time-lapse imaging derived morphokinetic variables reveal association with implantation and live birth following in vitro fertilization: A retrospective study using data from transferred human embryos. PloS one. 15, e0242377. DOI: https://doi.org/10.1371/journal.pone.0242377 (2020). Liao, Q. et al. Development of deep learning algorithms for predicting blastocyst formation and quality by time-lapse monitoring. Communications biology. 4, 415. DOI: https://doi.org/10.1038/s42003-021-0193Cho7-1 (2021). Ferraretti, A. P. et al. ESHRE consensus on the definition of 'poor response' to ovarian stimulation for in vitro fertilization: the Bologna criteria. Human reproduction. 26, 1616–1624. DOI: https://doi.org/10.1093/humrep/der092 (2011). Esteves, S. C. et al. The POSEIDON Criteria and Its Measure of Success Through the Eyes of Clinicians and Embryologists. Frontiers in endocrinology. 10, 814. DOI: https://doi.org/10.3389/fendo.2019.00814 (2019). Additional Declarations No competing interests reported. Supplementary Files SupplementaryInformationfile.pdf Cite Share Download PDF Status: Published Journal Publication published 27 Nov, 2024 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 05 Jul, 2024 Reviews received at journal 22 Jun, 2024 Reviews received at journal 11 Jun, 2024 Reviewers agreed at journal 27 May, 2024 Reviewers agreed at journal 22 May, 2024 Reviewers invited by journal 22 May, 2024 Editor assigned by journal 22 May, 2024 Editor invited by journal 22 May, 2024 Submission checks completed at journal 22 May, 2024 First submitted to journal 19 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4445375","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":309327520,"identity":"51fedc34-ffd3-4f0a-b09a-05cb2c21ca24","order_by":0,"name":"Sergei Sergeev","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1UlEQVRIiWNgGAWjYFACHijN3gAkDCxI0cJzAKRFghQtEglgkrAGfrGzxz7zVNTZ8898fnXDjwIJBv727gS8WiRn5yXP5jlzOHHG7Zyymz1Ah0mcObsBrxaD2znGzLxtBxIYbuek3eABajGQyMWvxR6ipc5e/uaZtJt/iNFiIA3Wwsy44Qb7sdtE2SJxOy+ZcQ7QLxvP5LDdljGQ4CHoF/7ZuYcZ3gBDTO748Wc33/yxkeNv78WvBQSYIFHDYwAmCSoHAcYfYIr9AVGqR8EoGAWjYOQBALYMRZi5rLZ2AAAAAElFTkSuQmCC","orcid":"","institution":"IVF and Genetic Center","correspondingAuthor":true,"prefix":"","firstName":"Sergei","middleName":"","lastName":"Sergeev","suffix":""},{"id":309327521,"identity":"7562207a-5321-4406-a36d-30be7ca812bc","order_by":1,"name":"Iuliia Diakova","email":"","orcid":"","institution":"ART-IVF Clinic","correspondingAuthor":false,"prefix":"","firstName":"Iuliia","middleName":"","lastName":"Diakova","suffix":""}],"badges":[],"createdAt":"2024-05-19 18:27:46","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4445375/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4445375/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-024-80759-7","type":"published","date":"2024-11-27T15:57:26+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":70382579,"identity":"a2f07ba1-ad6d-45cf-8d3f-101e02febed6","added_by":"auto","created_at":"2024-12-02 16:27:57","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":575998,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4445375/v1/bb8871e6-8750-45b3-b4ec-e4c031909663.pdf"},{"id":57717060,"identity":"fdd12289-df61-4591-a385-595bead1dc8d","added_by":"auto","created_at":"2024-06-04 17:31:24","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":50290,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformationfile.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4445375/v1/b25c4ecca0e91006c4a6c49f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Advanced KPI Framework for IVF Pregnancy Prediction Models in IVF protocols","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn recent decades, neural networks and machine learning models (ML) have become key tools in various fields, including assisted reproductive technology (ART). They serve as a foundation for creating algorithms capable of extracting complex dependencies from data, and making decisions based on these dependencies. In some cases, neural networks can identify a broader spectrum of associations than other statistical methods, thanks to their ability to recognize highly nonlinear associations among input parameters [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eMost ML models applied in the field of in vitro fertilization (IVF) are based on regression and logistic regression algorithms to identify relationships between the target variable (outcome metric of clinic success) and input parameters [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. However, most models used for that evaluation are based on patient data and their previous IVF protocols, often failing to track the patterns in changes of quality laboratory indicators relevant to the final transfer outcome [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe importance of the laboratory stage in IVF is explained in detail in consensus resolutions that outline quality control (QC) measures and the evaluation of key performance indicators (KPIs), in accordance with international standards.[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Nevertheless, it is hard to disagree that laboratory and clinic efforts have to be aimed in fact not towards achieving outstanding KPI values, but towards application and integration of quality monitoring to increase the number of protocols resulting in embryo implantation and birth of a healthy child.\u003c/p\u003e \u003cp\u003eModern infertility treatment methods involve a personalized approach to individual patients, their treatment protocols, and specific embryos within them [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. To facilitate this, many clinics have introduced medical information systems and time-lapse (TL) imaging equipment to monitor the fate of individual fertilized oocytes. Applying artificial intelligence strategies to TL, a tool for additional embryo ranking has been developed, estimating the implantation odds [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. However, this approach entails the acquisition of additional expensive equipment: TL incubators and software, as well as allocation of additional human and time resources. Assessing the feasibility of such an approach is beyond the scope of this study, but in practice, it may not always be fully achievable in the laboratory setting. For extrapolation to patients from different clinics in various regions and countries, and for successful operation, such algorithms require additional validation in a specific laboratory with individual patient population characteristics [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. However, there is very little open access data for validation assessments performing in such decision-making systems. Another deal is that the majority of the studies on the TL imaging effectiveness are conducted on private databases, which do not allow an independent quality assessment of the models offered as commercial solutions.\u003c/p\u003e \u003cp\u003eTo simplify the presentation of the necessary information for subsequent predicting, alternative methods have been developed, including principal component analysis for models of embryo selection based on previous IVF protocols and embryo morphology before transfer [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. But this method still requires embryo images, which are often impossible to achieve in some clinics. However, most laboratory KPIs, used for quality control and competence level maintenance, are overlooked by these algorithms [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eTaking into account the limitations of existing systems, the aim of this study was to develop a different approach to laboratory data analysis with deep neural networks (DNN) to predict the chances of clinical pregnancy occurrence in a specific protocol based on a combination of IVF laboratory KPIs with the sum of their rankings (KPIScore [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]) and clinical data. The major reason to select DNN was that factors influencing the outcome of embryo transfer are not always obvious, which is confirmed by abundance of \u0026ldquo;unexplained infertility\u0026rdquo; diagnoses. It is evident that manipulations with the embryo in laboratory conditions (\u003cem\u003ein vitro\u003c/em\u003e) and its individual development cannot be adequately described by conventional linear mathematical models and requires comprehensive approaches to enhance our understanding.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThe developed neural network makes predictions based on 19 parameters: 13 of them are recorded in the laboratory database, and 6 are mathematically calculated in the script code (Supplementary 1). Using XGBoost we justify correlation of values with pregnancy occurrence for selected parameters, considering their significance in predicting the embryo transfer outcome.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eModel training and performance\u003c/h2\u003e \u003cp\u003eA total of 8732 complete protocols were available for primary analysis. After selecting the protocols containing all the necessary information for neural network training, 3858 cases of embryo transfer with known outcomes have been used to form the training (80%) and validation (20%) set, and 4874 protocols without known transfer outcome were used for model pre-training process with partial teacher involvement for the purpose of propria weights forming. It was the first stage in the DNN development. The developed model exhibits a significantly higher accuracy (0.67; AUC 0.63, SD\u0026thinsp;=\u0026thinsp;0.058; U-Statistic 230.0, p-value 0.002) compared to the Gradient Boosting, Random Forest, and Decision Tree models.\u003c/p\u003e \u003cp\u003eA correlation analysis between model accuracy and various age groups of patients (up to 29; 30\u0026ndash;34; 35\u0026ndash;39; over 40 years) was conducted, considering different methods of fertilization (IVF and ICSI). No significant (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05) differences were observed. In cross-validation, Average Accuracy 0.68 (SD\u0026thinsp;=\u0026thinsp;0.05), Maximum Accuracy 0.83 were observed.\u003c/p\u003e \u003cp\u003eFor model propria training and fitting, we added transfer outcomes from cryo protocols with known implantation data to the previous dataset (totally all 8732 protocols have been used). We split that data to train (70%), validation (20%) and test (10%) sets. The stratified random sampling approach was used to ensure that all split data sets have the same distribution of pregnancy \u0026ldquo;positive\u0026rdquo; and \u0026ldquo;negative\u0026rdquo; classes. On this data, a high stability of model was observed: Accuracy 0.74; Sensitivity 0.6; Specificity 0.66; PPV 0.46; NPV 0.78; FPR 0.33; FNR 0.4; Overall Accuracy (test) 0.71, AUC 0.68 (SD\u0026thinsp;=\u0026thinsp;0.054).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eValidation and fine-tuning\u003c/h2\u003e \u003cp\u003eConcordance validation between actual and predicted values of pregnancy occurrence using DNN was carried out on 1600 IVF protocols with embryos after PGT-A at the validation dataset. Accuracy of 0.7685 with an 86% sensitivity for pregnancy class (the proportion of actual positive results detected by the model out of all actual positive results) and F1-score of 0.71 have been obtained.\u003c/p\u003e \u003cp\u003eWe compared our neural network model to Logistic Regression, Gradient Boosting, Random Forest, and AdaBoost using leave one out (LOO) cross-validation for performance evaluation in a single IVF protocol. Our research demonstrates that the neural network has performed favorably across all considered metrics, notably standing out with a high MCC (0.412) that indicates its ability for precise and balanced classification (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eLeave one out cross-validation for different ML models with performance metrics\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAUC\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCA\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eF1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePrecision\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eRecall\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eMCC\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDNN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.798\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.735\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.732\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.730\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.735\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.412\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLogistic Regression\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.787\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.720\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.711\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.712\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.720\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.366\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGradient Boosting\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.779\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.712\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.708\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.706\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.712\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.359\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRandom Forest\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.764\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.706\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.702\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.700\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.706\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.346\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdaBoost\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.739\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.694\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.692\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.690\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.694\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.325\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"7\"\u003e\u003cem\u003eDNN - Deep learning neural network; AUC - Area Under the ROC Curve; CA - Classification Accuracy; F1: The harmonic mean of precision and recall; MCC - Matthews Correlation Coefficient\u003c/em\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eAfter training the DNN, we utilized the probabilities obtained and applied logistic regression for calibration: Precision 0.837 and Recall 0.66. On 5-fold cross validation the DNN model demonstrated high specificity (0.864) and ability to correctly predict clinical pregnancy (Average Accuracy 0.781, SD\u0026thinsp;=\u0026thinsp;0.04, Maximum Accuracy 0.875) (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDNN model performance after tuning on 5 fold cross-validation\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"9\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFolds\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAUC\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSensitivity\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eSpecificity\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003ePPV\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNPV\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eFPR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eFNR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eAccuracy\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,849\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,714\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,833\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,769\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,789\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,167\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,286\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,781\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,888\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,583\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,850\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,700\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,773\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,150\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,417\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,750\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,895\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,400\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,909\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,667\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,769\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,091\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,600\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,750\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,823\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,700\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,773\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,583\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,850\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,227\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,300\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,750\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,864\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,700\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,955\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,875\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,875\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,045\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,300\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,875\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMean\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e86,37%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e61,95%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e86,39%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e71,88%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e81,13%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e13,61%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e38,05%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e\u003cb\u003e78,13%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSD\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0,024\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0,109\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0,057\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0,090\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0,039\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0,057\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0,109\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e0,044\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cem\u003eAUC - Area Under the ROC Curve; PPV \u0026ndash; Positive predictive value; NPV \u0026ndash; Negative predictive value; FPR \u0026ndash; False positive rate; FNR \u0026ndash; False negative rate.\u003c/em\u003e \u003c/p\u003e \u003cp\u003eThe model then was fine-tuned based on the database from 2018 to 2023 (3500 protocols with known implantation data) for all embryos with oocyte source specification (autological or donor). During this period, a comprehensive quality control system was being implemented in the laboratory, and there was no variability in approaches to oocyte pick-up, embryo culture and transfer. Average Accuracy 0.855 (SD\u0026thinsp;=\u0026thinsp;0.027), Maximum Accuracy 0.914, Minimum Accuracy 0.786, AUC 0.86 (SD\u0026thinsp;=\u0026thinsp;0.06).\u003c/p\u003e \u003cp\u003eIn the validation process for the pregnancy occurrence prediction accuracy, a quarterly analysis for 2022\u0026ndash;2023 was conducted, comparing the model's predictions with the actual clinical report and statistics. The average difference between the calculated predictions and reports was 2.56%, not significant (U-Statistics\u0026thinsp;=\u0026thinsp;20.0, p value\u0026thinsp;=\u0026thinsp;0.62).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eExternal validation\u003c/h2\u003e \u003cp\u003eFor external validation, we used a distinct patient population with significantly different data distribution (KPIScore t-statistics \u0026minus;\u0026thinsp;9.05378, p-value\u0026thinsp;\u0026lt;\u0026thinsp;0.0001) from independent 2 centers from different countries (Russia#1 with the majority of poor-prognosis patients (2013\u0026ndash;2023 years), 6240 protocols; and Georgia #2 with good prognosis patients and the majority of cycles with donor oocytes (2022\u0026ndash;2024 years) 3888 protocols).\u003c/p\u003e \u003cp\u003eFrom the protocols provided by clinics #1 and #2, those were selected executed by the current team of embryologists (2 embryologists from clinic #1 and 4 from clinic #2). Criteria for protocol exclusion: incomplete data, errors in information provision, and procedures performed by multiple embryologists. Protocols meeting the Vienna consensus criteria were chosen to prepare the dataset for individual staff KPI analysis.\u003c/p\u003e \u003cp\u003eQuarterly analysis (for years 2013\u0026ndash;2023 in clinic #1 and 2022\u0026ndash;2024 in clinic #2) of the input laboratory parameters showed stability in MII oocyte rate (med 87%, Q1-Q3 0.855\u0026ndash;0.893) above the Maribor consensus competency threshold (\u0026ge;\u0026thinsp;74%, target value\u0026thinsp;\u0026ge;\u0026thinsp;90%), and its invariance across years. Statistically significant differences in laboratory KPIs between IVF and ICSI procedures were not found for fertilization rate (U statistic 4687.0, p-value 0.2945), blastocyst development rate (U statistic 6096.5, p-value 0.0528), TGBDR (U statistic 4920.5, p-value 0.90306), and availability of oocytes for fertilization (U statistic 5404.0, p-value 0.8479), allowing the use of these datasets as reference examples for validation the training process of our model in a specific IVF center. The mean AUC 0.73 was reached during that external validation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eExplanation of the Model results\u003c/h2\u003e \u003cp\u003eThe importance of explaining and interpreting the results of neural network models used in reproductive medicine is critical for ensuring transparency and trust both for patients and medical professionals. Understanding which factors have influence on clinical pregnancy probability is particularly important to ensure the most effective treatment and to improve outcomes.\u003c/p\u003e \u003cp\u003eWe chose linear regression to analyze probabilities predicted by the neural network in order to find the simplest approach to increase its interpretability. The dependent variable was predicted probability of pregnancy, and the independent variables were parameters used in the model. The mean squared error value was 0.00267. This suggests that the model's predictions are close to the actual values of pregnancy probability. The coefficient of determination (R^2) was found to be 0.937, indicating that approximately 93.7% of the variance in pregnancy probability is explained by the variables in the model, confirming the model's high predictive potential.\u003c/p\u003e \u003cp\u003eWe utilized SHAP, ICE and LIME algorithms to interpret the results of DNN model. Combining these methods, we were able to determine the average minimum conditions of the IVF laboratory stage for pregnancy occurrence in a specific protocol.\u003c/p\u003e \u003cp\u003eFor our DNN predictions the following threshold values have been obtained: KPIScore\u0026thinsp;=\u0026thinsp;15, Number of follicles\u0026thinsp;=\u0026thinsp;4, OCC\u0026thinsp;=\u0026thinsp;3, MII\u0026thinsp;=\u0026thinsp;2, 2pN\u0026thinsp;=\u0026thinsp;2, Number of D3 embryos\u0026thinsp;=\u0026thinsp;2, Number of D5 embryos\u0026thinsp;=\u0026thinsp;1, Good quality blastocysts\u0026thinsp;=\u0026thinsp;1, Transferred embryos\u0026thinsp;=\u0026thinsp;1 and Patient Age\u0026thinsp;=\u0026thinsp;36. With the LIME algorithm we observed that for positive pregnancy result it is necessary to reach KPIScore\u0026thinsp;=\u0026thinsp;17, OCC retrieval rate\u0026thinsp;=\u0026thinsp;0.83, MII rate\u0026thinsp;=\u0026thinsp;0.78, Fertilization rate\u0026thinsp;=\u0026thinsp;0.64, Blastocyst development rate\u0026thinsp;=\u0026thinsp;0.44, TGBDR\u0026thinsp;=\u0026thinsp;0.35. It is remarkable that those values are consistent with Vienna and Maribor consensus opinion for good IVF practice.\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe first neural network prediction model in IVF based on the clinical data was introduced in 1997 [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. We simulated it in this study but unfortunately, its accuracy and reproducibility were insufficient to predict the likelihood of clinical pregnancy occurrence in IVF protocol (loss 0.6658, accuracy 0.5735). This led to analysis of additional publications and studies involving modeling of raw protocol data and its embryological component in outcome prediction [\u003cspan additionalcitationids=\"CR15 CR16\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. The aim was to select optimal input parameters to predict the embryo implantation. Analysis of linear dependencies for those parameters makes it quite challenging to trace their influence on the embryo transfer outcome. That's why we employed a DNN in our work instead of other ML algorithms.\u003c/p\u003e \u003cp\u003eA comparison between the developed model and other ML approaches described in the literature has been conducted on our data [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]: The DNN exhibits a significantly higher (U-Statistic 230.0, p-value 0.002) accuracy (0.67; AUC 0.63, SD\u0026thinsp;=\u0026thinsp;0.058). That means a high difference in data from multiple IVF clinics and importance of multicenter validation for all ML models before practical implementation. We observed comparative results with the van Loendersloot's model capacity (AUC 0.64, CI 0.61\u0026ndash;0.67) [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. That result demonstrated that our DNN was robust on the pretraining step.\u003c/p\u003e \u003cp\u003eAfter a complete training process, the DNN (AUC 0.68, SD\u0026thinsp;=\u0026thinsp;0.054), in comparison with AI models described in the literature, has revealed advantages in terms of implantation probability prediction (AUC 0.573 [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]; 0.629 [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]; 0.543 [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]; 0.638 [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]; 0.64 [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]).\u003c/p\u003e \u003cp\u003eAfter fine tuning we achieved DNN Average Accuracy 0.855, SD\u0026thinsp;=\u0026thinsp;0.027. We compared our DNN model with three fine-tuned pre-trained convolutional neural network (CNN) models for pregnancy prediction utilizing the ImageNet system: VGG16, ResNet50, DenseNet121. The best performance was obtained in the ResNet50 architecture in reconstructed image data, with an AUC 0.741, Accuracy 0.682, Sensitivity 0.7114, Specificity 0.669. Two other CNN showed comparative but slightly lower results [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]. Only sensitivity was higher than in our DNN model because of actual blastocyst images in the CNN models. Therefore, those embryos that did not develop to the blastocyst stage were excluded.\u003c/p\u003e \u003cp\u003eDuring validation, our model exhibited AUC\u0026thinsp;=\u0026thinsp;0.67\u0026ndash;0.75 (U- Statistic 145.0, p-value 0.471), comparable to the AI KIDScore\u0026trade; (for known implantation data AUC 0.66, CI 0.60\u0026ndash;0.75 across different clinics). Additionally, the model after fitting demonstrated a curve similar to the KIDScore\u0026trade; (0.86 vs 0.89) for all embryos. This convincingly illustrates performance in predicting the pregnancy occurrence frequency for the developed model equivalent to commercially utilized TL systems [\u003cspan additionalcitationids=\"CR27 CR28\" citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e], also fitted with additional clinical data (AUC 0.72\u0026ndash;0.78) [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e].\u003c/p\u003e \u003cp\u003ePossible discrepancies in quality metrics between our DNN model and traditional TL systems can be explained by different features used as the variables for outcome prediction. Our neural network allows looking at the formation path of necessary features for assessing the chances of clinical pregnancy occurrence using ML from a different perspective - introducing laboratory KPI to the analysis. Meanwhile, TL models utilize morphokinetic data of individual embryo development for forecasting. Traditional embryo quality assessment combined with KPI consideration enables reasonably accurate predictions, similar to constant video monitoring in TL. However, embryo developmental kinetics data utilization involves analyzing a large number of images, which introduces noise into the primary data and its final assessment, unlike calculated KPIs or static images.\u003c/p\u003e \u003cp\u003eThe accuracy of 78.13% and sensitivity of 62% of the model in clinical pregnancy prediction are just as good as in the Irvine Scientific Life Whisperer model [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] (accuracy 76.85%, sensitivity 70.1%, specificity 60.5%, combined accuracy 64.3%); and the FiTTE model [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] (accuracy 65.2%, AUC 0.71) and has a significantly (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01) better AUC (0.67\u0026ndash;0.75) than in traditionally used predictive models (AUC 0.6202\u0026ndash;0.6367) [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e] on external validation data.\u003c/p\u003e \u003cp\u003eFinally, our model validated on PGT-A protocols demonstrated higher performance than IDAScore V2 [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] for the same patient category (AUC 0.654) and similar performance to the Eeva model (AUC 0.698\u0026ndash;0.744) [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. Predictive ability of the model is comparable to those of GERI AI with overall accuracy of 67.8%, AUC 0.61\u0026ndash;0.65 [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] and of MIRI AI model with AUC 0.69 [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAt this point, it is difficult to assert confidently whether our algorithm is better or worse than traditionally used ones, additional multicenter studies are required. However, it enables a more comprehensive evaluation of implantation predictors related to intralaboratory conditions and is easier to interpret within the framework of quality control programs compared to TL assessment.\u003c/p\u003e \u003cp\u003eDNN performance metrics are the same that are reported by Alife Health artificial intelligence model with AUC 0.62\u0026ndash;0.64 [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e], Fairtility artificial intelligence model (AUC 68\u0026ndash;0.70) [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e] and CHLOE-EQ model (AUC 64\u0026ndash;0.726) [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eCompared to the available ALIFE IVF success rate online calculator, our DNN model provided lower pregnancy probabilities (0.43 vs 0.30) which are much more reliable to actual implantation rate in single embryo transfer protocols (0.33) from the analyzed data set. The results achieved have shown the real importance of laboratory KPIs for precise prediction in individual protocols. This approach gives less optimistic predictions than those based only on clinical data. ALIFE model has been trained on a different patient population, and assured model comparison may be held only after tuning of both models on the same training data.\u003c/p\u003e \u003cp\u003eAfter logistic regression calibration we compared our DNN model to the calibrated Large-scale simulation model reported at the ESHRE Annual Meeting [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. It has the same results of the average pregnancy rate for the top-ranked embryos (60% vs 59.4%) with the minimal pregnancy possibility of 18.4% in low grade embryos. These results convincingly demonstrate effectiveness and reproducibility of the DNN model prediction algorithm (mean AUC 0.86) compared with other ML (AUC 0.632) for single embryo transfer [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e], along with ML spatial stream model (AUC 0.76), temporal stream model (AUC 0.77), and ensemble model STEM (AUC 0.82) [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe model interpretation with SHAP, LIME and ICE methods has established the minimum requirements for pregnancy achievement. Concerning prognosis of the treatment cycle outcome, they accord with Bologna Criteria [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e] for poor responders and POSEIDON (Patient-Oriented Strategies Encompassing Individualized Oocyte Number Criteria) [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e]. Also, they demonstrate the same range of essential KPI levels that were proposed in the Vienna and the Maribor consensus.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThus, our DNN model provides the ability to determine a theoretically justified actual potential of pregnancy occurrence probability in a specific patient group by using KPIs data from individual protocol. Above all else, application of the model as a quality control tool is valid in terms of accurate determination of competency level threshold values in pregnancy rate for individual clinics with their unique patient subpopulation, allowing to identify specific time frames for audits, and areas of increased interest during quality control assessments.\u003c/p\u003e"},{"header":"Materials and methods","content":"\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\n \u003ch2\u003eData collection\u003c/h2\u003e\n \u003cp\u003eWe used retrospective data with known outcomes in \u0026ldquo;IVF and Genetic Center\u0026rdquo;, Moscow, Russia from SQL database in the medical informative system Medwork 3.5 from January 2013 till January 2024 to develop a data set for DNN model: totally 8732 protocols (3856 with fresh embryo transfer). Additional data from 2018\u0026ndash;2024 (3500 protocols) and for PGT-A embryos (1600 protocols) was used for model internal validation. For external model validation we used data from 2 independent ART centers: \u0026ldquo;#1\u0026rdquo; Moscow, Russia with 6240 protocols and \u0026ldquo;#2\u0026rdquo; Tbilisi, Georgia with 3888 protocols. All the protocols with missing data values were discarded from the study.\u003c/p\u003e\n \u003cp\u003ePatient informed consent for that study was not necessary because only retrospective and fully de-identified data from embryo development has been used, fully noninvasive for patients or their embryos (no medical intervention was performed on the subject, and no biological samples from the patient were collected to develop that model). For embryo evaluation, the ESHRE recommendations and Gardner blastocyst grading system was used, in which \u0026ldquo;good blastocysts\u0026rdquo; were identified as Bl3BB and higher grade.\u003c/p\u003e\n \u003cp\u003eOur DNN model incorporates KPI data on the base of the Vienna Consensus, showing a correlational dependence with occurrence of pregnancy. We also use fine-tuning KPI for total good blastocyst development rate (TGBDR) in our model according to publications [\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e]. TGBDR has positive correlation with all itralaboratory KPIs and negative correlation with patient age in our data. Model also includes the rank sum of laboratory and clinical parameters (KPIScore), used with adaptation to our data (SART embryo grading and antral follicle count on the trigger day) to forecast the chance of pregnancy occurrence [\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e]. The positive correlation (+\u0026thinsp;0.785) for KPIScore with TGBDR has been observed.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\n \u003ch2\u003eStatistical analysis\u003c/h2\u003e\n \u003cp\u003eFor data set description, statistical analysis of individual KPIs was conducted using the StatTech v. 3.0.6 software. Quantitative indicators with a normal distribution were described using the mean and standard deviation (SD), with 95% confidence intervals. Direction and strength of the correlation between two quantitative variables were evaluated using the Spearman rank correlation coefficient. For statistical approach P-value\u0026thinsp;\u0026lt;\u0026thinsp;0.05 was used as a significant threshold limit in analysis of clinical features. Comparison of groups based on quantitative indicators was performed with one-way analysis of variance (ANOVA), with post hoc comparisons using the Kruskal-Wallis test for non-normal distribution.\u003c/p\u003e\n \u003cp\u003eAnalysis was performed with Information Gain, Gini Coefficient, ANOVA, Chi-square (\u0026chi;\u0026sup2;) and ReliefF to evaluate feature importance in classification tasks (Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eEvaluating feature importance to identify the most informative variables for DNN training\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003ccolgroup cols=\"8\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMetric\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eInfo. gain\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGain ratio\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGini\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eANOVA\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u0026chi;\u0026sup2;\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eReliefF\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eKPIScore\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.107\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.054\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.062\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1421.400\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e906.190\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.017\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePatient age\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.085\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.043\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.051\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1045.865\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e747.026\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.016\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTransfer day\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.080\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.051\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.051\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1036.645\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1281.690\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNumber of oocytes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.073\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.045\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e750.572\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e721.893\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAntral follicle count\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.073\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.043\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e648.521\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e629.716\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDay 3 embryos number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.072\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.042\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e614.261\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e617.259\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.004\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTotal Blastocyst number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.068\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.042\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e629.322\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e880.509\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.002\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2pN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.065\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.033\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.038\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e557.655\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e548.234\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.007\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlastocyst rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.064\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.034\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.040\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e699.393\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e731.757\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.004\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNumber of inseminated\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.062\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.031\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e505.209\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e515.078\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.010\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGood Blastocyst number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.062\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.037\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.039\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e475.841\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e934.627\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.003\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTGBDR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.060\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.037\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.038\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e472.284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e800.634\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDay 5 embryos number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.059\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.030\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.036\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e571.130\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e643.703\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNumber of transferred embryos\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.039\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.025\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.025\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e281.349\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e254.111\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.002\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIVF attempt number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.037\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.022\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e313.250\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e515.722\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.008\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNumber of cryo embryos\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.028\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.015\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.018\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e236.567\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e325.789\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.003\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFertilization rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.026\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.013\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.017\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e106.370\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e13.275\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.018\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCleavage rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.022\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.031\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.013\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e476.941\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e49.713\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOocyte retrieval rate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.018\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.011\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.012\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.625\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e28.416\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.002\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\n \u003ch2\u003eTGBDR - total good quality blastocyst development rate\u003c/h2\u003e\n \u003cp\u003ePython 3.6, Scikit-learn 1.4.2 and Sklearn 1.4 were used to implement machine learning models and statistical modeling. A t-test was used to determine whether there was a significant difference between the means of two groups. Mann-Whitney U Test (Wilcoxon Rank Sum Test) was used to compare differences between two groups: model predicted values and real clinical pregnancy rate in different patient groups, staff members and time periods.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\n \u003ch2\u003eModel Compilation\u003c/h2\u003e\n \u003cp\u003eThe neural network model has been developed and executed in the GPU PyCharm 17.0.10 environment using Python 3.6 programming language with the Tensorflow 2.15.0 and Keras library 2.14.0.\u003c/p\u003e\n \u003cp\u003eThe proposed neural network model is a sequential Recurrent Neural Network (RNN) consisting of two layers with L2 and L1 regularization to control overfitting by adding penalties to the model\u0026apos;s weights. The first layer - SimpleRNN with 32 neurons with Rectified Linear Activation (ReLU). The second layer - SimpleRNN \u0026minus;\u0026thinsp;16 neurons and ReLU activation with added Dropout layer to prevent model overfitting by randomly excluding neurons during training. The final Dense layer with one neuron and Sigmoid activation, designed for binary classification. Sigmoid is used to predict the probability of a positive class - a positive outcome of the embryo transfer procedure.\u003c/p\u003e\n \u003cp\u003eThis DNN model was optimized using the gradient descent method with the Adam optimization algorithm with learning rate 0.0001. Binary cross-entropy was used as the loss function. The model was trained on a data array for 12 epochs (independently passes through the entire data frame, splitting it into train (70%), validation (20%) and test (10%) set) with a batch size of 8. Cross-validation for the model was conducted to evaluate it on 5 different data splits (folds).\u003c/p\u003e\n \u003cp\u003eCalibration of the DNN was performed using the CalibratedClassifierCV from Scikit-learn, which applies logistic regression to align probabilities. A comparative analysis of prediction errors was conducted with Area under the receiver operating characteristic curve (AUC), Accuracy, Positive Predictive Value (PPV), Negative Predictive Value (NPV), False Positive Rate (FPR), False Negative Rate (FNG), Specificity, Sensitivity and Matthews Correlation Coefficient (MCC).\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n \u003ch2\u003eExplanation of the Model results\u003c/h2\u003e\n \u003cp\u003eIn the context of approaches to interpret the DNN results, we have utilized SHAP (SHapley Additive exPlanations), ICE (Individual Conditional Expectation) and LIME (Local Interpretable Model-agnostic Explanations) methods.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eCompeting interests\u003c/h2\u003e \u003cp\u003eThe authors declare no conflict of interest and no competing interests.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eS.S. and I.D. designed the study. S.S. conducted studies on the establishment of DNN architecture, implemented machine learning for comparison, and analyzed the data. I.D. interpreted the external validation data and drafted the manuscript. All authors reviewed the manuscript.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003ehttps://github.com/embryossa/DNN.git\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eGlatstein, I., Chavez-Badiola, A., \u0026amp; Curchoe, C. L. New frontiers in embryo selection. Journal of assisted reproduction and genetics. 40, 223\u0026ndash;234. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10815-022-02708-5\u003c/span\u003e\u003cspan address=\"10.1007/s10815-022-02708-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlizadehsani, R. et al. Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991\u0026ndash;2020). \u003cem\u003eAnnals of operations research\u003c/em\u003e, 1\u0026ndash;42; \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10479-021-04006-2\u003c/span\u003e\u003cspan address=\"10.1007/s10479-021-04006-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFernandez, E. et al. Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classification of reproductive data. Journal of assisted reproduction and genetics. 37, 2359\u0026ndash;2376. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10815-020-01881-9\u003c/span\u003e\u003cspan address=\"10.1007/s10815-020-01881-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine. The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reproductive biomedicine online. 35, 494\u0026ndash;510. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.rbmo.2017.06.015\u003c/span\u003e\u003cspan address=\"10.1016/j.rbmo.2017.06.015\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMedenica, S. et al. The Future Is Coming: Artificial Intelligence in the Treatment of Infertility Could Improve Assisted Reproduction Outcomes-The Value of Regulatory Frameworks. Diagnostics. 12, 2979. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/diagnostics12122979\u003c/span\u003e\u003cspan address=\"10.3390/diagnostics12122979\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUyar, A., Bener, A., \u0026amp; Ciray, H. N. Predictive Modeling of Implantation Outcome in an In Vitro Fertilization Setting: An Application of Machine Learning Methods. Medical decision making: an international journal of the Society for Medical Decision Making. 35, 714\u0026ndash;725. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/0272989X14535984\u003c/span\u003e\u003cspan address=\"10.1177/0272989X14535984\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBamford, T. et al. A comparison of morphokinetic models and morphological selection for prioritizing euploid embryos: a multicentre cohort study. Human reproduction. 39, 53\u0026ndash;61. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dead237\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dead237\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlais, I., Koifman, M., Feferkorn, I., Dirnfeld, M., \u0026amp; Lahav-Baratz, S. Improving embryo selection by the development of a laboratory-adapted time-lapse model. F\u0026amp;S science. 2, 176\u0026ndash;197. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.xfss.2021.02.001\u003c/span\u003e\u003cspan address=\"10.1016/j.xfss.2021.02.001\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXi, Q. et al. Individualized embryo selection strategy developed by stacking machine learning model for better in vitro fertilization outcomes: an application study. Reproductive biology and endocrinology. 19, 53. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s12958-021-00734-z\u003c/span\u003e\u003cspan address=\"10.1186/s12958-021-00734-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRatna, M. B., Bhattacharya, S., Abdulrahim, B., \u0026amp; McLernon, D. J. A systematic review of the quality of clinical prediction models in in vitro fertilisation. Human reproduction. 35, 100\u0026ndash;116. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dez258\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dez258\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZac\u0026agrave;, C. et al. Fine-tuning IVF laboratory key performance indicators of the Vienna consensus according to female age. Journal of assisted reproduction and genetics. 39, 945\u0026ndash;952. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10815-022-02468-2\u003c/span\u003e\u003cspan address=\"10.1007/s10815-022-02468-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFranco, J. G. et al. Key performance indicators score (KPIs-score) based on clinical and laboratorial parameters can establish benchmarks for internal quality control in an ART program. JBRA assisted reproduction. 21, 61\u0026ndash;66. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.5935/1518-0557.20170016\u003c/span\u003e\u003cspan address=\"10.5935/1518-0557.20170016\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKaufmann, S. J., Eastaugh, J. L., Snowden, S., Smye, S. W., \u0026amp; Sharma, V. The application of neural networks in predicting the outcome of in-vitro fertilization. Human reproduction. 12, 1454\u0026ndash;1457. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/12.7.1454\u003c/span\u003e\u003cspan address=\"10.1093/humrep/12.7.1454\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (1997).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu, H. et al. Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study. eLife. 12, e83662. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.7554/eLife.83662\u003c/span\u003e\u003cspan address=\"10.7554/eLife.83662\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlank, C. et al. Prediction of implantation after blastocyst transfer in in vitro fertilization: a machine-learning perspective. Fertility and sterility. 111, 318\u0026ndash;326. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.fertnstert.2018.10.030\u003c/span\u003e\u003cspan address=\"10.1016/j.fertnstert.2018.10.030\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaef, B., Maleki, M., \u0026amp; Ferdousi, R. Computational prediction of implantation outcome after embryo transfer. Health informatics journal. 26, 1810\u0026ndash;1826. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/1460458219892138\u003c/span\u003e\u003cspan address=\"10.1177/1460458219892138\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBormann, C. L. et al. Performance of a deep learning based neural network in the selection of human blastocysts for implantation. \u003cem\u003eeLife\u003c/em\u003e. 9, e55301. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.7554/eLife.55301\u003c/span\u003e\u003cspan address=\"10.7554/eLife.55301\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L., Cui, X., Yang, J., Wu, X., \u0026amp; Zhao, G. Using feature optimization and LightGBM algorithm to predict the clinical pregnancy outcomes after in vitro fertilization. Frontiers in endocrinology. 14, 1305473. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fendo.2023.1305473\u003c/span\u003e\u003cspan address=\"10.3389/fendo.2023.1305473\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSarais, V. et al. Predicting the success of IVF: external validation of the van Loendersloot's model. Human reproduction. 31, 1245\u0026ndash;1252. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dew069\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dew069\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChamayou, S. et al. The use of morphokinetic parameters to select all embryos with full capacity to implant. Journal of assisted reproduction and genetics. 30, 703\u0026ndash;710. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10815-013-9992-2\u003c/span\u003e\u003cspan address=\"10.1007/s10815-013-9992-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2013).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBasile, N. et al. The use of morphokinetics as a predictor of implantation: a multicentric study to define and validate an algorithm for embryo selection. Human reproduction. 30, 276\u0026ndash;283. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/deu331\u003c/span\u003e\u003cspan address=\"10.1093/humrep/deu331\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDal Canto, M. et al. Faster fertilization and cleavage kinetics reflect competence to achieve a live birth after intracytoplasmic sperm injection, but this association fades with maternal age. Fertility and sterility. 115, 665\u0026ndash;672. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.fertnstert.2020.06.023\u003c/span\u003e\u003cspan address=\"10.1016/j.fertnstert.2020.06.023\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBori, L. et al. The higher the score, the better the clinical outcome: retrospective evaluation of automatic embryo grading as a support tool for embryo selection in IVF laboratories. Human reproduction. 37, 1148\u0026ndash;1160. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/deac066\u003c/span\u003e\u003cspan address=\"10.1093/humrep/deac066\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBenchaib, M., Labrune, E., Giscard d'Estaing, S., Salle, B., \u0026amp; Lornage, J. Shallow artificial networks with morphokinetic time-lapse parameters coupled to ART data allow to predict live birth. Reproductive medicine and biology. 21, e12486. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rmb2.12486\u003c/span\u003e\u003cspan address=\"10.1002/rmb2.12486\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKim, H. M. et al. Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images. Scientific reports. 14, 3240. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-024-52241-x\u003c/span\u003e\u003cspan address=\"10.1038/s41598-024-52241-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFr\u0026eacute;our, T. et al. External validation of a time-lapse prediction model. Fertility and sterility. 103, 917\u0026ndash;922. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.fertnstert.2014.12.111\u003c/span\u003e\u003cspan address=\"10.1016/j.fertnstert.2014.12.111\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTran, D., Cooke, S., Illingworth, P. J., \u0026amp; Gardner, D. K. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Human reproduction. 34, 1011\u0026ndash;1018. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dez064\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dez064\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReignier, A. et al. Performance of Day 5 KIDScore\u0026trade; morphokinetic prediction models of implantation and live birth after single blastocyst transfer. Journal of assisted reproduction and genetics. 36, 2279\u0026ndash;2285. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10815-019-01567-x\u003c/span\u003e\u003cspan address=\"10.1007/s10815-019-01567-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBerntsen, J., Rimestad, J., Lassen, J. T., Tran, D., \u0026amp; Kragh, M. F. Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences. PloS one. 17, e0262661. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1371/journal.pone.0262661\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0262661\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee, C. I. et al. Associations between the artificial intelligence scoring system and live birth outcomes in preimplantation genetic testing for aneuploidy cycles. \u003cem\u003eReproductive biology and endocrinology.\u003c/em\u003e 22, 12. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s12958-024-01185-y\u003c/span\u003e\u003cspan address=\"10.1186/s12958-024-01185-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVerMilyea, M. et al. Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF. Human reproduction. 35, 770\u0026ndash;784. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/deaa013\u003c/span\u003e\u003cspan address=\"10.1093/humrep/deaa013\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEnatsu, N. et al. A novel system based on artificial intelligence for predicting blastocyst viability and visualizing the explanation. Reproductive medicine and biology. 21, e12443. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rmb2.12443\u003c/span\u003e\u003cspan address=\"10.1002/rmb2.12443\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNelson, S. M., \u0026amp; Lawlor, D. A. Predicting live birth, preterm delivery, and low birth weight in infants born from in vitro fertilisation: a prospective study of 144,018 treatment cycles. PLoS medicine. 8, e1000386. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1371/journal.pmed.1000386\u003c/span\u003e\u003cspan address=\"10.1371/journal.pmed.1000386\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRatna, M. B., Bhattacharya, S., \u0026amp; McLernon, D. J. External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment. Human reproduction. 38, 1998\u0026ndash;2010. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dead165\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dead165\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUeno, S., Berntsen, J., Okimura, T., \u0026amp; Kato, K. Improved pregnancy prediction performance in an updated deep-learning embryo selection model: a retrospective independent validation study. Reproductive biomedicine online. 48, 103308. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.rbmo.2023.103308\u003c/span\u003e\u003cspan address=\"10.1016/j.rbmo.2023.103308\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTzukerman, N. et al. Using Unlabeled Information of Embryo Siblings from the Same Cohort Cycle to Enhance In Vitro Fertilization Implantation Prediction. \u003cem\u003eAdvanced science.\u003c/em\u003e 10, e2207711. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/advs.202207711\u003c/span\u003e\u003cspan address=\"10.1002/advs.202207711\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDiakiw, S. M. et al. An artificial intelligence model correlated with morphological and genetic features of blastocyst quality improves ranking of viable embryos. Reproductive biomedicine online. 45, 1105\u0026ndash;1117. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.rbmo.2022.07.018\u003c/span\u003e\u003cspan address=\"10.1016/j.rbmo.2022.07.018\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDuval, A. et al. A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems. Human reproduction. 38, 596\u0026ndash;608. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/dead023\u003c/span\u003e\u003cspan address=\"10.1093/humrep/dead023\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eErlich, I. et al. Pseudo contrastive labeling for predicting IVF embryo developmental potential. Scientific reports. 12, 2488. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-022-06336-y\u003c/span\u003e\u003cspan address=\"10.1038/s41598-022-06336-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoewke, K. et al. Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos. Fertility and sterility. 117, 528\u0026ndash;535. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.fertnstert.2021.11.022\u003c/span\u003e\u003cspan address=\"10.1016/j.fertnstert.2021.11.022\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCho, J.H. et al. Large-scale simulation of pregnancy rate improvements using an AI model for embryo ranking. \u003cem\u003e38th Hybrid Annual Meeting of the ESHRE\u003c/em\u003e (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSayed, S. et al. Time-lapse imaging derived morphokinetic variables reveal association with implantation and live birth following in vitro fertilization: A retrospective study using data from transferred human embryos. PloS one. 15, e0242377. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1371/journal.pone.0242377\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0242377\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiao, Q. et al. Development of deep learning algorithms for predicting blastocyst formation and quality by time-lapse monitoring. Communications biology. 4, 415. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s42003-021-0193Cho7-1\u003c/span\u003e\u003cspan address=\"10.1038/s42003-021-0193Cho7-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFerraretti, A. P. et al. ESHRE consensus on the definition of 'poor response' to ovarian stimulation for in vitro fertilization: the Bologna criteria. Human reproduction. 26, 1616\u0026ndash;1624. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/humrep/der092\u003c/span\u003e\u003cspan address=\"10.1093/humrep/der092\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEsteves, S. C. et al. The POSEIDON Criteria and Its Measure of Success Through the Eyes of Clinicians and Embryologists. Frontiers in endocrinology. 10, 814. DOI: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fendo.2019.00814\u003c/span\u003e\u003cspan address=\"10.3389/fendo.2019.00814\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2019).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Deep neural networks, IVF laboratory data analysis, Clinical pregnancy prediction, Personalized treatment, ART quality management","lastPublishedDoi":"10.21203/rs.3.rs-4445375/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4445375/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe utilization of neural networks in assisted reproductive technology is essential due to their capability to process complex and multidimensional data inherent in IVF procedures, offering opportunities for clinical outcome prediction, personalized treatment implementation, and overall advancement in fertility treatment.\u003c/p\u003e \u003cp\u003eThe aim of this study was to develop a novel approach to IVF laboratory data analysis, employing deep neural networks to predict the likelihood of clinical pregnancy occurrence within an individual protocol, integrating both key performance indicators and clinical data.\u003c/p\u003e \u003cp\u003eWe conducted a retrospective analysis spanning 11 years, encompassing 8732 protocols, to extract the most relevant features to our goal and train the model. Internal validation was performed on 1600 preimplantation genetic testing for aneuploidy embryo transfers, while external was conducted across two independent clinics (over 10,000 cases).\u003c/p\u003e \u003cp\u003eLeveraging recurrent neural networks, our model demonstrates high accuracy in predicting the likelihood of clinical pregnancy within specific IVF protocols (AUC: 0.68\u0026ndash;0.86; Test accuracy: 0.78, F1 Score: 0.71, Sensitivity: 0.62; Specificity: 0.86) comparable to time-lapse system but with a simpler approach. Our model facilitates both retrospective analysis of outcomes and prospective evaluation of clinical pregnancy chances, thus presenting a promising avenue for quality management programs and promotes their realization in medical centers.\u003c/p\u003e","manuscriptTitle":"Advanced KPI Framework for IVF Pregnancy Prediction Models in IVF protocols","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-04 17:31:19","doi":"10.21203/rs.3.rs-4445375/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-07-05T11:18:39+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-22T15:37:32+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-11T19:42:29+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"327668428418573956650125195145480317146","date":"2024-05-27T17:19:56+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"60866700210235399441931950757400852938","date":"2024-05-22T06:44:05+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-05-22T06:06:01+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-22T06:01:47+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-05-22T05:48:18+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-22T05:43:15+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2024-05-19T18:26:27+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"867826e4-4299-4a3b-a38a-4d72fd3bfe11","owner":[],"postedDate":"June 4th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":32686725,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":32686726,"name":"Health sciences/Health care"},{"id":32686727,"name":"Health sciences/Medical research"}],"tags":[],"updatedAt":"2024-12-02T16:01:25+00:00","versionOfRecord":{"articleIdentity":"rs-4445375","link":"https://doi.org/10.1038/s41598-024-80759-7","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2024-11-27 15:57:26","publishedOnDateReadable":"November 27th, 2024"},"versionCreatedAt":"2024-06-04 17:31:19","video":"","vorDoi":"10.1038/s41598-024-80759-7","vorDoiUrl":"https://doi.org/10.1038/s41598-024-80759-7","workflowStages":[]},"version":"v1","identity":"rs-4445375","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4445375","identity":"rs-4445375","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}