Embryo selection through artificial intelligence versus embryologists: a systematic review.

Infertility, defined as the absence of conception after 1 year of unprotected intercourse ( WHO, 2022 ), affects one to six couples trying to conceive during their life time ( Njagi et al. , 2023 ), causing considerable emotional and financial distress ( Gnoth et al. , 2003 ; Herbert et al. , 2009 ; Inhorn and Patrizio, 2015 ). Annually, the estimated number of couples experiencing infertility worldwide is around 17.5% of the adult population (one in six) ( Njagi et al. , 2023 ). As part of the treatment process, IVF requires the fertilization of multiple oocytes. Each oocyte is inseminated simultaneously under the same conditions and, if successful, eventually develops into an embryo that can be transferred into the patient’s uterus. Though touted as a cure for infertility, the success rate of ART procedures does not exceed 30% ( Kanakasabapathy et al. , 2019 ). Indeed, for women above the age of 42 years, this rate can be <4% ( Smith et al. , 2015 ). This low efficiency of ART success rate has been partly attributed to embryo selection assessments ( Simon and Laufer, 2012 ; Bashiri et al. , 2018 ; Fishel et al. , 2020 ), as the majority of transferred embryos fail to implant in the uterus—a phenomenon which increases with female age ( Bashiri et al. , 2018 ). Under current practice, embryo selection involves a visual analysis by embryologists to select the most viable embryo for implantation, while discarding poor-quality and slow-growing embryos. Success is therefore dependent on the embryologist’s analysis of each embryo, which is prone to human error and variability due to the intrinsic subjectivity of the process ( Baxter Bendus et al. , 2006 ; Choucair et al. , 2021 ; Klimczak et al. , 2021 ; Geampana and Perrotta, 2023 ). Considering the high probability of treatment failure, patients often undergo five or more ART cycles to achieve a successful live birth ( Smith et al. , 2015 ), leading to a high economic and emotional cost ( Wu et al. , 2013 ; Drazba et al. , 2014 ; Mounce et al. , 2022 ). As such, there is a clinical need for a more reliable methodology of embryo assessment that can increase the likelihood of successful healthy live birth for couples undergoing ART treatments. Recently, there has been a growing interest in applying artificial intelligence (AI) in ART, particularly during embryo selection ( Kragh and Karstoft, 2021 ), treatment planning ( Chow et al. , 2021 ), quality assurance in ART laboratories ( Fernandez et al. , 2020 ), and sperm selection in ICSI ( Mendizabal-Ruiz et al. , 2022 ). Indeed, AI in healthcare is a developing field that has demonstrated considerable promise, with successful applications in radiology ( Hirsch et al. , 2022 ), genomics, and pharmaceuticals ( Dias and Torkamani, 2019 ; Savage, 2021 ). In the case of embryo selection, AI could reduce the subjectivity associated with the embryo selection process by providing an accurate and reliable interpretation of embryo development status, embryo morphology, and potentially predicting embryo viability ( VerMilyea et al. , 2020 ). However, owing to the niche technology introduced by AI, there are no current standards and expectations for its performance. Therefore, there are many questions regarding the methodology for comparing vital performance metrics, especially regarding generalizability and precision ( Alipour and Harris, 2020 ). This is currently due to basic AI requirements, such as data quality and variety and large sample size, which can be resolved by the utilization of discarded embryos ( de Hond et al. , 2022 ). Furthermore, the introduction of AI technology in IVF is in its infancy, including commercial AI systems already in clinical use where most of these factors are not being considered in a standardized manner. Therefore, this will likely result in different performances, particularly when models are compared against embryologists or in new sets of data. This has influenced the diffusion of AI in the IVF field in development phase without any prospective verification against embryologists using real-world data. It is also essential to analyse the inter- and intra-cohort embryo-assessment to precisely accommodate the application of the AI-model. Although unbiased samples are useful for the broader population, better predictions specific within the local culture can be more beneficial ( Sundvall et al. , 2013 ). Considering the current state of knowledge of AI in the setting of ART and the early stages of clinical implementation, this systematic review aims to provide an overview of AI application and AI techniques in the field of embryo selection, identifying the performance of these systems against the current embryologists’ assessments for both embryo morphology and reproductive outcome predictions. In doing so, we hope to raise awareness of the current landscape and discuss possible directions to inform AI developers, IVF clinicians, embryologists, and patients alike of the clinical and technical considerations that can facilitate clinical uptake.

A total of 20 studies were included in this review. The article selection process is outlined in Fig. 1 . The characteristics of the models included in this review are reported in Table 1 . All the articles included in this review compared the performance of the AI against an embryologist or clinician. Each study had developed its own customized database from local patient populations making the ground truth subjective to the respective study. Each database had a different sample size, ranging from 224 samples to 14 144 samples. There was no specific embryo assessment day across the studies—Day 1 until Day 5/6 of embryo development was investigated. In addition, any reference to patient clinical data (e.g. age, BMI, etc.) has been referred to as clinical information throughout the review. PRISMA flow diagram of study selection for a systematic review of embryo selection through artificial intelligence versus embryologists. The literature search included studies published since 2005. AI, artificial intelligence. Characteristics of studies included in a systematic review of embryo selection by using artificial intelligence or embryologists. Day of assessment: day of embryo development when images or time-lapse videos were used to train artificial intelligence (AI) models; image segmentation: image processing techniques used to facilitate training of AI models; time-lapse: images captured over a period (usually days); clinical records: patient information used to train AI models; AI technologies: machine learning techniques and/or pre-trained AI models employed by included study; ResNet, residual neural network; DenseNet, dense neural network; LSTM, long short term memory; ICM, inner cell mass; TE, Trophectoderm. Accuracies were reported by studies in the form of a number between zero and one (Percentage), the numbers were converted to a whole number for tabulation purposes. No confidence intervals or P -values were reported. Note: Any grade that contained B− or C and an extension rate equal to or less than three was considered part of the poor-quality group. In addition, any score with two A or A− grades, or one A with B, with an extension of three or greater could be labelled as good-quality. Some scores (e.g. 3BB, 3BA−) were debatable thus putting them in a separate category (fair-quality). Only three studies out of 20 scored greater than 7 out of 10 points on the Down and Black bias assessment tool, primarily due to the method of acquiring samples representing the population of their local country, which would signify a lack of generalizability if the AI were to be applied in another region or ethnic group ( Supplementary Fig. S1 ). Despite so few studies scoring >70%, all studies were included in the analyses of this review. This is expected since this checklist was created with the intent of analysing clinical studies rather than preclinical AI validation studies. However, given the generalizability of this tool, it still provides a reasonable assessment of bias for the included studies. In terms of the various networks used by each study, Table 1 shows the methods decided by the purpose and application of each study. For embryo images, variants of image-based convolutional neural networks (CNN) were utilized. CNNs use multiple layers of small filters passed over the entire image to identify features and make predictions. The variants used were: ResNet (Residual Network), which includes skip connections which allows the network to converge more quickly, while stabilizing the training ( He et al. , 2016 ); DenseNet (Dense Network), which has extra connections to improve the network’s efficiency ( Huang et al. , 2017 ); LSTM (Long Short Term Memory); and a time series model which can ‘remember’ information ( Staudemeyer and Morris, 2019 ). On the other hand, clinical information studies relied on vectorization and linear models: support vector machine, which linearly separates samples in groups for prediction ( Hearst et al. , 1998 ); random forest and decision trees, a branching tree structure that uses data features to make a binary decision at each branch until it reaches an output ( Bernard et al. , 2009 ), multilayer perceptron, which unlike the previously described clinical models is a neural network, and can have multiple layers of fully connected neurons ( Murtagh, 1991 ). Sensitivity and specificity were reported only in three out of 20 studies ( Uyar et al. , 2015 ; Chavez-Badiola et al. , 2020 ; VerMilyea et al. , 2020 ), while accuracy was the main metric reported in all studies. Figure 2 shows the accuracy reported by the studies, categorized by the type of input. The images category presented models which used images and time-lapses of embryos as inputs, typically resulting in an output of embryo morphological grade; clinical information represented patient information (often demographic data and clinical history) as an input to the models, with mainly clinical pregnancy prediction as the output. It is important to note that the accuracies reported for images only had slightly different outcomes as each AI had a different embryo morphology output (e.g. cell count versus cell features versus viability versus grade). This lack of standardization does not permit the comparison of models with each other. The lack of standardization was also inherited from the lack of ground truth. Indeed, the AI models used embryo images assessed following the specific grading guidelines of each clinic involved, while embryologist used the datasets following the same ground truth provided to the respective AI models. Hence, there was no common trend between the databases to create unionized inputs and outputs. Finally, four studies presented a mixture of embryo features combined with clinical information as inputs. Table 2 shows the sample size used by the presented studies. Each study developed its own database for AI validation; however, this was reported by only five out of the 20 reported studies (25%). Accuracy of the AI model defined by the data input utilized for model training. Images input: studies including still embryo images and embryo images from timelapse videos. Clinical information input: studies including patient information features, demographics, and treatment information. Images and clinical information input: studies including the use of both embryo images and clinical information. The graph shows the accuracy output of the prediction defined by each study’s input sample type, such as embryo morphology prediction. Embryo grade though images and their quality assessments; clinical pregnancy prediction: prediction of possible successful clinical pregnancy. Embryo aneuploidy prediction: embryo aneuploidy prediction through embryo images. Live birth prediction: prediction of possible successful healthy live birth delivery. AI, artificial intelligence. Sample size of each study in a systematic review of embryo selection methods, including training, internal validation, and test set. AI, artificial intelligence. Ten out of 20 studies (50%) performed the classification with a solely image-based input system ( Rad et al. , 2018 , 2019 ; Kragh et al. , 2019 ; Chavez-Badiola et al. , 2020 ; VerMilyea et al. , 2020 ; Bormann et al. , 2020a , 2021 ; Coticchio et al. , 2021 ; Liao et al. , 2021 ; Wu et al. , 2021 ). These studies used a CNN as a backbone to create predictions. Only one out of the 10 studies (10%) used time-lapse as an input ( Kragh et al. , 2019 ), by further employing a recurrent neural network. Rad et al. (2018) produced an accuracy of 93.8% on cell counting of embryo images, whilst Coticchio et al. (2021) scored 82.6% accuracy in applying a binary classifier to determine the ability of an embryo to reach the blastocyst stage. Six out of 20 studies (30%) focused on the use of clinical records (both patient and embryo-related clinical information) as an input to their models for predicting successful pregnancy ( Uyar et al. , 2009 , 2015 ; Blank et al. , 2019 ; Raef et al. , 2020 ; Bormann et al. , 2020b ; Hariton et al. , 2021 ) except for ( Bormann et al. , 2020b ) where embryo morphology was the output of the model. Table 3 shows the clinical features used by each study as inputs to their respective models, where the female age can be seen as the common feature across all models and the embryo development stage the common embryo morphology feature. Bormann et al. (2020b ) used a CNN approach, whilst the other five studies tested algorithms using statistical models. Uyar et al. (2015) achieved an accuracy of 80.4% in classifying implantation outcomes. Reported clinical characteristics included as features in the AI models. AI, artificial intelligence; RIF, repeat implant failure; PESE, percutaneous epididymal sperm extraction; TESE, testicular sperm extraction; AFC, antral follicle count; AMH, anti-Müllerian hormone; TSH, thyroid-stimulating hormone; E2, estradiol; vitD3, vitamin D3; ET, embryo transfer; 2PN, 2 pronuclei; ICM, inner cell mass; TE, trophectoderm; MNB, multinucleated blastomeres. (*) indicates the feature with most influence on the performance of the AI model. Four out of 20 studies (20%) utilized a combination of clinical information and embryo images as an input to the AI models ( Khosravi et al. , 2019 ; Patil et al. , 2019 ; Sawada et al. , 2021 ; Loewke et al. , 2022 ). All four studies utilized an open-source AI model (GoogleNet and ResNet— Table 1 ) to deliver accuracies of 97.5% ( Khosravi et al. , 2019 ), 86% ( Patil et al. , 2019 ), 75% ( Loewke et al. , 2022 ), and 67% ( Sawada et al. , 2021 ). When comparing the accuracies of AI models against embryologists, all the studies reported a better performance in favour of the AI models by at least 4% higher prediction accuracy. The greatest difference can be seen in Khosravi et al. (2019) where the AI outperformed the embryologists by 45% when correlating embryo quality with implantation outcome. Figure 3 shows the performance of embryologists when compared with the AI, when both are exposed to blind tests of data. The overall median accuracy of AI models was 74% (range 64–98%, IQR = 68.5–83.5) compared to embryologists' accuracy of 64% (range 53–76%, IQR = 55.5–69.5) ( Uyar et al. , 2015 ; Khosravi et al. , 2019 ; VerMilyea et al. , 2020 ). AI model accuracy compared to prediction accuracy of embryologists according to type of data input. Blue: AI model accuracies among studies. Red: embryologists accuracies among studies. Images input: studies including still embryo images and embryo images from timelapse videos. Clinical information input: studies including patient information features, demographics, and treatment information. Images and clinical information input: studies including the use of both embryo images and clinical information. AI, artificial intelligence. The reported studies in this review fall under three categories, which are dependent on the type of input used: images, clinical information, and combined (images and clinical information). The median accuracy and IQRs are reported in Fig. 4 , where the integration of both images and clinical information had the highest median accuracy (81.5%, IQR = 69–95%) compared to the embryologists (51%, IQR = 49–53%). Furthermore, AI models had an overall higher median accuracy; for image samples as an input, AI produced median accuracy of 75.5%, IQR = 59–94%, while embryologists scored 65.2%, IQR = 64–75%. A similar trend can be seen in clinical information as an input, where AI models produced median accuracy as an output of 77.8%, IQR = 68–90% while embryologists predictions output were 64%, IQR = 58–76%. As noted, all AI models also presented higher IQR for all type of predictions. Median accuracy and interquartile range for AI models compared to embryologists’ predictions. Images input: studies including still embryo images and embryo images from timelapse videos. Clinical information input: studies including patient information features, demographics, and treatment information. Images and clinical information input: studies including the use of both embryo images and clinical information. Sample size (n) refers to number of studies on each analysis. AI, artificial intelligence.

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( Page et al. , 2021 ). The study protocol and review were registered with the International Prospective Register of Systematic Reviews (PROSPERO) database (CRD42021256333). A systematic search was undertaken independently by two review authors (R.W. and M.S.) across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. This date restriction was selected to ensure that only relevant technologies that reflect the present AI landscape were identified and to prevent outdated models that employed principles from the early days of AI from adding confounding factors that could influence the study conclusions. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: (‘Artificial intelligence’ OR ‘Machine Learning’ OR ‘Deep learning’ OR ‘Neural network’) AND (‘IVF’ OR ‘ in vitro fertili*’ OR ‘assisted reproductive techn*’ OR ‘embryo’), where the character ‘*’ refers the search engine to include any auto completion of the search term. The inclusion criteria involved papers that had an application of AI in the field of embryo selection and were compared to at least one embryologist (user input); time-lapse or image segmentation technologies. Any other form of studies or procedure beside embryo selection were excluded. Articles from reference lists of studies being screened were also considered suitable for assessment. Additionally, articles focused on other fields of ART besides embryo selection (e.g. sperm and oocyte selection, controlled ovarian stimulation), did not contrast AI performance against an embryologist, focused on ultrasound, oncology, or non-human models, or focused on signal processing were excluded. Articles were independently screened by M.S., M.M., and R.W. for suitability of inclusion and a list of eligible articles was created using Covidence ® , a tool used primarily for screening and data extraction for Cochrane authors conducting standard intervention reviews efficiently by uploading search results, and screening abstracts and full-text study reports ( VH Innovation, 2022 ). Following this, the full-text articles were read thoroughly by M.S., C.T., M.M., R.W., and F.H. Inclusion of an article for the review was based on consensus between these authors. Studies that did not relate to AI techniques with IVF were excluded. Quality assessment was independently performed by two authors (M.M. and M.S.) using the Modified Down and Black checklist ( Aubut et al. , 2013 ) ( Supplementary Fig. S1 ). A third assessor, C.A., was involved in the event where a consensus could not be reached. This checklist provides scores for the quality of reporting, internal validity, and external validity, and can be applied to a broad range of study designs including non-traditional designs such as AI validation studies ( Adom et al. , 2017 ). Data were extracted manually through the full-text review for analysis by three independent authors (M.S., C.T., and M.M.). The summary measures were presented in tabular form in this systematic review. Importance was given to the AI techniques implemented in the studies as well as their mode of classification for clinical interpretation (images/time-lapse videos, clinical data, or both) and performance (accuracy, sensitivity, and specificity) ( Hossin and Sulaiman, 2015 ). For all machine learning techniques investigated, the medians and interquartile ranges (IQR) of accuracies were calculated to determine a summary accuracy between methodologies. Similarly, accuracies determined by clinical embryologists were also compared through medians and IQR. All other variables were expressed as a percentage. Owing to study heterogeneity, variations in AI techniques (in terms of inputs, output, the architecture of the AI, and the application of the AI), number of embryologists tested against for external validation, and accuracy as a performance metric, the performance of a meta-analysis was not considered appropriate.

The current findings on the application of AI for embryo selection indicates that AI models can outperform embryologists in terms of embryo morphology assessments as well as reproductive outcome predictions. However, it is important to highlight that the current accuracies reported by studies present different performances owing to the diversity of methods employed, sample size as well as the dataset used for AI training and AI validation. Thus, it is critical to establish the minimum AI models features required for clinical implementation as well as the AI benchmark performance and correct datasets for validation in the IVF field. The best performing prediction models for clinical outcomes combined clinical information and images, indicating that the use of clinical data could improve the performance of models that only rely on the use of images or time-lapse videos. Future studies should focus on the prediction of live birth, which is clinically the most relevant outcome of ART.

This systematic review discusses the present landscape of AI application studies for embryo selection. To do this, we have analysed the current literature to develop a comparison of AI performance against embryologists, as the key clinicians involved in embryo selection procedures. The current results show a higher accuracy of AI models against embryologists; however, there is a strong variability in the performance, indicating a necessity to establish what are the correct AI model features required for clinical implementation, as well as the minimum AI benchmark performance and correct datasets for validation in the IVF field. In comparison, when blind test samples are presented to both AI and embryologists, there was a consistency in AI outperforming the embryologists, whether data were embryo images or clinical information. The results also showed that the choice of AI is essential for the application performance. Moreover, the size of the sample base plays a key role in the number of layers of the AI and their size, which significantly varies among included studies. This indicates that there is no commonality between the various models in the current literature as well as current AI models used for clinical applications. The ability of computer vision, and more specifically CNNs, to study an image pixel by pixel and recognize contextual patterns makes it highly useful in this application ( Yamashita et al. , 2018 ). This enables the model to perform sufficiently in a clinical setting; however, it does have the setback of requiring considerable computational power for training ( Willemink et al. , 2020 ). This power requirement is dependent on the size of the training dataset, where a considerable number of samples are required for the outputs of the AI to be generalizable to the broader population. For instance, Chavez-Badiola et al. (2020) developed ERICA, which produced a 79% accuracy compared to the two embryologists involved in the study (71% and 63%). Similarly, VerMilyea et al. (2020) scored a higher accuracy (64%) than the embryologist (47%). Overall, CNN has been the primary choice for the studies included in this review. Rad et al.  (2018 , 2019) , Khosravi et al.  (2019) , VerMilyea et al.  (2020) , Sawada et al.  (2021) and Loewke et al.  (2022) have all employed pre-trained networks (ResNet, DenseNet, and U-Net), which were then further trained on their datasets, reducing the total training time required. Pre-trained networks also provide a validated system as they are already proven to work on a more general dataset with near 100% accuracy ( Dan Hendrycks et al. , 2019 ). The difference can be found in the output layers, which are configured for the different applications, especially for time-lapse samples, where 3D samples can also be provided as an input. In the case of time-lapse data, the samples are still broken down to single images dependent on the point in time under investigation; however, this will give the user the option to define the better image capture to utilize as the input. This reduces the need for the immense computational power necessary to interpret large time-lapse videos while preventing the compromise of its quality; hence the highest performing study focused on time-lapse ( Rad et al. , 2018 , 94%). Clinical information has been evaluated as an input to machine learning models. This is because clinical records can provide considerable amounts of information regarding the treatment and patient condition such as patient phenotype. Here, principal component analysis (PCA) would be more effective than machine learning techniques for considering the essential features to be used in the AI algorithm ( Karamizadeh et al. , 2013 ). This is because the PCA technique finds the common trends within the information and helps to determine the relative importance of the variables based on their influence on the outcome, which is necessary given the multivariable nature of the information contained within clinical records of patients with infertility. Studying clinical records can sometimes introduce various variables that can be deemed insignificant to the AI model ( Lee and Kim, 2021 ). Unlike AI, in clinical settings, the embryologists would also consider risks that are associated with treatments performed by the clinical team ( Kragh and Karstoft, 2021 ). Hence, embryologists tend to make conservative, educated decisions ( Petersen et al. , 2016 ). Unlike images and time-lapse, clinical information lacks the ability to understand embryo quality. Uyar et al. (2015) found that the main two features of interest in clinical information are women’s age and serum FSH level. Whilst such clinical information is presently generalized to the broader population, it is distinct to the patient, signifying the potential for AI to enable a personalized prediction as individual patient information could facilitate tailored outputs according to patients’ clinical history. Although at present, AI models have not well defined the integration of clinical information, women’s age, and FSH are the preferred data points in AI classification of clinical pregnancy, implantation, and live birth ( Goyal et al. , 2020 ). However, other features should not be neglected as there is a discernible trend whereby incorporating more features into the AI models could lead to more confident predictions ( Pettit et al. , 2021 ). For example, Raef et al. (2020) performed the best, achieving 90% accuracy, when presented with the highest number of features as compared to the other studies included in this systematic review. The idea of combining the clinical information and images into one AI model has the potential to provide a tailored approach that is specific to the patient and their needs, for a more personalized treatment. This is a direction that medical technologies are taking towards the future, as each patient has their own traits for which personalized approaches can play an important role ( Thimbleby, 2013 ). The category of combining clinical information and characteristics with embryo images and time-lapse has demonstrated to be a valuable approach in providing the best support by achieving the highest accuracy in the selected studies in this systematic review (81.5%). Although the current studies include limited clinical information integrated into AI models, the incorporation of patients, and IVF treatment information into AI models could lead to the creation of various additional suggestions to prompt embryologists and clinicians to support clinical decisions at the time of embryo transfer. Incorporating this information into AI models could provide insights into the history behind patients’ infertility, which could have implications regarding the prognosis of patients. For instance, patients with poor prognosis could be identified by the number of ART cycles they have attempted as well as the presence of increased reproductive age in the couple, which is known to negatively impact live birth outcomes in ART ( Horta et al. , 2019 ). The addition of different age brackets can help AI models to determine the output produced by patients of different ages and help the model generalize better on the overall sample size ( de Hond et al. , 2022 ). Furthermore, under circumstances of different patient prognosis, such as advanced reproductive age, the traditional embryo selection criteria probably have a lower weight in the final reproductive outcomes, as for the vast majority of the embryos produced by those patients, embryo selection will not lead to an improved outcome ( Sanchez et al. , 2019 ). However, it could be hypothesized that the number of embryos to transfer under some circumstances could increase the chance of achieving pregnancy, despite the known negative implications, particularly when only poor-quality embryos are present ( Kemper et al. , 2021 ). Additional possibilities around embryo transfer include the decision to transfer fresh or frozen embryos, as this is also dependent on clinical information ( Bushaqer et al. , 2020 ). Importantly, although the use of poor-quality embryos is controversial, they could be of significant value for patients with poor prognosis as the transfer of low-grade embryos still has the potential to achieve live birth ( Kemper et al. , 2021 ). Thus, incorporating the patient information into AI models in the future could help to further develop clinical decision support tools tailored to the needs of each patient, primarily based on their prognosis. However, it is important to highlight that these systems will not be able to improve the efficacy of ART treatments in all cases. Nevertheless, information about patient prognosis could be used to counsel patients and potentially explore different reproductive alternatives. All of the included studies have created their own datasets. This can result in datasets that do not adequately represent common practices or the broader population owing to limited access to data from different ethnic groups ( Fremont et al. , 2016 ). It also makes it difficult to compare models as there is no similarity in datasets or intended prediction outcome ( Lee and Lee, 2018 ). However, creating individual datasets can allow them to be relevant to the local population. This is, therefore, a suitable practice to avoid data shifting, where the older samples in the dataset become more negligible compared to the newer data, and provides relevant exposure to the present challenges in embryo features detection and embryo selection ( Racowsky et al. , 2010 ). Hence, these databases provide sufficient samples to justify developing and testing modern machine learning techniques. The type of sample will define the appropriate model to be used; relevant patient information will require a small shallow model, while embryo images and time-lapse videos will require a deeper model to extract the useful features from the images. Another idea is combining both inputs (clinical information and embryo images), where the model can make a more confident prediction with the features extracted from the images and relate the features to the clinical patient information; therefore, a deeper model comprising both inputs combined will be preferred. Another point that must be raised is that the method of quality assessment of the embryo was dependent on the researcher rather than utilizing a standardized grading system for the development of the AI as an input and output ( ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, 2017 ; Puga-Torres et al. , 2017 ). Therefore, without a consensus for which grading system to use, the clinical relevance of these models is restricted to clinics that are willing to adopt a new grading system or already use the system employed by the AI model. Although the quality evaluation has been correlated to pregnancy outcome ( Diakiw et al. , 2022 ; Park et al. , 2022 ), embryologist-defined quality evaluations do not necessarily correlate to the outcome of ongoing pregnancy ( Zhao et al. , 2018 ; Kirillova et al. , 2020 ). This is particularly relevant as there is a high degree of subjectivity in embryo quality assessment ( Fishel et al. , 2020 ; Bormann et al. , 2020b ; Fordham et al. , 2022 ). A seemingly ‘poor’ embryo can still result in an ongoing pregnancy whilst a ‘good’ embryo may not result in successful implantation ( Wintner et al. , 2017 ). Hence, the requirement of ground truth labelling and the inability to consistently do so in the different studies owing to the lack of standard definitions can limit the reliability and prediction capabilities of the AI model. However, the common use of successful implantation as an outcome measure is also a limiting factor for these AI models as the intended outcome of any IVF cycle is to produce a live birth. Finally, the current clinical practice of discarding ‘poor’ embryos is itself a limiting factor for AI development in the field as these embryos could possess value in training the machine learning algorithm by adding a stronger comparison class of samples ( Simopoulou et al. , 2019 ). Based on the findings of this review, AI has demonstrated the potential to predict embryo quality with high accuracy. However, none of these systems has been proved to provide predictions to the degree necessary for clinical implementation, as the training datasets are skewed by a lack of understanding of the complexity of the clinical challenge. So far, the models have only been tested retrospectively; thus, have not been validated clinically. As such, there is limited evidence that the models will perform at the same accuracy prospectively. Although the present validation is promising and justifies further technical development, more concrete clinical trials are still essential for considering if implementation is feasible. The findings of this review should be interpreted with caution. Firstly, although the review discusses the involvement of embryologists in the validation of the developed technology, the included studies did not show any intent to take the developed technology to a clinical trial stage. The studies mentioned in this review stated that further research is required in applying AI in ART. Secondly, a fair comparison of all the studies is deemed unfeasible as each study tested on their own generated databases without an external verification step through the research centres associated with the team carrying out the study. Hence, the review has only considered the technique of AI used, in addition to the sample size and the performance of the embryologists consulted in each study. Finally, whilst the databases were interpreted correctly across all studies, differences in skill, experience, and training of the developer can influence the accuracy and reliability of the AI model and influence the decision of the embryologist. The lack of consistency between embryo stage and clinical features in the studies further constraints the ability to perform meta-analysis owing to heterogeneity between studies. Despite these limitations, it must be highlighted that the included studies have demonstrated promise and have shown potential application for the integration of AI as a decision-support tool in the field of IVF.

Click here for additional data file.