Utilizing Artificial Intelligence: Machine Learning Algorithms to Develop a Preoperative Endometriosis Prediction Model

This retrospective case-control study was conducted at a tertiary referral center specializing in pelvic pain and minimally invasive gynecologic surgery. Institutional review board approval was obtained (202202409). Subjects who underwent laparoscopic or robot-assisted excision of lesions of the ovary, pelvic viscera, or peritoneal surface were identified using the institution’s Integrated Data Repository. Inclusion criteria were defined as women aged 18 to 55 years who had undergone laparoscopic or robot-assisted surgery between 2011 and 2022 by a single surgeon (NM) who is fellowship-trained in minimally invasive surgery. Exclusion criteria included women who required emergent surgery within 48 hours of presentation ( n = 16), as well as those lacking surgical specimens submitted for pathological analysis ( n = 3) or a documented preoperative pelvic examination ( n = 27) ( Fig. 1 ). Controls, were defined as those without endometriosis on pathologic evaluation of surgically excised specimens. Included subjects were randomized and divided among 4 independent reviewers (DS, CC, ST, MS), and inter-rater reliability was optimized with additional training by the senior author. De-identified data were extracted from medical records and stored in REDCap, a secure, institutionally hosted database with access restricted to approved study personnel. Data points included patient demographics, presenting symptoms, gynecologic/obstetric history, past medical, family, and surgical history, detailed pelvic exam findings, imaging findings, intraoperative findings, and pathology results. Variables encompassed a broad spectrum of clinical data routinely collected across patient encounters. A complete list of variables (209) is outlined in Supplemental Table S1 . Variables with greater than 20% missingness were excluded from analysis, except for 4 critical variables (ovarian/adnexal mass on ultrasound, years since pain began (1–5 years), hours to soak pad/tampon, and pain level (0–10 Likert scale)), as their possible clinical significance warranted inclusion. Five MLAs, including logistic regression, random forest (RF), support vector machine, multilayer perceptron (MLP), and extreme gradient boosting (XGBoost), were used to construct predictive models. All MLAs were developed using Python 3.12.5 (a programming language) and Jupyter Notebook 7.2.1 (an interactive digital notebook to write and run code). 788 subjects were randomly divided into two nonoverlapping subsets—80% for model training ( n = 630) and 20% for model testing ( n = 158). Model training involved identifying patterns within labeled data, while testing assessed how accurately those patterns predicted outcomes in new cases. To address missing values, iterative multivariate imputation was employed to estimate each missing value as a function of other features through iterative regression modeling. There was no difference in missing data between the training and testing sets ( Supplemental Table S2 ). The outcome variable of endometriosis on final pathology was excluded from the imputation process to prevent data leakage and bias. XGBoost, however, was trained without imputation, leveraging its inherent ability to handle missing data. Continuous features were scaled via Z-score normalization to ensure a mean value of 0 and a standard deviation of 1, enabling consistent comparisons between features during the learning process. Categorical variables were one-hot encoded where necessary. Optimal hyperparameters were determined using scikitlearn’s GridSearchCV to iterate through individual algorithm parameters. Internal validation was performed during model training using 5-fold cross-validation. Algorithm-specific hyperparameters can be found in Supplemental Table S3 . Despite the imbalance that existed within our dataset between endometriosis cases and controls, class distribution was not altered via techniques such as propensity score matching. Stratified sampling preserved class distribution across training and testing sets. Preserving the natural prevalence of endometriosis in the patient population studied was intentional to ensure that the models were trained on representative data and dataset authenticity was preserved. To interpret model performance and explain the contributions of individual features within the XGBoost model, Shapley Additive Explanation (SHAP) values were employed [ 23 ]. SHAP values represent the average marginal contribution of a feature in a numerical format, indicating what the model considers to be most important when determining the outcome. Thus, SHAP values are especially useful in determining the importance of clinical variables in complex diagnostic machine learning (ML) models [ 23 , 24 ]. Descriptive statistical analyses were performed (JMP Pro 17 software) to understand variable distributions and aid in model interpretability. Continuous variables are presented as medians with interquartile ranges and were compared using Mann–Whitney U test. Categorical variables are presented as absolute and relative frequencies and analyzed using chi-square or Fisher’s exact test, as appropriate ( Table 1 ). A p-value of less than .05 was considered statistically significant. Adjustments for multiple comparisons were not made due to the exploratory nature of our analyses. ML models were evaluated based on accuracy, precision, sensitivity, specificity, F 1-score, and area under the receiver operator characteristic curve (ROC-AUC).

A total of 788 subjects met inclusion criteria. Pathology confirmed endometriosis in 654 (83%) subjects. The remaining 134 (17%) had peritoneal biopsies performed without evidence of endometriosis (controls). Results of univariate analyses for all 209 variables comparing cases and controls are shown in Supplemental Tables S4 and S5 . Age differed between subjects with and without endometriosis (32 [26, 37] vs 34 [29, 39], p = .003). Our population, based on self-reported race, included 74.6% White ( n = 588), 15.2% Black ( n = 120), 2.7% Asian ( n = 21), 0.13% American Indian or Alaska Native ( n = 1), 0.13% Pacific Islander ( n = 1), and 7.2% of Other ( n = 57) races, with this distribution differing between groups (p = .03). Additionally, 6.73% of subjects self-identified as Hispanic. Obstetric history differed between groups, with a median gravidity of 0 for those with endometriosis, and 1 for those without (0 [0, 2] vs 1 [0, 3], p < .001). Each MLA was trained on all 209 variables to optimize the ROC-AUC, which served as the primary outcome for evaluating model performance in predicting endometriosis ( Table 2 ). XGBoost was utilized in outcome analysis due to its explainability via SHAP values, and other MLAs were used for internal validation of XGBoost performance. RF achieved the highest accuracy (0.848), followed by logistic regression (0.835) and support vector machine (0.835). All three models performed with a sensitivity of 1.00, correctly predicting all cases of endometriosis in our test set. MLP achieved the highest specificity of 0.222. While the RF model outperformed others in several metrics, including accuracy (0.848), F 1-score (0.916), and ROC-AUC (0.764), it ultimately overestimated endometriosis cases in the test set, as evidenced by its low specificity (0.111). Fig. 2 displays the top 20 features from the XGBoost model using SHAP values in order of their impact on the model’s prediction, either positive or negative. Beeswarm plots are explained in more detail in Text S1 and Table S7 of the Supplementary Appendix . In the order of relative contribution to the model, the following are the top 5 features with the greatest positive predictive value and their respective univariate comparisons between subjects with and without endometriosis: emesis (141 [21.56%] vs 10 [7.46%], p < .001), crampy nature of pain (325 [49.69%] vs 38 [28.36%], p < .001), regular periods (429 [65.60%] vs 60 [44.78%], p < .001), severity of dysmenorrhea (0–3 Likert scale) (3 [2, 3] vs 3 [2, 3], p = .02), and rectovaginal exam (RVE) with retrocervical tenderness (126 [19.27%] vs 7 [5.22%], p < .001). Conversely, only 3 of the top 20 features with the greatest impact on the model had a negative predictive value, including higher body-mass index (26.3 [22.50, 32.00] vs 27.0 [22.60, 33.65], p = .09), greater age (32 [26, 37] vs 34 [29, 39], p = .003), and history of sexually transmitted infection (127 [19.39%] vs 37 [27.61%], p = .03).

Endometriosis remains a diagnostic challenge, both due to heterogenous presentation and lack of universally reliable noninvasive testing. To our knowledge, the present study is the first of its kind to consider patient-reported history and physical exam findings to generate an explainable prediction model for endometriosis with high sensitivity and strong accuracy. This study first shows that ML can be used in cases of diagnostic difficulty to link an array of clinical features to outcomes of interest, revealing potentially hidden or complex associations. We found that emesis, crampy pain, regular periods, increased severity of dysmenorrhea, and retrocervical tenderness on physical exam were predictive of endometriosis with strong performance metrics, with the XGBoost model performing with an accuracy of 82%. Future analyses should expand the patient population across providers and healthcare centers to widen the pool of training and validation data in order to increase generalizability. In addition, it will be beneficial to expand the age range to include adolescents and postmenopausal women to allow for the use of the model in frequently underdiagnosed populations. To ensure generalizability of our findings, our next steps involve external validation with data from a partner institution with a distinct patient population that differs demographically from our initial cohort, followed by further validation with data from other partner institutions. This is a necessary step to the development of a preoperative risk stratification score that can be deployed for point-of-care use in a clinical setting.

The diagnosis of endometriosis remains a challenge; patients often present with nonspecific and heterogenous symptoms, leading to protracted workup, multiple lengthy therapeutic trials, referrals, unnecessary procedures, and ultimately delays in diagnosis and effective management [ 5 , 9 , 17 ]. To address this critical gap in women’s health, we trained MLAs according to the gold-standard diagnostic method: pathologic evaluation of surgically excised lesions [ 6 , 25 ]. Our cohort of nearly 800 subjects offers one of the largest datasets in the literature, providing a robust foundation to identify clinical features predictive of endometriosis. Additionally, we focused on the XGBoost model in conjunction with SHAP values to allow for model explainability. Our model found that features most predictive are emesis, crampy nature of pain, regular periods, severe dysmenorrhea, and retrocervical tenderness on RVE. Additionally, it performed exceedingly well with correct prediction of the presence of endometriosis in any pelvic region, including lesions of the ovaries, uterosacral ligaments, peritoneum, intestines, etc., with 96% sensitivity and 82% accuracy. To our knowledge, this is the first study of this magnitude to utilize AI to develop a well-performing prediction model for endometriosis that considers a holistic approach and evaluates nearly all aspects of a patient’s history and exam findings. Producing multiple acceptable performing ML models supports that ML can be effectively utilized in this setting. Our best-performing models achieved 100% sensitivity in detecting endometriosis. This high sensitivity came at the expense of decreased specificity with the MLP model exhibiting the highest specificity among the models at 0.222. The model’s low specificity may result in false positives and overdiagnosis; however, given that endometriosis is commonly underdiagnosed, we have prioritized models performing with high sensitivity to facilitate early referral and intervention. The XGBoost model offers a better balance between sensitivity and specificity in addition to the invaluable advantage of explainability, a component essential for understanding the clinical implications of complex ML models. Race differed between groups via univariate analysis, with White race found to be positively predictive via our XGBoost model. It should be noted that a majority of patients in this cohort were White (588 [74.6%]) and this may have impacted its positive predictive value. Our results are consistent with previous literature, though it is possible this may reflect socioeconomic barriers and inherent referral bias rather than lower prevalence within minority groups [ 26 ]. However, our center is unique in our ability to offer equitable services to uninsured and federally funded populations from a diverse geographic region, minimizing the potential impact of these socioeconomic barriers on our study population and dataset. Sensitivity analyses of our model were performed after the removal of demographic information, without any significant difference in performance metrics (see Supplemental Table S6 ). Strong data on differences in prevalence between racial or ethnic groups are lacking, highlighting the need for model validation in more diverse settings and continued research in this area. In our study, crampy pain was more common in women with endometriosis and ranked as the 2 nd most influential feature in our model. Additionally, association of pain with menses was determined to be an important differentiating feature from similar presenting conditions, with pain unrelated to menses negatively predictive of endometriosis. Pain intensity/severity scores were comparable between those with and without endometriosis, with both groups demonstrating an interquartile range of 7 to 10. This likely reflects the population studied, in which surgical intervention was commonly pursued for pain symptoms. As demonstrated by Ballard et al [ 27 ], characteristics and timing of pain, rather than pain location or severity have shown to be more specific to endometriosis. History of infertility produced one of the greatest mean SHAP values, indicating the greatest influence on the model’s ability to correctly predict the presence of endometriosis. This is in accordance with previous literature, with one study showing a two-fold greater risk of infertility in women with endometriosis [ 2 ]. Furthermore, higher gravidity was negatively predictive of endometriosis. Menstrual characteristics contributed to prediction, with regular periods and average flow as positive predictors. While evidence on cycle regularity in endometriosis is mixed, these features may help distinguish it from conditions like PCOS or adenomyosis [ 1 , 9 , 15 ]. Patient-reported emesis emerged as an unexpected but highly predictive feature of endometriosis. Although emesis is not traditionally linked to the disease, gastrointestinal symptoms are highly prevalent in individuals with endometriosis, with one study demonstrating gastrointestinal symptoms present in over 90% of suspected cases [ 28 ]. As a symptom associated with visceral pain, its presence in the appropriate clinical context can heighten clinical suspicion and justify further evaluation [ 28 , 29 ]. It should again be emphasized that ML models do not consider features independent of others, and each feature is analyzed in conjunction with all other features. A notable contribution to the literature is the integration of physical examination findings, as other proposed prediction models rely solely on patient-reported symptoms via questionnaires [ 15 , 17 , 21 , 22 , 27 , 30 ]. Further, all women in our dataset were evaluated by and operated on by the same specialist, eliminating biases with inter-provider variability. Our model highlighted that retrocervical tenderness on bimanual exam (BME) and RVE, as well as cul-de-sac tenderness on BME, was of great importance, with their presence strongly predictive of endometriosis. Additionally, identification of uterosacral ligament nodularity was found to be predictive. These findings underscore the value of provider training and experience with pelvic exams. Increasing awareness and emphasizing effective BME skills among primary care providers and general gynecologists may improve the early identification of endometriosis and facilitate prompt referrals and effective interventions. Waterfall plots ( Fig. 3 ) visually depict the additive contributions of features to individual predictions, showing how our model has the potential to serve as a clinical decision tool in future studies. For patient A who presents with history of infertility, crampy pelvic pain radiating to the groin, regular periods, and retrocervical and cul-de-sac tenderness on exam was correctly predicted to have endometriosis using our model. In contrast, patient B’s pain unrelated to menses, mild dysmenorrhea, history of sexually transmitted infection, absence of regular periods, and BME without cul-de-sac or retrocervical tenderness contributed to correctly predicting the absence of endometriosis. Future work should focus on external validation in more diverse clinical settings and development of a portable clinical screening tool for real-time use during initial evaluation. Our study has several limitations that warrant consideration. First, our dataset comprises subjects from a tertiary referral center who were evaluated by a physician with advanced training in pelvic pain and endometriosis care. This inherently introduces referral bias, as patients seen in such centers exhibit a high pretest probability of endometriosis. This may limit the generalizability of our model in primary care settings at this stage, particularly with regard to the more nuanced physical exam findings. We envision a focused program to teach specific BME skills to primary care providers to improve early identification of highly predictive exam findings. Furthermore, our definition of cases only relied on pathologic confirmation, which likely excluded some patients with undiagnosed endometriosis who elected expectant or medical management. While this introduces selection bias, it also strengthens the model by enabling it to detect subtle differences in patients strongly suspected of having endometriosis but ultimately lacking pathologic confirmation. Second, our model exhibited low specificity, likely due to our strict definition of controls as subjects lacking endometriosis on pathology. This approach, combined with an imbalanced dataset, likely reduced the model’s ability to correctly identify individuals without endometriosis. The low specificity should be considered in clinical context as implementation of the current model would likely result in overdiagnosis. The retrospective and manual data collection introduced bias and inherent variation. Variation was minimized through randomized patient assignment to the reviewers, a standardized data collection form, and inter-rater reliability analysis. Additionally, most ultrasounds from the earlier years of our study period were performed by radiology, limiting their contribution to the model. We have since transitioned to specialized dynamic ultrasound performed by experienced gynecologic sonographers and physicians; however, these cases represented a limited portion of our cohort and thus had minimal contribution to the model. As a result, our model primarily focused on history and physical exam findings as potential predictors of endometriosis. Finally, we included all patients over a 10-year study period, encompassing patients across a wide age range with varying fertility goals. Thus, many patients were using hormonal suppression or contraception. However, we do not believe this impacted diagnostic accuracy, as all patients underwent thorough peritoneal evaluation with wide excisional biopsies, including subtle lesions, to minimize underdiagnosis and ensure accurate pathologic confirmation.

Endometriosis is a chronic condition characterized by the presence of endometrial-like glands and stroma outside the uterine cavity, leading to a range of nonspecific symptoms including dysmenorrhea, pelvic pain, dyspareunia, and/or infertility [ 1 , 2 ]. Despite affecting up to 10% of women of reproductive age, endometriosis remains notoriously difficult to diagnose due to its heterogeneous clinical presentation [ 3 , 4 ]. This, in conjunction with the need for diagnostic laparoscopy for definitive diagnosis, has led to the true prevalence remaining unknown and likely underestimated [ 5 , 6 ]. The reliance on surgical intervention for definitive diagnosis often results in considerable diagnostic delays up to 7 to 10 years [ 7 ]. There is a notable lack of accurate, appropriately powered, noninvasive diagnostic tests to confirm the diagnosis of endometriosis other than surgical exploration [ 8 – 12 ]. Advanced ultrasound and magnetic resonance imaging have emerged as methods for mapping deep endometriosis but still have substantial limitations in diagnosing the most common types of endometriosis (superficial disease) and advanced training is still concentrated in specialized centers [ 10 – 12 ]. Although biomarkers have shown promising results in small-scale studies, they currently lack the evidence needed for universal clinical implementation [ 8 , 13 ]. Thus, noninvasive diagnostic methods are urgently needed to facilitate early detection and management. Prediction models for endometriosis have demonstrated acceptable performance using traditional statistical techniques [ 14 – 17 ]. However, these models restricted the number of variables to avoid overfitting, a common limitation of classical statistical approaches [ 17 ]. Additionally, the populations studied were limited to individuals presenting with a preset duration of pelvic pain, those experiencing infertility/subfertility, or were designed to predict only site-specific endometriosis [ 14 , 16 , 18 – 20 ]. These limitations restrict the models’ ability to perform well in real-world settings. The rise of artificial intelligence (AI) utilization in healthcare offers the opportunity to use machine learning algorithms (MLAs) to analyze large datasets and detect patterns in presentation beyond what is evident in isolated comparisons. Other studies have utilized AI in this setting, but these models lack a rigorous delineation of cases and controls, potentially confounding the findings, and focus on subjective, patient-reported features [ 21 , 22 ]. Our study sought to examine the predictive value of various clinical features in the diagnosis of endometriosis by utilizing MLAs, a class of AI methods that identify patterns in data to make predictions. We hope that these findings will ultimately improve outcomes by hastening time to referral, diagnosis, and surgical intervention.