Machine Learning-Based Detection of Endometriosis: A Retrospective Study in A Population of Iranian Female Patients

Endometriosis, characterized by the presence of ectopic endometrial glands, affects approximately 5% of women in their reproductive age. Notably, endometriosis shows notably higher rates, up to 10-fold, in younger women experiencing dysmenorrhea. Despite being commonly associated with the childbearing age, endometriosis imposes a significant burden on young women, often leading to infertility and pelvic pain ( 1 - 3 ). While a definitive diagnosis of endometriosis typically requires a laparoscopic investigation, guidelines suggest that non-invasive imaging techniques, such as ultrasonography, should be performed as the initial step. It is concerning, however, that there is frequently a delay of up to 10 years in diagnosing endometriosis after the onset of symptoms. This delay can result in unnecessary pain, diminished quality of life, impaired work performance, and reduced fertility. Facilitating the timely diagnosis of endometriosis is crucial to mitigate these adverse effects on women's health and well-being ( 2 , 4 - 6 ). Promoting ultrasonography and more expertise in practitioners can reduce the delay in diagnosis. Comprehensive ultrasonography can be as accurate as a magnetic resonance imaging (MRI) study ( 5 , 7 ). There are suggestive signs of endometriosis in a pelvic examination or an ultrasonogram, like adenomyosis, reduced ovarian motility, and adhered ovaries to the uterus or each other, also known as the kissing ovaries ( 8 ). Machine learning (ML) algorithms can process high-dimensional data and identify associations that may not be apparent to humans ( 9 ). These algorithms are increasingly being used in medical imaging techniques and diagnosis, contributing to improved accuracy in screening across various medical fields ( 4 , 10 , 11 ). In the field of obstetrics and gynecology, ML has found utility in diverse areas such as infertility research and treatment, fetal diagnosis and surgery, prenatal care, and oncology ( 12 ). A recent systematic review outlined three main domains of ML applicability: outcome prediction, research, and diagnosis ( 13 ). While there have been some studies evaluating the accuracy of ML-assisted endometrial diagnosis, most of them have been limited by small data sets. Additionally, there is a lack of sufficient ML research specifically focused on Iranian females with endometriosis. In light of these gaps, the objective of this study was to assess the diagnostic accuracy of various ML algorithms for endometriosis in Iranian females. To achieve this, we designed a study that involved training ML algorithms to predict the presence of endometriosis in Iranian females. Subsequently, we aimed to validate the performance of these ML algorithms in accurately diagnosing endometriosis.

The recruitment details of the data are presented in Figure 1. In the test group, out of 127 patients, 37 (29.1%) were diagnosed with endometriosis, while in the training set, 112 out of 378 patients (29.6%) had an endometriosis diagnosis. The mean age ± standard deviation (SD) of patients in the endometriosis group was 34.58 ± 6.66 years, while in the control group, it was 33.52 ± 7.84 years. Patients were significantly less sexually active and reported more complaints of infertility ( Table 1 ). The flowchart of data set recruitment. Table 2 presents the confusion matrix for the models. A confusion matrix is a tabular representation that summarizes the performance of a classification model by showing the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions. In the context of this study, the confusion matrix provides insights into the models’ diagnostic accuracy. It allows us to assess how well the models classify patients with endometriosis and those without it. Table 1 Demographic characteristics of the data set Characteristics Patient Control P value Age (Y) 34.58 ± 6.66 33.52 ± 7.84 0.12 BMI (kg/m 2 ) 24.82 ± 3.40 24.73 ± 4.47 0.80 Sexual activity (%) Yes No 81.818.1 71.628.3 <0.001 Infertility (%) Yes No 50.349.6 32.867.1 <0.001 Data are presented as mean ± SD or %. BMI; Body mass index. Table 2 Confusion matrix of the best prediction results Classifier Prediction Actual value Threshold Positive Negative Nearest Neighbors Positive 23 17 0.42 Negative 14 73 Logistic Regression Positive 28 25 0.38 Negative 9 65 RBF SVM Positive 26 26 0.20 Negative 11 64 Random Forest Positive 25 20 0.39 Negative 12 70 Extra-Trees Positive 26 18 0.39 Negative 11 72 AdaBoost Positive 22 15 0.50 Negative 15 75 Gradient Boosting Positive 27 24 0.31 Negative 10 66 SVM; Support Vector Machines and RBF; Radial Basis Function. Demographic characteristics of the data set Data are presented as mean ± SD or %. BMI; Body mass index. Confusion matrix of the best prediction results SVM; Support Vector Machines and RBF; Radial Basis Function. Additionally, Table 3 provides a comprehensive overview of the diagnostic performance of each model. This includes metrics such as sensitivity, specificity, and AUC. Sensitivity represents the proportion of actual positive cases correctly identified by the model, while specificity measures the proportion of actual negative cases correctly identified. The AUC is a measure of the model’s overall discriminative ability, with higher values indicating better performance. Furthermore, Table 1 specifically focuses on the diagnostic performance of the models using a 50% threshold. The optimal threshold for classification may vary depending on the specific model and the desired balance between sensitivity and specificity. According to previous research ( 23 ), the latter model demonstrated higher sensitivity, making it a preferable option for endometriosis screening. In our study, we observed a significant increase of 28% in the mean sensitivity of our models when thresholds were adjusted, without any decrease in the mean AUC. Notably, among all the models evaluated, the logistic regression model exhibited the highest sensitivity, while still maintaining a comparable AUC to the machine learning-based models within the study. To complement these findings, Figure 2 illustrates the ROC curve for each model. The ROC curve visually depicts the trade-off between sensitivity and specificity across various threshold values, providing a graphical representation of the model’s overall performance. Comparison of different Algorithms for Binary Classification PPV; Positive predictive value, NPV; Negative predictive value, AUC; Area under the curve, SVM; Support Vector Machines, and RBF; Radial Basis Function. The ROC curve and AUC for each model in the present study. ROC; Receiver operating characteristic, AUC; Area under the curve, SVM: Support Vector Machines, and RBF; Radial Basis Function.

In this study, we aimed to estimate the diagnostic accuracy of different ML algorithms based on age, BMI, history of infertility, state of sexual activity, and ultrasonographic signs of endometriosis. We observed the best performance in the RBF SVM, Random Forest and ExtraTrees, and Gradient Boosting models. There have been few other ML studies on endometriosis ( 18 ). Guerriero et al. ( 18 ) developed 7 models with the same sonographic features as ours to classify the presence of rectosigmoid endometriosis. They set a threshold of 50% chance of endometriosis as a prediction. According to AUCs, the best performance belonged to neural networks, with an AUC of 0.81. Sensitivities and specificities ranged from 66 to 84% and 71 to 77%, respectively. There was no significant difference in the diagnostic performance of different models and logistic regression. The authors were unable to show any meaningful superiority in ML models in comparison with logistic regression; they also raised concerns about the more automated interpretation of data without expert supervision in ML-based diagnosis. We depicted roughly the same AUC values in our models, although we did not categorize the site of endometrial implantation in our patients. Our models showed sensitivities below 50% with the same fixed thresholds. In other words, we missed every other endometriosis patient in our data set. As we gained better results in predicted values, there was almost a 70% chance of correctness in every classification choice the models made when the threshold was set at 50%. In a study conducted by Bendifallah et al. ( 24 ), involving 1,126 surgically-confirmed endometriosis patients, 16 demographic and clinical features were selected based on expert opinion to develop AI models. The control group consisted of patients who exhibited at least one endometriosis-related symptom. The study reported AUC values ranging from 0.88 to 0.93, with the eXtreme Gradient Boosting (XGB) model achieving the highest AUC. The Voter classifiers demonstrated sensitivities and specificities exceeding 90% in their analysis. The study highlighted that the implementation of ML models [Random Forest, Logistic Regression, Decision Tree, eXtreme Gradient Boosting (XGB), and Voting Classifier] could aid in the development of screening tools accessible to patients or first-line health care workers. This approach holds the potential to deliver more affordable and accessible diagnostic solutions, particularly in scenarios such as the COVID-19 pandemic with limited medical care access or in resource-limited settings ( 25 ). Furthermore, the study emphasized the significance of comprehensive clinical evaluation, where clinical features may offer greater accuracy in diagnostic challenges compared to imaging or signs, as observed in their study ( 25 ). In a population-based study involving 5,924 endometriosis patients and 142,723 age-matched controls, a ML algorithm exhibited a high predictive value for female genitalia disorders and menometrorrhagia in the diagnosis of endometriosis. Interestingly, the authors found no additional benefit in incorporating genetic features into the ML models. Among the models evaluated, the XGB model showed the highest AUC as a measure of performance ( 26 ). Contrary to our findings, a previous study reported the poorest performance in the Random Forest model, with an AUC of 0.71, while Support Vector Machines (SVMs) demonstrated the highest AUC. This previous study went beyond the established conclusions derived from standard statistical modules by extracting characteristics related to endometriosis ( 26 ). This highlights one of the promising advantages of ML in medicine, as it enables the discovery of new relationships ( 27 ). The United States healthcare system claims their data set was used to extract data from 314,101 endometriosis patients and a 3 million age- and past medical historymatched control group ( 28 ). Thirty percent of the data set was preserved for the test. They introduced Logistic Regression and XGB models to patients’ medical history up to the point of diagnosis. Their criteria included non-inflammatory diseases of the female genital tract and menstrual disorders, which is in agreement with the observations of Blass et al. ( 26 ). Patients with AUCs of 0.96 and 0.88 were classified by Logistic Regression and XGB models, respectively ( 28 ). A study by Kleczyk et al. ( 28 ) showed, that AUCs increased, and the same features were extracted from participants' records with a more extensive data set. Integrating such tools into electronic health records can provide diagnostic assistance to health care providers. Our study had many limitations. First, we had a relatively small data set and did not use feature selection. We instead used previous literature for choosing imaging features, so we only introduced a few clinical characteristics of our patients into the models. Despite these limitations, we enriched the few available results on the applications of the ML’s algorithms in endometriosis diagnosis. Some imitation in our study is the absence of endometriosis patients with atonal features with negative physical examination, ultrasonographic or MRI records.

ML algorithms have shown promising results in different data sets and ethnic groups with different features.

This study was approved by the Ethical Committee of Shahid Beheshti University of Medical Sciences (IR. SBMU.RETECH.REC.1401.475). Participants gave their informed written consent for being included in medical research and publication. Our data set was gathered retrospectively from patients who were referred to the Department of Gynecology and Obstetrics at Shohadaye Tajrish Hospital in Tehran, Iran, between March 2020 and May 2022. The data set included 149 patients with a confirmed diagnosis of endometriosis and 357 control participants. The age range of the participants was between 14 and 55 years old. To identify patients for inclusion, we employed the International Classification of Diseases (ICD) coding algorithm, specifically the 10th Version (ICD-10) codes (please provide the reference number for the codes used). In the patient group, the inclusion criteria were a definite diagnosis of stage 3 or 4 endometriosis, determined through transvaginal or transabdominal ultrasonography, pelvic MRI, or laparoscopy with pathology. Additionally, the patients in the endometriosis group presented with symptoms of dysmenorrhea or dyspareunia. This retrospective study comprised patients who were suspected of having endometriosis due to complaints of dysmenorrhea or dyspareunia. We collected available data on demographic and ultrasonographic characteristics. Patients who were excluded from the study did not meet the criteria for endometriosis. Our data set was retrospectively collected from patients referred to the Gynecology and Obstetrics Department of the Shohadaye Tajrish Hospital, Tehran, Iran, between March 2020 and May 2022. It consisted of 149 patients with a definite diagnosis of endometriosis and 357 controls. Participants aged between 14-55 years old. The inclusion criteria of the patient group were: a definite diagnosis of stages 3 or 4 endometriosis (with either transvaginal or transabdominal ultrasonography of pelvic MRI, or laparoscopy and pathology), and dysmenorrhea or dyspareunia. This retrospective study consists of patients suspicious of endometriosis with complaints of dysmenorrhea or dyspareunia, with available data on demographic and ultrasonographic characteristics and excluded for endometriosis. We used an independent t test and (Chi-square) to determine whether patient and control groups were different in demographic and related medical history. We used Python (version: 3.10.4) (modules to develop our models: NumPy 1.23.4 (Scientific Computing Tools For Python—Numpy, http://numpy.scipy.org), pandas 1.5.0 (an open source ML library that supports supervised and unsupervised learning), scikit-learn 1.1.2 (an open source ML library that supports supervised and unsupervised learning, https://scikit-learn.org), and matplotlib 3.6.1 (comprehensive library for interactive visualizations in Python, https://matplotlib.org) ( 14 - 17 ). Our trained, validated and tested different ML models for the diagnosis of endometriosis with features of age, body mass index (BMI), infertility history, state of sexual activity, ultrasonographic signs of adenomyosis, kissing ovaries, and sliding signs. Other features were endometrioma and ovarian adhesion to the uterus (absent, one, two) ( 18 ). A binary model was developed to classify patient and control groups. Since our data set was imbalanced in the distribution of the target classes, our strategy was to randomly split the data set into train (75%) and test (25%) subsets in a stratified fashion. Some algorithms are sensitive to the scaling of the data. Therefore, we ensured that for each feature the mean is 0 and the variance is 1 to bring them to the same magnitude. We use this 5-fold cross-validation was applied to train and validate our models. The performance of the models was evaluated using the area under the receiver operating characteristic curve (AUC). The receiver operating characteristic (ROC) curve is constructed by plotting the sensitivity fraction against the complement of specificity to improve the model’s generalization performance, we performed hyperparameter tuning by exhaustive grid search for all models except Random Forest and Extremely Randomized Trees, which we used a randomized parameter optimization method. In our first attempts, we set a 50% chance of the disease as the threshold. The sensitivities were much lower than the specificities, so we used the Youden index criteria to reach the optimal thresholds for every test ( 19 ). To assess the performance of each model on the test set, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and AUC, as the main classification performance gauges, were calculated ( 20 ). We tested 8 different models as follows ( 16 , 21 , 22 ):