{"paper_id":"9f41605f-ccc9-4e7b-aa60-582a390dacf6","body_text":"Endometriosis, characterized by the presence of ectopic\nendometrial glands, affects approximately 5% of women\nin their reproductive age. Notably, endometriosis shows\nnotably higher rates, up to 10-fold, in younger women experiencing dysmenorrhea. Despite being commonly associated with the childbearing age, endometriosis imposes\na significant burden on young women, often leading to\ninfertility and pelvic pain ( 1 - 3 ).\nWhile a definitive diagnosis of endometriosis typically requires a laparoscopic investigation, guidelines\nsuggest that non-invasive imaging techniques, such\nas ultrasonography, should be performed as the initial\nstep. It is concerning, however, that there is frequently\na delay of up to 10 years in diagnosing endometriosis\nafter the onset of symptoms. This delay can result in unnecessary pain, diminished quality of life, impaired\nwork performance, and reduced fertility. Facilitating the timely diagnosis of endometriosis is crucial to\nmitigate these adverse effects on women's health and\nwell-being ( 2 ,  4 - 6 ). Promoting ultrasonography and\nmore expertise in practitioners can reduce the delay in\ndiagnosis. Comprehensive ultrasonography can be as\naccurate as a magnetic resonance imaging (MRI) study\n( 5 ,  7 ). There are suggestive signs of endometriosis in\na pelvic examination or an ultrasonogram, like adenomyosis, reduced ovarian motility, and adhered ovaries\nto the uterus or each other, also known as the kissing\novaries ( 8 ).\nMachine learning (ML) algorithms can process high-dimensional data and identify associations that may not be\napparent to humans ( 9 ). These algorithms are increasingly\nbeing used in medical imaging techniques and diagnosis,\ncontributing to improved accuracy in screening across\nvarious medical fields ( 4 ,  10 ,  11 ).\nIn the field of obstetrics and gynecology, ML has found\nutility in diverse areas such as infertility research and\ntreatment, fetal diagnosis and surgery, prenatal care, and\noncology ( 12 ). A recent systematic review outlined three\nmain domains of ML applicability: outcome prediction,\nresearch, and diagnosis ( 13 ).\nWhile there have been some studies evaluating the accuracy of ML-assisted endometrial diagnosis, most of them\nhave been limited by small data sets. Additionally, there\nis a lack of sufficient ML research specifically focused on\nIranian females with endometriosis. In light of these gaps,\nthe objective of this study was to assess the diagnostic\naccuracy of various ML algorithms for endometriosis in\nIranian females.\nTo achieve this, we designed a study that involved training ML algorithms to predict the presence of endometriosis in Iranian females. Subsequently, we aimed to validate\nthe performance of these ML algorithms in accurately diagnosing endometriosis.\n\nThis study was approved by the Ethical Committee\nof Shahid Beheshti University of Medical Sciences (IR.\nSBMU.RETECH.REC.1401.475). Participants gave their\ninformed written consent for being included in medical\nresearch and publication.\nOur data set was gathered retrospectively from patients who were referred to the Department of Gynecology and Obstetrics at Shohadaye Tajrish Hospital in\nTehran, Iran, between March 2020 and May 2022. The\ndata set included 149 patients with a confirmed diagnosis of endometriosis and 357 control participants. The\nage range of the participants was between 14 and 55\nyears old.\nTo identify patients for inclusion, we employed the\nInternational Classification of Diseases (ICD) coding algorithm, specifically the 10th Version (ICD-10) codes (please provide the reference number for the\ncodes used). In the patient group, the inclusion criteria\nwere a definite diagnosis of stage 3 or 4 endometriosis, determined through transvaginal or transabdominal\nultrasonography, pelvic MRI, or laparoscopy with pathology. Additionally, the patients in the endometriosis\ngroup presented with symptoms of dysmenorrhea or\ndyspareunia.\nThis retrospective study comprised patients who\nwere suspected of having endometriosis due to complaints of dysmenorrhea or dyspareunia. We collected\navailable data on demographic and ultrasonographic\ncharacteristics. Patients who were excluded from the\nstudy did not meet the criteria for endometriosis. Our\ndata set was retrospectively collected from patients\nreferred to the Gynecology and Obstetrics Department of the Shohadaye Tajrish Hospital, Tehran, Iran,\nbetween March 2020 and May 2022. It consisted of\n149 patients with a definite diagnosis of endometriosis and 357 controls. Participants aged between 14-55\nyears old. The inclusion criteria of the patient group\nwere: a definite diagnosis of stages 3 or 4 endometriosis (with either transvaginal or transabdominal\nultrasonography of pelvic MRI, or laparoscopy and\npathology), and dysmenorrhea or dyspareunia. This\nretrospective study consists of patients suspicious of\nendometriosis with complaints of dysmenorrhea or\ndyspareunia, with available data on demographic and\nultrasonographic characteristics and excluded for endometriosis.\nWe used an independent t test and (Chi-square) to determine whether patient and control groups were different in\ndemographic and related medical history.\nWe used Python (version: 3.10.4) (modules to develop\nour models: NumPy 1.23.4 (Scientific Computing Tools\nFor Python—Numpy, http://numpy.scipy.org), pandas\n1.5.0 (an open source ML library that supports supervised\nand unsupervised learning), scikit-learn 1.1.2 (an open\nsource ML library that supports supervised and unsupervised learning, https://scikit-learn.org), and matplotlib\n3.6.1 (comprehensive library for interactive visualizations\nin Python, https://matplotlib.org) ( 14 - 17 ). Our trained,\nvalidated and tested different ML models for the diagnosis of endometriosis with features of age, body mass\nindex (BMI), infertility history, state of sexual activity,\nultrasonographic signs of adenomyosis, kissing ovaries,\nand sliding signs. Other features were endometrioma and\novarian adhesion to the uterus (absent, one, two) ( 18 ). A\nbinary model was developed to classify patient and control groups.\nSince our data set was imbalanced in the distribution of\nthe target classes, our strategy was to randomly split the\ndata set into train (75%) and test (25%) subsets in a stratified fashion. Some algorithms are sensitive to the scaling\nof the data. Therefore, we ensured that for each feature\nthe mean is 0 and the variance is 1 to bring them to the\nsame magnitude.\nWe use this 5-fold cross-validation was applied to train\nand validate our models. The performance of the models\nwas evaluated using the area under the receiver operating\ncharacteristic curve (AUC). The receiver operating characteristic (ROC) curve is constructed by plotting the sensitivity fraction against the complement of specificity to\nimprove the model’s generalization performance, we performed hyperparameter tuning by exhaustive grid search\nfor all models except Random Forest and Extremely Randomized Trees, which we used a randomized parameter\noptimization method.\nIn our first attempts, we set a 50% chance of the disease\nas the threshold. The sensitivities were much lower than\nthe specificities, so we used the Youden index criteria to\nreach the optimal thresholds for every test ( 19 ). To assess\nthe performance of each model on the test set, sensitivity, specificity, positive predictive value (PPV), negative\npredictive value (NPV), and AUC, as the main classification performance gauges, were calculated ( 20 ). We tested\n8 different models as follows ( 16 ,  21 ,  22 ):\n\nThe recruitment details of the data are presented in Figure 1. In the test group, out of 127 patients, 37 (29.1%)\nwere diagnosed with endometriosis, while in the training\nset, 112 out of 378 patients (29.6%) had an endometriosis\ndiagnosis. The mean age ± standard deviation (SD) of patients in the endometriosis group was 34.58 ± 6.66 years,\nwhile in the control group, it was 33.52 ± 7.84 years. Patients were significantly less sexually active and reported\nmore complaints of infertility ( Table 1 ).\nThe flowchart of data set recruitment.\nTable 2  presents the confusion matrix for the models. A\nconfusion matrix is a tabular representation that summarizes the performance of a classification model by showing the counts of true positive (TP), true negative (TN),\nfalse positive (FP), and false negative (FN) predictions.\nIn the context of this study, the confusion matrix provides\ninsights into the models’ diagnostic accuracy. It allows us to assess how well the models classify patients with endometriosis and those without it.\n Table 1 Demographic characteristics of the data set\n Characteristics Patient  Control  P value Age (Y) 34.58 ± 6.66 33.52 ± 7.84 0.12 BMI (kg/m 2 ) 24.82 ± 3.40 24.73 ± 4.47 0.80 Sexual activity (%) Yes No 81.818.1 71.628.3 <0.001 Infertility (%) Yes No 50.349.6 32.867.1 <0.001 Data are presented as mean ± SD or %. BMI; Body mass index. \n Table 2 Confusion matrix of the best prediction results Classifier Prediction  Actual value Threshold Positive Negative Nearest Neighbors Positive 23 17 0.42 Negative 14 73 Logistic Regression Positive 28 25 0.38 Negative 9 65 RBF SVM Positive 26 26 0.20 Negative 11 64 Random Forest Positive 25 20 0.39 Negative 12 70 Extra-Trees Positive 26 18 0.39 Negative 11 72 AdaBoost Positive 22 15 0.50 Negative 15 75 Gradient Boosting Positive 27 24 0.31 Negative 10 66 SVM; Support Vector Machines and RBF; Radial Basis Function.\nDemographic characteristics of the data set\nData are presented as mean ± SD or %. BMI; Body mass index.\nConfusion matrix of the best prediction results\nSVM; Support Vector Machines and RBF; Radial Basis Function.\nAdditionally,  Table 3  provides a comprehensive overview of the diagnostic performance of each model. This\nincludes metrics such as sensitivity, specificity, and AUC.\nSensitivity represents the proportion of actual positive\ncases correctly identified by the model, while specificity\nmeasures the proportion of actual negative cases correctly\nidentified. The AUC is a measure of the model’s overall\ndiscriminative ability, with higher values indicating better performance. Furthermore,  Table 1  specifically focuses on the diagnostic performance of the models using\na 50% threshold. The optimal threshold for classification\nmay vary depending on the specific model and the desired\nbalance between sensitivity and specificity. According to\nprevious research ( 23 ), the latter model demonstrated\nhigher sensitivity, making it a preferable option for endometriosis screening. In our study, we observed a significant increase of 28% in the mean sensitivity of our models\nwhen thresholds were adjusted, without any decrease in\nthe mean AUC. Notably, among all the models evaluated,\nthe logistic regression model exhibited the highest sensitivity, while still maintaining a comparable AUC to the\nmachine learning-based models within the study.\nTo complement these findings, Figure 2 illustrates the\nROC curve for each model. The ROC curve visually\ndepicts the trade-off between sensitivity and specificity\nacross various threshold values, providing a graphical\nrepresentation of the model’s overall performance.\nComparison of different Algorithms for Binary Classification\nPPV; Positive predictive value, NPV; Negative predictive value, AUC; Area under the\ncurve, SVM; Support Vector Machines, and RBF; Radial Basis Function.\nThe ROC curve and AUC for each model in the present study. ROC;\nReceiver operating characteristic, AUC; Area under the curve, SVM: Support Vector Machines, and RBF; Radial Basis Function.\n\nIn this study, we aimed to estimate the diagnostic accuracy of different ML algorithms based on age, BMI,\nhistory of infertility, state of sexual activity, and ultrasonographic signs of endometriosis. We observed the best\nperformance in the RBF SVM, Random Forest and ExtraTrees, and Gradient Boosting models.\nThere have been few other ML studies on endometriosis\n( 18 ). Guerriero et al. ( 18 ) developed 7 models with the\nsame sonographic features as ours to classify the presence\nof rectosigmoid endometriosis. They set a threshold of\n50% chance of endometriosis as a prediction. According\nto AUCs, the best performance belonged to neural networks, with an AUC of 0.81. Sensitivities and specificities ranged from 66 to 84% and 71 to 77%, respectively.\nThere was no significant difference in the diagnostic performance of different models and logistic regression. The\nauthors were unable to show any meaningful superiority\nin ML models in comparison with logistic regression; they\nalso raised concerns about the more automated interpretation of data without expert supervision in ML-based diagnosis. We depicted roughly the same AUC values in our\nmodels, although we did not categorize the site of endometrial implantation in our patients. Our models showed\nsensitivities below 50% with the same fixed thresholds. In\nother words, we missed every other endometriosis patient\nin our data set. As we gained better results in predicted\nvalues, there was almost a 70% chance of correctness in\nevery classification choice the models made when the\nthreshold was set at 50%.\nIn a study conducted by Bendifallah et al. ( 24 ), involving 1,126 surgically-confirmed endometriosis patients, 16\ndemographic and clinical features were selected based on\nexpert opinion to develop AI models. The control group\nconsisted of patients who exhibited at least one endometriosis-related symptom. The study reported AUC values\nranging from 0.88 to 0.93, with the eXtreme Gradient\nBoosting (XGB) model achieving the highest AUC. The\nVoter classifiers demonstrated sensitivities and specificities exceeding 90% in their analysis. The study highlighted that the implementation of ML models [Random\nForest, Logistic Regression, Decision Tree, eXtreme Gradient Boosting (XGB), and Voting Classifier] could aid in\nthe development of screening tools accessible to patients\nor first-line health care workers. This approach holds the\npotential to deliver more affordable and accessible diagnostic solutions, particularly in scenarios such as the\nCOVID-19 pandemic with limited medical care access or\nin resource-limited settings ( 25 ).\nFurthermore, the study emphasized the significance of\ncomprehensive clinical evaluation, where clinical features\nmay offer greater accuracy in diagnostic challenges compared to imaging or signs, as observed in their study ( 25 ).\nIn a population-based study involving 5,924 endometriosis patients and 142,723 age-matched controls, a ML\nalgorithm exhibited a high predictive value for female\ngenitalia disorders and menometrorrhagia in the diagnosis of endometriosis. Interestingly, the authors found no\nadditional benefit in incorporating genetic features into\nthe ML models. Among the models evaluated, the XGB\nmodel showed the highest AUC as a measure of performance ( 26 ).\nContrary to our findings, a previous study reported the\npoorest performance in the Random Forest model, with\nan AUC of 0.71, while Support Vector Machines (SVMs)\ndemonstrated the highest AUC. This previous study went\nbeyond the established conclusions derived from standard\nstatistical modules by extracting characteristics related to\nendometriosis ( 26 ). This highlights one of the promising\nadvantages of ML in medicine, as it enables the discovery\nof new relationships ( 27 ).\nThe United States healthcare system claims their data\nset was used to extract data from 314,101 endometriosis\npatients and a 3 million age- and past medical historymatched control group ( 28 ). Thirty percent of the data\nset was preserved for the test. They introduced Logistic\nRegression and XGB models to patients’ medical history up to the point of diagnosis. Their criteria included\nnon-inflammatory diseases of the female genital tract and\nmenstrual disorders, which is in agreement with the observations of Blass et al. ( 26 ). Patients with AUCs of 0.96\nand 0.88 were classified by Logistic Regression and XGB\nmodels, respectively ( 28 ). A study by Kleczyk et al. ( 28 )\nshowed, that AUCs increased, and the same features were\nextracted from participants' records with a more extensive data set. Integrating such tools into electronic health\nrecords can provide diagnostic assistance to health care\nproviders.\nOur study had many limitations. First, we had a relatively small data set and did not use feature selection. We\ninstead used previous literature for choosing imaging features, so we only introduced a few clinical characteristics\nof our patients into the models. Despite these limitations,\nwe enriched the few available results on the applications\nof the ML’s algorithms in endometriosis diagnosis. Some\nimitation in our study is the absence of endometriosis patients with atonal features with negative physical examination, ultrasonographic or MRI records.\n\nML algorithms have shown promising results in different data sets and ethnic groups with different features.","source_license":"CC0","license_restricted":false}