AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach

Spontaneous rupture of ovarian endometriomas (OEs) may cause serious injury to patients. However, in traditional clinical diagnosis, it is vulnerable to be ignored or misdiagnosed for the symptoms of the acute abdomen caused by it are quite similar to those of some common gynecological emergencies, which leads to serious complications to patients. In view of this, this study investigates the AI-enabled, early and accurate diagnosis of spontaneous rupture of OEs. Although artificial intelligence (AI) has been proved to be a powerful tool to help to make accurate clinical diagnosis, however, as far as we know, there is so far no report on AI-enabled diagnosis of spontaneous rupture of OEs yet. Specifically, this study proposes a particle swarm optimization (PSO) enhanced random forest (RF) classification model, called PSO-RF, to make diagnosis, where RF is used to rank feature importance and make diagnosis considering that an OE is ruptured or not is a typical 0-1 classification problem, and PSO is leveraged to fine-tune the essential parameters of RF. The performance of the proposed PSO-RF model is evaluated with practical data collected from a local hospital and fully benchmarked by comparing with eight other machine learning models whose key parameters are sufficiently optimized as well by grid search or PSO, for the sake of fairness. The experiment results show that the proposed PSO-RF model outperforms all the other models, with the accuracy of 97.47%, the area under the ROC curve (AUC) of 0.996, the sensitivity of 94.12% and the specificity of 98.39%. It can be concluded that the PSO-RF model is a highly effective AI-enabled tool for preoperatively diagnosing spontaneous rupture of OEs. INDEX TERMS Ovarian endometrioma, spontaneous rupture, random forest, particle swarm optimization, machine learning. I. INTRODUCTION Ovarian endometrioma is a kind of endometriosis, patients of which have ectopic endometrial tissues in ovary [1] and suffer from various clinical symptoms including pelvic pain, irregular menstrual bleeding, dysmenorrheal, dyspareunia and infertility [2]. It is one of the most common diseases in reproductive women aging from 25 to 45 years old, with a high incidence of 10%-15% [3]. Although ovarian endometri- oma is a common benign disease, the size of the cyst typically increases gradually, and with a small probability, spontaneous The associate editor coordinating the review of this manuscript and approving it for publication was Derek Abbott . rupture of cyst may occur during or after menstruation [4], [5]. If the rupture can not be diagnosed and treated correctly in time, the severe cases may cause massive hemorrhage in the abdominal cavity, leading to hemorrhagic shock [6]. However, the symptoms of the acute abdomen caused by spontaneous rupture of ovarian endometrioma are similar to those of ectopic pregnancy, rupture and hemorrhage of corpus luteum, torsion of ovarian cyst pedicle and appendici- tis [7], [8]. Consequently, it is vulnerable to be ignored or misdiagnosed clinically which causes serious complications to patients. Early and accurate diagnosis the spontaneous rupture of ovarian endometrioma could significantly help to avoid serious injury. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 132253 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach Considering the high misdiagnosis of spontaneous rupture of ovarian endometrimas, researches on it has been widely carried out. Some of them have tried to find suitable biomark- ers to assist diagnosis of rupture of ovarian endometrimas preoperatively and have reported that spontaneous rupture of ovarian endometrimas are typically accompanied by high levels of serum CA125 and CA19-9 [1], [7], [9]–[12]. In particular, Xinyu Dai et al. have evaluated the clinical sig- nificance of serum CA125 and CA19-9 in the patients with spontaneous rupture of ovarian endometromas by statisti- cal methods [2]. The results have shown that the levels of CA125 and CA19-9 in patients with spontaneous rupture of ovarian endometromas increased significantly. Currently, it is believed that the combined biomarkers of CA125 and CA19-9 is helpful for the early diagnosis of spontaneous rupture of ovarian endometriomas. In the other hand, artificial intelligence (AI), especially machine learning (ML), has been widely used in medi- cal diagnosis [13]. The innovative AI-enabled methods are important assistances in precision medicine, which may help to make accurate clinical diagnosis at low cost and early. It has been frequently reported recently that the innova- tive AI-enabled approaches are important tools in precision medicine, which may help to make accurate clinical diagnosis at low cost and early. By leveraging various machine learning methods, Kawakami et al. have developed an ovarian cancer-specific predictive framework for clinical stage, histotype, residual tumor burden, and prognosis based on multiple biomarkers [14]. The results have revealed that AI-enabled models can provide accurate diagnoses and prognostic predictions for patients with epithelial ovarian cancer prior to initial inter- vention. In [15], an optimized random forest (RF) classifier has been used to detect nodules inside the lungs. In [16], Y . Shi et. al have proposed a hybrid QPSO-RF (Quantum particle swarm optimization - RF) model to predict the disease progression of Idiopathic Pulmonary Fibrosis using Computed Tomography (CT). Specifically, QPSO algorithm has been used to search the optimal feature subsets and RF has been used for classification. In [17], four machine learning algorithms, namely support vector machine (SVM), partial least squares discriminant analysis (PLS-DA), RF, and logis- tic regression (LR), have been applied to develop the model for identifying metabolomic biomarkers in cervicovaginal fluid for EC detection. Pergialiotis et al. have investigated the diagnostic accuracy of three different machine learning algorithms, i.e. LR, artificial neural networks and classifi- cation and regression trees (CARTs) for the prediction of endometrial cancer in postmenopausal women with vagi- nal bleeding or endometrial thickness ≥ 5 mm, as deter- mined by ultrasound examination [18]. E. Pashaei et al. have proposed a binary version of black hole algorithm for solving feature selection problem in biological data [19]. A novel one-dimensional deep densely connected neural network (DDNN) has been constructed in [20] to detect atrial fibrillation in 12-lead electrocardiogram waveforms. The proposed model has been proved to be sufficiently effective to be applied to clinical diagnosis of atrial fibril- lation. In the study of Obrzut et al., the usefulness of six AI-enabled methods have been evaluated for 5-year overall survival prediction in patients with cervical cancer treated by radical hysterectomy [21]. It has demonstrated that the best model, namely the probabilistic neural network model, achieves a high prediction accuracy of 0.892 and sensitivity of 0.975. Machine learning models have also been used in the diagnosis of endometrial tumor related diseases. In [22], deci- sion trees have been used to analyze the effectiveness of treat- ment of patients with recurrent pelvic cyst who underwent surgical intervention. The study of Günakan has constructed a naive Bayes model to make predictions of lymph node involvement in endometrial cancer [23]. In [24], automated image analysis and RF models have been used to classify normal, premalignant, and malignant endometrial tissues. However, as far as we known, no study on ML-enabled diagnostics of spontaneous rupture of ovarian endometriomas has been reported so far. Inspired by the above observations, this paper proposes a machine learning model to make accurate diagnosis of spontaneous rupture of ovarian endometriomas preopera- tively. Specifically, we first model the problem of cyst rup- ture as a 0-1 classification problem. Practical physiological data involving plenty of features are collected from women with ovarian endometriomas who have treated in the First Affiliated Hospital of Wenzhou Medical University between 2006 and 2017. Considering the effectiveness of random for- est algorithm [25] in feature selection and classification and therefore it has been widely used to make intelligent medical diagnoses [15]–[17], [26], [27], we design a random forest model to rank the importance of different features and further solve the 0-1 classification problem. In order to improve the performance of random forest model, we further leverage particle swarm optimization (PSO) algorithm [28] to optimize the parameters of the random forest model. To the best of our knowledge, this is the first time to adopt the machine learning technique in the diagnosis of spontaneous rupture of ovarian endometriomas. The proposed PSO enhanced RF model, PSO-RF for short, is comprehensively benchmarked by comparing with other fine-tuned machine learning models, including the gridsearch optimized random forest model (GO-RF), the naive Bayesian classification model [29] (NBC), the grid- search optimized k-nearest neighbor [30] model (GO-KNN), the PSO enhanced KNN (PSO-KNN), the gridsearch opti- mized lightGBM [31] model (GO-lightGBM), the PSO enhanced lightGBM model (PSO-lightGBM), the gridsearch optimized logical regression [32] model (GO-LR), and the PSO enhanced logical regression model (PSO-LR). The

show that the PSO-RF model outperforms all the other models, with the accuracy of 95.57%, the area under the ROC curve (AUC) of 0.996, the sensitivity of 94.12% and the specificity of 98.39%. 132254 VOLUME 8, 2020 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach TABLE 1. Baseline characteristics of endometriosis patients. The rest of this paper is organized as follows. The methodology of the proposed PSO-RF is given in detail in Section II. Section III presents the experimental verification and analysis. Finally, Section IV concludes the paper and proposes the future work. II. METHODS A. DATA ACQUISITION AND PREPROCESSING Physiological data were collected by complete blood counts, laparoscopic surgery and laparotomy from premenopausal female patients with ovarian endometriomas. The dataset includes 193 records where 53 patients have been diag- nosed with spontaneous ruptured ovarian endometriomas and 140 patients with unruptured ovarian endometriomas. Each sample in the dataset includes the age, the leukocyte count and the physiological features obtained by laparo- scopic surgery or laparotomy, such as the carcinoembryonic antigen (CEA) level, the CA125 level, the alpha fetopro- tein (AFP) level, the CA19-9 level, the position of the ovarian endometrioma, the endometrioma stage 1 and the size of the ovarian endometrioma, as shown in Table1. All the data have been normalized by x∗ = x−min max − min . The dataset has been divided into the training set and the test set by repeated random sampling until the P values with respect to all the features between the two sets are greater than 0.2. This results in a division of 114 patients in the training set (32 patients with ruptured ovarian endometriomas and 82 patients with unruptured ovarian endometriomas in the training set) and 79 patients in the test set (21 patients with ruptured ovarian endometriomas and 58 patients with unruptured ovarian endometriomas), as listed in Table 1. 1These patients were classified into four stages according to the revised classification made by the American Society for Reproductive Medicine (ASRM). FIGURE 1. Structure of a RF model. B. RANDOM FOREST MODEL Random forest (RF) is an ensemble learning model for classification and regression. RF runs efficiently on large data bases, seldom overfits, and has the ability of giving estimates of which features are important in the classification. In particular, RF is able to achieve a good accuracy rate for the classification of discrete high-dimensional datasets. With this in mind, in this work, RF is leveraged to diagnose the spontaneous rupture of ovarian endometriomas. The RF model consists of multiple classification and regression trees (CARTs) [33]. Each CART is a binary deci- sion tree that is grown by recursively partitioning the data in a parent node into child nodes. The structure of the RF model is shown in Fig. 1. 1) SAMPLING AND SPLITTING The overall samples in the training set are further divided into two parts. Specifically, 60% of the samples are used to train the CARTs, the set of which is denoted as D = {s1, s2, . . . ,s|D|} where si, i = 1, 2, . . . ,|D| is the ith VOLUME 8, 2020 132255 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach multi-dimensional sample in D. The rest are the pretest sam- ples used to measure the classification accuracy rate of this CARTs, the set of which is denoted as ˆD = { ˆs1, ˆs2, . . . ,ˆs| ˆD|} where ˆsi, i = 1, 2, . . . ,| ˆD| is the ith multi-dimensional sample in ˆD. Each CART is grown with a bootstrap sample set which is taken by randomly sampling with replacement from D. Concretely, one sample is randomly drawn from D as a train- ing sample at a time, and then, this sample is put back. This process of sampling with replacement is repeated m times so that to generate a new sample set with the same size as D. The new created sample set is then used as the root, e.g. the training set of a CART, denoted as Ri, i = 1, . . . ,NT , where NT is the number of CA TRs in RF. Apparently, in this way, some samples may appear multiple times in the new training set, while others, namely about 36.8% 2 of the samples in the original dataset, may never appear. Therefore, the samples that do not appear in the new dataset are taken as the test set. The training set of each CART will be split as a binary tree. Let NF be the number of features of each sample. At each node of the decision tree, randomly select Nf , Nf ≪ NF features for splitting. In the other words, at each node, the splitting feature is selected among a random subset of the original set of features. This randomness further enhances the generalization of RF model. The criteria of splitting in this work is the Gini index of impurity. Formally, denote the sample set at node t as St = {s t,1, . . . ,st,|St |} where st,i = {f t,i,1, . . . ,ft,i,NF } is the ith sample in node t and ft,i,j is the value of the jth feature in st,i. The class assignment of st,i is denoted as Ct,i ∈ {−1, 1} where -1 indicates an unruptured ovarian endometrioma and 1 indicates a ruptured ovarian endometrioma. The Gini index at node t is given by Gt = ∑ k=−1,1 pk|t (1 − pk|t ), (1) where pk|t is the probability of class k estimated from the samples in node t, e.g. pk|t = 1 |St | ∑ st,i∈St Ct,i = k. (2) At each step in the algorithm, the node t is split at the feature j and value ft,j into a pair of child nodes, denoted as tL = {x ∈ t|xj ≤ ft,j} and tR = {x ∈ t|xj > ft,j}, respectively, where j and ft,j are determined by arg max j,ft,j 1Gt |j,ft,j, (3) where 1Gt |j,ft,j = ( Gt − pL|t GL|t − pR|t GR|t )⏐⏐j,ft,j , (4) 2Assuming there are m samples in the original data set, the probability of each sample being selected is 1/m, and the probability of a sample not being selected in all of the m sampling is (1 − 1/m)m. Moreover, limm→∞ (1 − 1/m)m = e−1 ≈ 0.368 pL|t and pR|t are the probabilities of assigning a sample to the left child and right child, respectively. The process of splitting repeats until the terminal condition is satisfied, that is, 1) the depth of the CART reaches the predefined maximum value dmax, or 2) |St | is less than the predefined minimum value Ns,min. In this way, a CART is formed. The set of the CARTs in RF is denoted as T = {T 1, . . . ,TNT }. 2) VOTING Finally, all the CARTs involved in a RF cast votes for their inputs, after which RF aggregates their results and determines the output by majority voting of the CARTs. Taking in con- sideration the different classification abilities of CARTs in RF model, in this work, the voting weight of each CART is set separately according to its classification accuracy. In essence, after the training, the weight of each CART is estimated by the pretest samples in ˆD as follows: wTi = N correct i ⏐⏐⏐ ˆD ⏐⏐⏐ , i = 1, 2, . . . ,NT , (5) where N correct i is the number of samples classified correctly by Ti. The voting strategy is to summarize the classification

of all the CARTs, and then take the weighted mode of the classifications as the final classification results as follows: Csi = arg max k=−1,1 ∑ Ti∈T wTi · ( Csi | Ti = k ) . (6) where Csi|Ti is the classification result of feature si made by CART Ti. In summary, the procedure of random forest model is shown in Algorithm 1. 3) FEATURE IMPORTANCE One of the most attractive advantages of RF is that it gives estimates of what features are important in the classification. There are many biomarkers and factors associated with the rupture of ovarian endometriomas among which some may have very weak correlation with it. So, it will be of great significance to rank the importance of all the features to elim- inate irrelevant or weak correlated features, so as to reduce the complexity of RF model and improve the accuracy of the model. RF uses Gini index to evaluate the importance of a feature. Specifically, the importance of feature j to node t of Ti, denoted as V im j,t,i, is evaluated as follows. V im j,t,i = max ft,j 1Gj,t,i|ft,j . (7) Let Mj,i be the set of nodes where feature j is included in Ti. The importance of feature j to Ti can be calculated as V im j,i = ∑ t∈Mj,i V im j,t,i. (8) Further, the importance of feature j to RF model is V im j = ∑ Ti∈T V im j,i . (9) 132256 VOLUME 8, 2020 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach Algorithm 1 Procedure of Random Forest Model 1: Initialize NT , NF , Nf , dmax, Ns,min 2: Preprocess the data set and obtain D and ˆD 3: for i in [1, NT ] do 4: for j in [1, |D|] do 5: Randomly draw sj from D with replacement. 6: Insert sj to Ri 7: end for 8: t ← Ri 9: SPLIT(St , Nf ) 10: end for 11: Aggregates the results by Eq. (6). 12: 13: function split(St , Nf ) 14: if Termination condition is satisfied then return 15: else 16: Randomly select Nf out of NF features at t. 17: Determine the feature among Nf features and the value for splitting by Eq. (3) and Eq. (4). 18: Split the tagged node into tL and tR. 19: SPLIT(StL , Nf ) 20: SPLIT(StR, Nf ) 21: end if 22: end function Finally, V im j is normalized as follows: ˜V im j = V im j ∑ fi∈F V im i , (10) where F is the set of features. C. PARAMETER OPTIMIZATION BASED ON PSO Generally, in RF, some parameters such as the maximum depth of a CART (dmax), the minimum number of samples in a node (Ns,min) and the number of CARTs (NT ), etc., need to be tune for achieving the optimal performance of RF. A feasible solution is to fine-tune these parameters by an appropriate optimization algorithm, such as various heuristic algorithms. In this study, we leverage particle swarm optimization [34], which is a metaheuristic algorithm simulating the behavior of birds catching food, to tune the essential parameters of the RF model to further improve the accuracy. The essence of PSO is to use the current position, global extremum and individual extremum information to guide the next iteration position of particles, which enables it to approach the optimal solution with a fast convergence speed, and hence effectively optimize the parameters of the model [16], [35]. Another significant advantage of PSO is that it can adjust the maximum step size at each iteration, making it possible to find an approximate optimal solution in a wide range of possible parameters [36]. In addition, PSO has some attractive features, such as easy to implement, less parameters, computationally cheap and so on. PSO searches the optimal solution through agents, or called particles. A large number of particles form a swarm. A particle i is characterized by its location vector Li and velocity vector vi which are updated iteratively by Li(t + 1) = Li(t) + vi(t + 1), (11) and vi(t + 1) = wivi(t) + c1r1 ( Li,best − Li(t) ) +c2r2 ( Lgbest − Li(t) ) , (12) where wi is the selected weighting factor for particle i, Li,best is the location at which particle i previously had the best fitness measure, Lgbest is the global optimal location of the whole swarm, c1 and c2 are the cognitive acceler- ation constant and the social acceleration constant, respec- tively, generally with the value of 2, and r1 and r2 are two random parameters within the range [0, 1]. The iteration terminates if 1) reach the maximum number of iterations, or 2) convergence is reached based on the fitness measure. D. PSO-RF In summary, the procedure of the proposed PSO-RF is as follows: S1 Determine the parameters of PSO, including the maximum number of iterations, the maximum speed of particles and the search space, i.e., the feasible region of solutions. S2 Randomly generate a swarm where each particle is a three dimensional vector {dmax, Ns,min, NT }. The size of the swarm is denoted as NPSO. S3 Start PSO iteration procedure to obtain the optimal dmax, Ns,min and NT . S3.1 For each particle in the swarm, perform the RF algorithm defined in Algorithm 1 to evaluate the fitness, i.e., the diagnostic accuracy. S3.2 For each particle i, i ∈ {1, 2, . . . ,NPSO} in the swarm, compare the particle’s current fitness with the fitness of Li,best . If the current location fits better, then replace Li,best with the current location. S3.3 Update Lgbest if the new global optimal location of the current swarm fits better. S3.4 For each particle i, i ∈ {1, 2, . . . ,NPSO} in the swarm, update vi and Li by Eq.(12) and Eq.(11). S3.5 Repeat S3.1 - S3.4 until convergence is reached. S4 Perform the RF algorithm defined in Algorithm 1 with the optimal dmax, Ns,min and NT obtained in Step S3 to make diagnosis. The Pseudo code and the flow diagram of the PSO-RF procedure are shown in Algorithm 2 and Fig. 2, respectively. III. RESULTS All the involved algorithms and models have been imple- mented using Python 3.8. The computational analyses have been conducted on a Server running Windows Server 2016 Standard 64 bits operating system with Intel (R) Xeon (R) CPU E5-2620 v4, 32GB of RAM and Nvidia GeForce GTX 1080 Ti graphics card. Unless otherwise stated, VOLUME 8, 2020 132257 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach FIGURE 2. PSO-RF flowchart. Algorithm 2 Procedure of PSO-RF 1: Initialize the parameters for PSO. 2: for each particle i, i ∈ {1, 2, . . . ,NPSO} do 3: Randomly initiate vi and Li. 4: end for 5: repeat 6: for each particle i, i ∈ {1, 2, . . . ,NPSO} do 7: Perform Algorithm 1 to evaluate the fitness. 8: if fitness(particle i) > fitness(Li,best ) then 9: Li,best ← Li 10: end if 11: if fitness(particle i) > fitness(Lgbest ) then 12: Lgbest ← Li 13: end if 14: end for 15: until Termination conditions are satisfied. 16: Perform Algorithm 1 with Lgbest . the parameter settings of all the involved algorithms and models are listed in Table 2. A. PERFORMANCE INDICATORS 1) ACCURACY Diagnosis of spontaneous rupture of ovarian endometrioma is a typical binary classification problem, in which the diagnosis results are labeled either as positive (P), indi- cating a ruptured ovarian endometrioma, or negative (N), indicating an unruptured ovarian endometrioma. There are four possible combinations of the diagnosis results and the actuality, namely TP, FP, TN and FN which are explained as follows: • TP: both the diagnosis result and the actuality are P . • FP: the diagnosis result is p and the actuality is N. • TN: both the diagnosis result and the actuality are N. • FN: the diagnosis result is n and the actuality is P . TABLE 2. Parameters and settings of algorithms. The classification accuracy rate, or called the diagnostic accuracy rate, of the proposed PSO-RF model is evaluated by ACC = TP + TN TP + TN + FP + FN (13) 132258 VOLUME 8, 2020 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach 2) SENSITIVITY & SPECIFICITY Besides accuracy rate, sensitivity and specificity are also used to measure the effectiveness of PSO-RF model. Sensitivity, also known as true-positive rate (TPR), is used to measure the ability of correctly identifying the cases with ruptured ovarian endometriomas, whereas specificity, also known as true-negative rate (TNR), is used to measure the ability of the identifying those without the disease. The higher the sensi- tivity or the specificity is, the more effective the model is. Sensitivity (TPR) and specificity (TNR) can be calculated by TPR = TP TP + FN = TP P , (14) and TNR = TN FP + TN = TN N , (15) respectively. 3) ROC AND AUC Moreover, the receiver operating characteristic curve (ROC) is used to illustrate the diagnostic ability of the PSO-RF. ROC is a comprehensive indicator that is capable of evaluat- ing TPR and false-positive rate (FPR) at various thresholds. It is a two-dimensional graph in which TPR is plotted on the Y axis and FPR is plotted on the X axis. FPR can be calculated by FPR = FP FP + TN = FP N = 1 − TNR. (16) The AUC (area under the curve) is a concomitant of ROC obtained by calculating the area under the ROC. AUC tells how much the model is capable of distinguishing between ruptured and unruptured ovarian endometriomas. Higher the AUC, better the model is at predicting positives as positives and negatives as negatives. B. RESUL TS 1) FEATURE SELECTION The ranking of the feature importance for diagnosing the rupture of ovarian endometriomas derived by PSO-RF is shown in Fig. 3. The P values and the Pearson correlation coefficients (PCC) for the features related to the rupture of OE are listed in Table 3. It can be observed the ranking obtained by PSO-RF is on the whole consistent with those obtained by sta- tistical analyses. The level of CA19-9 and CA125 are the top two important biomarkers, with the normalized importance of 0.316 and 0.273, respectively, the P values of 4.08 × 10−11 and 2.47 × 10−14, respectively, and the PPC values of 0.4522 and 0.5128, respectively. This conclusion is consistent with the clinical experience. Serum CA19-9 and CA125, tumor associated antigens, have been used clini- cally to identify the recurrence and severity of endometriosis. The endometrioma cyst fluid was suspected to be rich of biomarkers such as CA19-9 and CA125. However, in nor- mal conditions, the large CA19-9 and CA125 glycoprotein FIGURE 3. Ranking of the importance of features of ovarian endometriomas. TABLE 3. Association of ruptured/unruptured OEs with clinicopathologic features. molecules were preventing by the thick wall of endometrioma cyst from entering the peripheral circulation. The sponta- neous rupture of the OE may lead the biomarkers into the blood circulation, resulting the high elevation of the CA19-9 and CA125 in the serum [11]. The level of CEA, with the normalized importance of 0.231, P value of 7.35×10−11, and PCC value of -0.4468, is the third important biomarker. The leukocytes count and AFP level are also significant biomark- ers, however, are less important than CA19-9, CA125 and CEA. In order to investigate the optimal combinations of features to be input into the PSO-RF model, the effectiveness of any single feature and various feature combinations on diagnosis of ruptured ovarian endometriomas is evaluated. Specifically, Fig. 4 plots the ROC curves and the corresponding AUC values of the classification results derived from PSO-RF using a single feature. Not surprisingly, CA19-9, CA125 and CEA output the most effective results, with the values of AUC of 0.948, 0.902 and 0.819, respectively. In addition, although the importance of the Leukocyte level is a bit higher than that of the AFP level, the AUC value of AFP level (0.806) is much greater than that of Leukocyte level (0.725). The ROC Curves, AUC values and accuracy rates of classifications using various features combinations derived from PSO-RF are depicted in Fig. 4(b). It can be seen that the best performance is achieved in the case of inputting VOLUME 8, 2020 132259 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach FIGURE 4. (a) ROCs Curves and values of AUC of all the features derived from PSO-RF for segregating ruptured ovarian endometriomas from ovarian endometriomas, (b) ROCs Curves, values of AUC and accuracy of various features combinations derived from PSO-RF for segregating ruptured ovarian endometriomas from ovarian endometriomas. the following four features: CA19-9 level, CA125 level, leukocyte count and AFP level, with the AUC of 0.996 and the accuracy of 97.47%. The feature combination of CA19-9 level, CA125 level, leukocyte count, AFP level and CEA level also results in the same high accuracy of 97.47%, however, the AUC value is a bit lower, with the value of 0.99. The box and jitter plots representing the distributions of the selected four features for PSO-RF-based classification of ruptured and unruptured OEs are shown in Fig. 5. It can be observed that the distributions are significantly different between the ruptured and unruptured OEs. The levels of CA19-9, CA125, leukocyte count and AFP of the patients with ruptured OEs are all significantly higher than those of patients with unruptured OEs. 2) COMPARISONS WITH OTHER ALGORITHMS The proposed PSO-RF model is comprehensively benchmarked by comparing with eight fine-tuned machine learning models, including four gridsearch optimized (GO) models, namely the GO-RF, GO-lightGBM, GO-LR and GO-KNN, three PSO enhanced models, namely FIGURE 5. Box and jitter plots representing distribution of the selected biomarkers between the ruptured and unruptured ovarian endometriomas. PSO-lightGBM, PSO-LR and PSO-KNN, and the NBC model. The parameters obtained by grid search are shown in Table 2. NBC model has no parameter tuning for there is no parameter to be fine-tuned in it. Table 4 summarizes the performance of all the nine models, including the classification accuracy rates, the sen- sitivity and the specificity. Besides the performance results obtained from the dataset fragmentation shown in Table 1, Table. 4 also lists the accuracy rates obtained by hold- out cross-validations (40% holdout samples, 100 times) and 10-fold cross-validations, respectively, to evaluate the per- formance more comprehensively. The distributions of the cross-validation results are presented as box plots, as shown in Fig. 6. For each test in the 40% holdout cross-validations, the dataset is divided into a training set and a test set by repeated random sampling until the P values with respect to all the four features between the two sets are greater than 0.2 to ensure that there is no significant correlation between the samples of the two sets. As to the 10-fold cross-validations, the P values are not guaranteed to be greater than 0.2. 132260 VOLUME 8, 2020 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach TABLE 4. Performance comparisons among PSO-RF and benchmark models. FIGURE 6. Box plots of cross-validations. It can be observed that the performances of PSO-RF in both cross-validations are the highest. Specifically, for 40% holdout cross-validations, PSO-RF achieves the highest aver- age accuracy rate of 95.47%, with the minimum standard deviation (S.D.) of 1.36%. The red box of PSO-RF in Fig. 6 is the highest one, and also the narrowest one, with an interquartile range (IQR) of 1.27% and a min-max range of 7.59%. PSO-RF also performs the best in the 10-fold cross-validations, with the highest average accuracy rate of 92.84%. The GO-RF model and the PSO-lightGBM model also achieve high average accuracy rates, i.e., 93.58% and 93.68% in 40% holdout cross-validations, and 91.26% and 92.54% in 10-fold cross-validations, respectively. Table 4 also shows that PSO does improve the performance of these models. It slightly improves the accuracy rates of the RF model, the lightGBM model, the LR model and the KNN model by 1.89%, 2.13%, 1.52% and 1.01%, respec- tively, in the 40% holdout cross-validations, and 1.58%, 0.97%, 2.53% and 0.47%, respectively, in the 10-fold cross- validations. The reason why the performance improvements are limited is that the parameters of the original models have also been optimized by grid search. However, the time complexity of PSO is O(NiterNPSO log NPSO) where Niter is the number of iterations and NPSO is the size of the swarm. Whereas that of the gridsearch approach is O (∏ i ni ) where ni, i = 1, 2, . . . ,j is number of potential values of feature i and j is the number of features to be fine-tuned, which is much higher than that of PSO. The superiority of the PSO-RF model is also presented in Fig. 7 where the ROCs of these models are plotted. It can be observed intuitively that the curve of PSO-RF is the closest one to the upper left corner (AUC = 0.996), implying that it outperforms all the others for its highest the overall accuracy. The ROCs of the PSO-lightGBM model and the GO-RF model are also quite closer to the upper left corner, with slightly lower AUC values, i.e., 0.996 for the PSO-lightGBM model and 0.995 for GO-RF model, respectively. The confusion matrices of the PSO-RF model and the benchmark models are shown in Fig. 8. PSO-RF performs pretty well. There is only one false positive case and one false negative case, respectively. By contrast, one false nega- tive and two false positives are made by the GO-RF model and the PSO-lightGBM model. The PSO-LR model, the GO-lightGBM model and the NBC model recognize all the patients with unruptured ovarian endometriomas, however, make three, four and five false positives, respectively. In addition, the PSO-KNN model makes two false positives and two false negatives, the GO-LR makes one false negative and four false positives, whereas the GO-KNN model makes four false negatives and two false positives, all of which are inferior to the proposed PSO-RF model. In addition to the classification accuracy, the efficiency of these classifiers is also evaluated. For PSO enhanced models, the convergence performances are evaluated since they train the models iteratively to approach the sub-optimal solutions. Fig. 9 shows the comparisons of the convergence VOLUME 8, 2020 132261 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach FIGURE 7. ROCs of PSO-RF and benchmark models. FIGURE 8. Confusion matrices of PSO-RF and benchmark models. performance among PSO enhanced models. The fitness of all the algorithm is the accuracy rate. It can be observed that the accuracy rate of the PSO-RF model grad- ually rises in the beginning (within 80 iterations). Then it converges to the stable value of 97.47%. The other three PSO enhanced models, namely the PSO-LR model, the PSO-KNN model and the PSO-lightGBM model con- verge after the 120th iteration, moreover, with lower accuracy rates. Fig. 9 only presents the convergence performance along with iterations. However, the time complexities per iteration of difference models vary greatly. In view of this, next, we evaluate the per iteration time complexities of these mod- els. Considering that RF, lightGBM, KNN, NBC and LR are not iterative algorithms, the CPU time required to train a model and then to make a classification are measured instead of the convergence rate. The results averaged from 100 tests are shown in Fig. 9(b), where it can be observed that the CPU time needed by RF is the longest. The reason is that the time complexity of RF is O(NT dmax · mn), that is, it is determined not only by the number of samples (m) and the number of features (n), but also by the number of CARTs (N T ) and the depth of a CART (d max). As a contrast, the time complexity of LR is only O(mn); that of NBC is O(cmn) where c is the number of classes; that of KNN is O(kmn) where k is the number of neighbors; and the time complexity of lightGBM is co-determined by the procedures of gradient-based one- side sampling and exclusive feature bundling, with the time complexities of O(lmn) and O(lmnb), respectively, where l is the number of leaves and nb ≫ n is the number of bundles. Specifically, in this work, NT = 94, dmax = 2, c = 2, k = 7 and l = 9, so that NT · dmax ≫ k, l, 2, 1,

in a much longer CPU time of RF than those of other models. Nevertheless, about 0.2 second CPU time training and classification can fully satisfy the requirements of the classification system. 132262 VOLUME 8, 2020 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach FIGURE 9. Evaluations on convergence rates and CPU times. IV. CONCLUSIONS Spontaneous rupture of OE is vulnerable to be ignored or misdiagnosed clinically, which may probably lead to serious complications to patients. In this work, we proposed a PSO enhanced RF model, namely PSO-RF, to assist in the preoper- ative diagnosis of spontaneous rupture of ovarian endometri- omas. As far as we know, it is the first work on ML-enabled diagnostics of spontaneous rupture of OEs. The data used in this work were collected by complete blood counts, laparo- scopic surgery and laparotomy from premenopausal female patients with ovarian endometriomas who have treated in the First Affiliated Hospital of Wenzhou Medical University. We first leveraged the RF model to rank the importance of the features and select the proper feature combination. Then, we employed PSO to fine-tune the essential param- eters of the RF model to further improve the diagnosis accuracy. The PSO-RF model was comprehensively benchmarked by comparing with other machine learning models, including three PSO enhanced models and five gridsearch-optimized models. The results showed that the PSO-RF model outper- forms all the other models in accuracy, with the accuracy of 97.47%, the AUC value of 0.996, the sensitivity of 94.12% and the specificity of 98.39%, due to the excellent classi- fication ability of RF and parameter tuning based on PSO. Although the PSO-RF model costs the highest time complex- ity, the overall complexity is totally acceptable practically. Our work concludes that the PSO-RF model is a highly effective AI-enabled tool for diagnose spontaneous rupture of ovarian endometriomas. It is fully qualified to assist surgeons in diagnosing whether a patient’s ovarian endometrioma is ruptured preoperatively. V. ACKNOWLEDGMENT (Mingyan Zhou and Feng Lin contributed equally to this work.)

[1] A. Simonelli, R. Guadagni, P . De Franciscis, N. Colacurci, M. Pieri, P . Basilicata, P . Pedata, M. Lamberti, N. Sannolo, and N. Miraglia, ‘‘Envi- ronmental and occupational exposure to bisphenol a and endometriosis: Urinary and peritoneal fluid concentration levels,’’Int. Arch. Occupational Environ. Health, vol. 90, no. 1, pp. 49–61, Jan. 2017. [2] X. Dai, C. Jin, Y . Hu, Q. Zhang, X. Y an, F. Zhu, and F. Lin, ‘‘High CA- 125 and CA19-9 levels in spontaneous ruptured ovarian endometriomas,’’ Clinica Chim. Acta, vol. 450, pp. 362–365, Oct. 2015. [3] T. A. Gelbaya and L. G. Nardo, ‘‘Evidence-based management of endometrioma,’’ Reproductive Biomed. Online, vol. 23, no. 1, pp. 15–24, Jul. 2011. [4] J. H. Pratt and W. R. Shamblin, ‘‘Spontaneous rupture of endome- trial cysts of the ovary presenting as an acute abdominal emer- gency,’’ Amer . J. Obstetrics Gynecol., vol. 108, no. 1, pp. 56–62, Sep. 1970. [5] K. Tanaka, Y . Kobayashi, K. Dozono, H. Shibuya, Y . Nishigaya, M. Momomura, H. Matsumoto, and M. Iwashita, ‘‘Elevation of plasma D-dimer levels associated with rupture of ovarian endometriotic cysts,’’ Taiwanese J. Obstetrics Gynecol., vol. 54, no. 3, pp. 294–296, Jun. 2015. [6] M. Králíčková, A. S. Laganà, F. Ghezzi, and V . V etvicka, ‘‘Endometriosis and risk of ovarian cancer: What do we know?’’Arch. Gynecol. Obstetrics, vol. 301, pp. 1–10, Nov. 2019. [7] V . F. D. Amaral, R. A. Ferriani, M. F. S. D. Sá, A. A. Nogueira, J. C. R. E. Silva, A. C. J. D. S. R. E. Silva, and M. D. D. Moura, ‘‘Positive correlation between serum and peritoneal fluid CA-125 levels in women with pelvic endometriosis,’’ Sao Paulo Med. J., vol. 124, no. 4, pp. 223–227, 2006. [8] F. Zullo, E. Spagnolo, G. Saccone, M. Acunzo, S. Xodo, M. Ceccaroni, and V . Berghella, ‘‘Endometriosis and obstetrics complications: A systematic review and meta-analysis,’’Fertility Sterility, vol. 108, no. 4, pp. 667–672, Oct. 2017. [9] D. B. Manders, S. C. Purinton, J. S. Lea, D. L. Richardson, D. S. Miller, and S. M. Kehoe, ‘‘Predictive value of carcinoembryonic antigen and cancer antigen 125 in identifying mucinous ovarian tumors,’’ Gynecol. Oncol., vol. 123, no. 2, pp. 445–446, Nov. 2011. [10] T. Ren, S. Wang, J. Sun, J.-M. Qu, Y . Xiang, K. Shen, and J. H. Lang, ‘‘Endometriosis is the independent prognostic factor for survival in Chi- nese patients with epithelial ovarian carcinoma,’’ J. Ovarian Res., vol. 10, no. 1, p. 67, Dec. 2017. [11] B.-J. Park, T.-E. Kim, and Y .-W. Kim, ‘‘Massive peritoneal fluid and markedly elevated serum CA125 and CA19-9 levels associated with an ovarian endometrioma,’’ J. Obstetrics Gynaecol. Res., vol. 35, no. 5, pp. 935–939, Oct. 2009. [12] P .-H. Wang, Y .-T. Huang, K.-K. Ng, H.-H. Chou, H.-Y . Lu, S.-H. Ng, and G. Lin, ‘‘Detecting recurrent ovarian cancer: Revisit the values of whole-body CT and serum CA 125 levels,’’ Acta Radiol., vol. 60, no. 10, pp. 1360–1366, Oct. 2019. [13] K. R. Foster, R. Koprowski, and J. D. Skufca, ‘‘Machine learning, medical diagnosis, and biomedical engineering research-commentary,’’ Biomed. Eng. OnLine, vol. 13, no. 1, p. 94, 2014. [14] E. Kawakami, J. Tabata, N. Y anaihara, T. Ishikawa, K. Koseki, Y . Iida, M. Saito, H. Komazaki, J. S. Shapiro, C. Goto, Y . Akiyama, R. Saito, M. Saito, H. Takano, K. Y amada, and A. Okamoto, ‘‘Application of arti- ficial intelligence for preoperative diagnostic and prognostic prediction in epithelial ovarian cancer based on blood biomarkers,’’ Clin. Cancer Res., vol. 25, no. 10, pp. 3006–3015, May 2019. [15] M. P . Paing, K. Hamamoto, S. Tungjitkusolmun, S. Visitsattapongse, and C. Pintavirooj, ‘‘Automatic detection of pulmonary nodules using three-dimensional chain coding and optimized random forest,’’ Appl. Sci., vol. 10, no. 7, p. 2346, Mar. 2020. VOLUME 8, 2020 132263 M. Zhou et al.: AI-Enabled Diagnosis of Spontaneous Rupture of Ovarian Endometriomas: A PSO Enhanced Random Forest Approach [16] Y . Shi, W. K. Wong, J. G. Goldin, M. S. Brown, and G. H. J. Kim, ‘‘Prediction of progression in idiopathic pulmonary fibrosis using CT scans at baseline: A quantum particle swarm optimization–random forest approach,’’ Artif. Intell. Med., vol. 100, Sep. 2019, Art. no. 101709. [17] S.-C. Cheng, K. Chen, C.-Y . Chiu, K.-Y . Lu, H.-Y . Lu, M.-H. Chiang, C.-K. Tsai, C.-J. Lo, M.-L. Cheng, T.-C. Chang, and G. Lin, ‘‘Metabolomic biomarkers in cervicovaginal fluid for detecting endometrial cancer through nuclear magnetic resonance spectroscopy,’’Metabolomics, vol. 15, no. 11, p. 146, Nov. 2019. [18] V . Pergialiotis, A. Pouliakis, C. Parthenis, V . Damaskou, C. Chrelias, N. Papantoniou, and I. Panayiotides, ‘‘The utility of artificial neural net- works and classification and regression trees for the prediction of endome- trial cancer in postmenopausal women,’’ Public Health, vol. 164, pp. 1–6, Nov. 2018. [19] E. Pashaei and N. Aydin, ‘‘Binary black hole algorithm for feature selec- tion and classification on biological data,’’ Appl. Soft Comput., vol. 56, pp. 94–106, Jul. 2017. [20] W. Cai, Y . Chen, J. Guo, B. Han, Y . Shi, L. Ji, J. Wang, G. Zhang, and J. Luo, ‘‘Accurate detection of atrial fibrillation from 12-lead ECG using deep neural network,’’ Comput. Biol. Med., vol. 116, Jan. 2020, Art. no. 103378. [21] B. Obrzut, M. Kusy, A. Semczuk, M. Obrzut, and J. Kluska, ‘‘Prediction of 5–year overall survival in cervical cancer patients treated with radical hysterectomy using computational intelligence methods,’’ BMC Cancer, vol. 17, no. 1, p. 840, 2017. [22] Y .-F. Wang, M.-Y . Chang, R.-D. Chiang, L.-J. Hwang, C.-M. Lee, and Y .-H. Wang, ‘‘Mining medical data: A case study of endometriosis,’’ J. Med. Syst., vol. 37, no. 2, p. 9899, Apr. 2013. [23] E. Günakan, S. Atan, A. N. Haberal, İ. A. Küçükyildiz, E. Gökçe, and A. Ayhan, ‘‘A novel prediction method for lymph node involvement in endometrial cancer: Machine learning,’’ Int. J. Gynecol. Cancer, vol. 29, no. 2, pp. 320–324, Feb. 2019. [24] M. J. Downing, D. J. Papke, S. Tyekucheva, and G. L. Mutter, ‘‘A new clas- sification of benign, premalignant, and malignant endometrial tissues using machine learning applied to 1413 candidate variables,’’ Int. J. Gynecol. Pathol., vol. 39, no. 4, pp. 333–343, 2020. [25] L. Breiman, ‘‘Random forests,’’ Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001. [26] A. Subasi, E. Alickovic, and J. Kevric, ‘‘Diagnosis of chronic kidney disease by using random forest,’’ in Proc. CMBEBIH, Singapore, 2017, pp. 589–594. [27] E. Alickovic and A. Subasi, ‘‘Medical decision support system for diagno- sis of heart arrhythmia using DWT and random forests classifier,’’ J. Med. Syst., vol. 40, no. 4, p. 108, Apr. 2016. [28] S. M. Raafat, A. M. Hasan, T. W. A. Khairi, and K. G. Ali, ‘‘Opti- mized performance of consensus algorithm in multi agent system using PSO,’’ Al-Nahrain J. Eng. Sci., vol. 21, no. 2, pp. 292–299, Apr. 2018. [29] M. A. P . Andrade and M. A. M. Ferreira, ‘‘Bayesian networks use in simple maternity problems,’’ Appl. Math. Sci., vol. 8, pp. 6963–6967, 2014. [30] P . Sala, A. Colatutto, D. Fabbro, L. Mariuzzi, S. Marzinotto, B. Toffoletto, A. R. Perosa, and G. Damante, ‘‘Immunoglobulin k light chain deficiency: A rare, but probably underestimated, humoral immune defect,’’ Eur . J. Med. Genet., vol. 59, no. 4, pp. 219–222, Apr. 2016. [31] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Y e, and T.-Y . Liu, ‘‘LightGBM: A highly efficient gradient boosting decision tree,’’ in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 3146–3154. [32] S. Dreiseitl and L. Ohno-Machado, ‘‘Logistic regression and artificial neural network classification models: A methodology review,’’ J. Biomed. Informat., vol. 35, nos. 5–6, pp. 352–359, Oct. 2002. [33] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification Regression Trees. Boca Raton, FL, USA: CRC Press, 1984. [34] J. Kennedy and R. Eberhart, ‘‘Particle swarm optimization,’’ in Proc. Int. Conf. Neural Netw., vol. 4, Nov./Dec. 1995, pp. 1942–1948. [35] Y . Zhang, S. Y u, R. Xie, J. Li, A. Leier, T. T. Marquez-Lago, T. Akutsu, A. I. Smith, Z. Ge, J. Wang, T. Lithgow, and J. Song, ‘‘PeNGaRoo, a combined gradient boosting and ensemble learning framework for pre- dicting non-classical secreted proteins,’’ Bioinformatics, vol. 36, no. 3, pp. 704–712, Aug. 2019. [36] B. Xue, M. Zhang, W. N. Browne, and X. Y ao, ‘‘A survey on evolutionary computation approaches to feature selection,’’ IEEE Trans. Evol. Comput., vol. 20, no. 4, pp. 606–626, Aug. 2016. MINGYAN ZHOU is currently pursuing the bachelor’s degree with the College of Computer Science and Artificial Intelligence, Wenzhou Uni- versity, Wenzhou, China. She is currently work- ing on machine learning applications in intelligent medicine. FENG LIN received the B.S. degree in obstetrics and gynecology and the M.S. degree in gynecol- ogy from Wenzhou Medical University, Zhejiang, China, in 2003 and 2011, respectively. She is currently an Associated Physician and a Profes- sor with the First Affiliated Hospital of Wenzhou Medical University. She mainly specializes in malignant tumors of gynecology and women infertility, and is good at surgical techniques in laparoscopy and hysteroscopy. QIAN HU received the B.S. degree in communi- cation engineering from the Nanjing Institute of Posts and Telecommunications, Nanjing, China, in 2001, and the M.S. degree in circuits and sys- tems from Zhejiang University, Hangzhou, China, in 2005. She is currently an Associated Professor with the College of Computer Science and Artifi- cial Intelligence, Wenzhou University, Wenzhou, China. Her research interests are in the areas of machine learning, intelligent systems, and wireless networks. ZHENZHOU TANG (Senior Member, IEEE) received the B.S. degree in electronic engineering and the M.S. degree in communications and information system from Zhejiang University, in 2001 and 2004, respectively, and the Ph.D. degree in communications and information sys- tem from the Dalian University of Technology, in 2015. In March 2004, he joined Wenzhou University. He is currently a Professor with the College of Computer Science and Artificial Intel- ligence, Wenzhou University, and the Director of the Intelligent Network Innovation Research Center, Wenzhou University. From February 2015 to August 2015, he was a Visiting Scholar with the Department of Electrical and Computer Engineering, Worcester Polytechnic Institute, Worcester, MA, USA. From November 2015 to June 2018, he was a Postdoctoral Research Fellow with Zhejiang University. Since September 2018, he has been a joint Ph.D. Supervisor with Chonnam National University, South Korea. He is currently serving as an Associate Editor of IEEE ACCESS. His current research interests include machine learning, intelligent systems, and wireless networks. CHU JIN received the M.Sc. degree in information security from the Royal Holloway and Bed- ford New College, University of London, U.K., in 2002. He is currently a Lecturer with the Renji College, Wenzhou Medical University, Wenzhou, China. His research interests are in the areas of computer security and network security. 132264 VOLUME 8, 2020