A novel and interpretable risk model for predicting distant metastasis in appendiceal malignant neoplasms: A retrospective study based on the SEER database

A novel and interpretable risk model for predicting distant metastasis in appendiceal malignant neoplasms: A retrospective study based on the SEER database | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 11 February 2025 V1 Latest version Share on A novel and interpretable risk model for predicting distant metastasis in appendiceal malignant neoplasms: A retrospective study based on the SEER database Authors : Zhou jinhong 0009-0008-4410-8853 and Xie xiaojun [email protected] Authors Info & Affiliations https://doi.org/10.22541/au.173927358.81987754/v1 228 views 111 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Background Timely identification and treatment can greatly enhance the prognosis of patients with distant metastases(DM). This study aims to reveal the risk factors for DM in appendiceal malignant neoplasms patients, and establish a machine learning (ML) model for predicting the risk of DM. Methods A total of 9,376 patients with appendiceal malignant neoplasms, categorized based on the AJCC 8th TNM staging system, were chosen for the study from the Surveillance, Epidemiology, and End Results (SEER) database. Building on this, we developed four machine learning algorithm models along with a Nomogram model. The model was assessed based on confusion matrix, receiver operating characteristic (ROC) curve AUC, calibration curve analysis, and decision curve analysis (DCA). Additionally, the relationship between clinical pathological features and target variables was explored using the SHAP algorithm based on the optimal model. To assess its performance and generalizability, we validated the best model using 52 cases of appendiceal malignant neoplasms from the First Affiliated Hospital of Shantou University Medical College, China. Results Univariate logistic regression analysis and multivariable logistic regression analysis suggested that gender, histological type, grade, T stage, N stage, CEA level, tumor size and distant lymph nodal metastasis were risk factor for DM, while age and race may be related to DM rather than independent risk factors. Five models were constructed incorporated the clinical features. The XGBoost model demonstrated the best performance, achieving an AUC of 0.9917 in the training group and 0.9738 in the internal validation group, respectively. The accuracy was higher than 0.9 in both cohorts. Furthermore, the XGBoost model was evaluated in the outer-validation group, achieving an accuracy of 0.8654 and an AUC of 0.8792. Both the DCA and calibration curves further supported its robust predictive capability. Conclusions Our model was capable of accurately predicting the risk of DM in patients with appendiceal malignant neoplasms, which is crucial for the early identification of high-risk patients and subsequent clinical decision-making. A novel and interpretable risk model for predicting distant metastasis in appendiceal malignant neoplasms: A retrospective study based on the SEER database Zhou jinhong 1 , Xie xiaojun 1 * 1.Department of General Surgery, the First Affiliated Hospital of Shantou University Medical College, Shantou, 515000, China. * Correspondence: Xie xiaojun( [email protected] ) Abstract Background Timely identification and treatment can greatly enhance the prognosis of patients with distant metastases(DM). This study aims to reveal the risk factors for DM in appendiceal malignant neoplasms patients, and establish a machine learning (ML) model for predicting the risk of DM. Methods A total of 9,376 patients with appendiceal malignant neoplasms, categorized based on the AJCC 8th TNM staging system, were chosen for the study from the Surveillance, Epidemiology, and End Results (SEER) database. Building on this, we developed four machine learning algorithm models along with a Nomogram model. The model was assessed based on confusion matrix, receiver operating characteristic (ROC) curve AUC, calibration curve analysis, and decision curve analysis (DCA). Additionally, the relationship between clinical pathological features and target variables was explored using the SHAP algorithm based on the optimal model. To assess its performance and generalizability, we validated the best model using 52 cases of appendiceal malignant neoplasms from the First Affiliated Hospital of Shantou University Medical College, China. Results Univariate logistic regression analysis and multivariable logistic regression analysis suggested that gender, histological type, grade, T stage, N stage, CEA level, tumor size and distant lymph nodal metastasis were risk factor for DM, while age and race may be related to DM rather than independent risk factors. Five models were constructed incorporated the clinical features. The XGBoost model demonstrated the best performance, achieving an AUC of 0.9917 in the training group and 0.9738 in the internal validation group, respectively. The accuracy was higher than 0.9 in both cohorts. Furthermore, the XGBoost model was evaluated in the outer-validation group, achieving an accuracy of 0.8654 and an AUC of 0.8792. Both the DCA and calibration curves further supported its robust predictive capability. Conclusions Our model was capable of accurately predicting the risk of DM in patients with appendiceal malignant neoplasms, which is crucial for the early identification of high-risk patients and subsequent clinical decision-making. Keywords Machine learning, appendiceal malignant neoplasms, distant metastasis, SEER database Introduction Primary appendiceal neoplasms are rare but highly lethal diseases, with an increasing incidence observed in recent years[1,2]. Appendiceal mucinous neoplasms(AMN), characterized by mucinous epithelial proliferation with extracellular mucin secretion, represent the most common type of appendiceal neoplasms. These neoplasms can lead to the spread of tumor cells to the peritoneum, omentum, and the surfaces of other pelvic organs. Appendiceal neuroendocrine tumors(NETs) are also among the more common malignant neoplasms of the appendix. These neoplasms are similarly aggressive and often present with nonspecific or absent symptoms during the early stages. By the time of diagnosis, they are frequently associated with liver or lymph node metastases. In summary, malignant neoplasms of the appendix are aggressive and challenging to detect in the early stages. Consequently, they are often diagnosed at an advanced stage, leading to a significant decline in survival rates[3–8]. Although the incidence of appendiceal malignant neoplasms is relatively low, the rate of distant metastasis(DM) is often underestimated. In a study based on public databases, approximately 30.4% of patients with appendiceal adenocarcinoma were found to have DM at the time of surgery[9]. The DM rate of NETs is also significant, reaching up to 10%[10]. In patients with high grade appendiceal neoplasms, peritoneal metastasis is even more common than lymph node metastasis[11]. In a retrospective study, the incidence of DM in patients with non-mucinous tumors was as high as 73.8%. The peritoneum, liver, and lungs were the most common sites of metastasis, in that order. Their 5-year survival rate was less than 30%[12]. For the rarer appendiceal signet ring cell carcinoma(SRCC) and goblet cell carcinoma(GCC), the most common mode of DM is peritoneal spread, affecting the peritoneum, omentum, bladder, ovaries, and other pelvic and abdominal organs. These highly aggressive neoplasms often exhibit an dormant growth pattern and are frequently diagnosed at an advanced stage[13,14]. Appendiceal malignancies are often curable through appendectomy alone. However, for patients with perforation, DM, or unknown metastatic status, the surgical approach should be modified to include a right hemicolectomy. For patients with unresectable neoplasms, cytoreductive surgery (CRS) is performed to reduce tumor burden, followed by postoperative heated intraperitoneal chemotherapy (HIPEC) to improve outcomes. Even for some advanced patients with certain pathological types, only palliative chemotherapy is available as a treatment option[13–15]. The 2020 Chicago Consensus recommended that patients with low-grade mucinous appendiceal tumors or low-grade mucinous carcinoma of appendiceal origin with peritoneal involvement, who are candidates for surgery, should undergo CRS followed by HIPEC. In cases of advanced patients with unresectable or inoperable neoplasms, systemic therapy following the treatment protocols for mucinous colorectal carcinoma may be an option. Systemic chemotherapy can be administered either preoperatively, postoperatively, or as a combination of both for patients eligible for CRS and HIPEC[16]. Specifically, peptide receptor radionuclide therapy (PRRT) or everolimus may also be a choice for those with advanced non-secretory appendiceal tumors[6]. Unfortunately, even with adequate treatment, the 5-year survival rate for patients with metastases remains frustratingly low[17]. Prior to appendectomy, most patients undergo ultrasound or computed tomography (CT) scanning to confirm the diagnosis. However, even CT imaging rarely provides a reliable preoperative diagnosis of appendiceal tumors. Appendiceal-derived peritoneal metastases are often incidentally discovered during appendectomy or diagnostic laparoscopy[16]. Building on the above discussion, preoperative identification of risk factors for DM is crucial. Unfortunately, large-scale studies investigating the risk factors for distant metastases in appendiceal malignant neoplasms are currently lacking. Therefore, the aim of this study is to investigate the risk factors for DM in appendiceal malignant neoplasms using large-scale data from public databases and to develop a clinical prediction model to assess the potential risk of DM of appendiceal malignant neoplasms. 2. Materials and methods 2.1. Data sources and study population The Surveillance, Epidemiology, and End Results (SEER) database is among the largest and most thoroughly researched population-based cancer registries, offering a wealth of reliable data. Data were obtained from three distinct datasets, namely the ”SEER Research Data, 17 Registries, Nov 2023 Sub (2000–2021),” ”SEER Research Data, 17 Registries, Nov 2022 Sub (2000–2020),” and ”SEER Research Data, 17 Registries, Nov 2021 Sub (2000–2019).” These datasets were sourced from SEER database, which provides comprehensive cancer incidence and survival data. The data extraction and were performed using SEER*Stat version 8.4.3, a specialized software designed for the of SEER data. Cases diagnosed with appendiceal malignant neoplasms after 2018 were selected for analysis. The pathological types included were categorized as follows: neuroendocrine tumors (NETs) (ICD-O-3: 8240, 8244), mucinous adenocarcinoma (MAC) (ICD-O-3: 8480-8481), non-mucinous adenocarcinoma (NMAC) (ICD-O-3: 8140, 8255), signet ring cell carcinoma (SRCC) (ICD-O-3: 8490), neuroendocrine carcinomas (NECs) (ICD-O-3: 8013, 8246), and goblet cell carcinoid (GCC) (ICD-O-3: 8243, 8245). These classifications were based on the International Classification of Diseases for Oncology, Third Edition (ICD-O-3) codes. The exclusion criteria were as follows: (1)patients with unknown T stage or Tx/T0/Tis; (2)patients with unknown M stage or M1(do not further clarify whether it is M1a or M1b or M1c stage); (3)patients with borderline carcinoembryonic antigen(CEA) level; (4)patients with unknown tumor size; (5)patients with unknown distant nodal metastasis; (6)patients with more than one primary one primary neoplasm sites; (7)patients with repeated ID number. Fig.1(A) illustrates the process of enrolling patients. For outer-validation, we used data from 52 patients with pathological diagnosis diagnosed appendiceal neoplasms in the First Affiliated Hospital of Shantou University Medical College of China. The study was retrospective and an ethics committee approval was granted (Ethics Numbers: B-2024-277). According to the 8th edition of the American Joint Committee on Cancer (AJCC) TNM staging system, M1a is defined as the presence of intraperitoneal acellular mucin, despite extensive examination. In contrast, M1b is characterized by peritoneal metastasis, including the presence of neoplasms cells within the mucin[18]. Research has demonstrated that patients with appendiceal neoplasms presenting with intraperitoneal acellular mucin have significantly more acceptable progression risk, recurrence risk, and overall survival compared to those with cellular metastasis to the peritoneum[19–23]. Therefore, M1b and M1c were identified as DM, while M0 and M1a were non-distant metastasis(NDM) in this study. 2.2. Risk factor screening and model establishment Statistical analysis was performed using SPSS software (version 26.0; IBM Corporation). Univariate logistic regression analysis was conducted to initially identify clinical characteristics linked to DM. Furtherly，risk factors for DM was screened through multivariable logistic regression analysis(P<0.05). Model for predicting the risk of DM was established through RStudio(version 4.3.1) and Python(version 3.11). The dataset after the sampling process was divided into a randomly training group and an inner- validation group at a ratio of 7:3. The nomogram was constructed using above selected risk factors via the “rms” R package. Simultaneously, four novel machine learning model was constructed, including random forest (RF), K-nearest neighbor (KNN), eXtreme gradient boosting(XGBoost) and support vector machine (SVM) via “sklearn” Python package. Random Forest (RF) is a machine learning algorithm that tackles classification and regression tasks by building multiple decision trees. The K-Nearest Neighbors (KNN) algorithm is a key classification method in supervised machine learning, commonly used in pattern recognition and data mining. XGBoost is a well-established decision tree algorithm applied to classification or regression prediction models. Support Vector Machine (SVM) is a binary classifier that effectively divides objects with multidimensional attributes into two distinct categories. In the construction of four novel machine learning models, Grid Search and 10-fold cross-validation were employed to optimize a series of hyperparameters. The performance of the models was primarily assessed by calculating the accuracy, precision, specificity, negative predictive value (NPV), recall, and F1 score for both the training group samples and the inner-validation group samples. Meanwhile, the area under the Receiver Operating Characteristic (ROC) curve(AUC) for each models was calculated. The model with the highest AUC is ultimately selected as the best model. Additionally, Calibration curves were constructed using mean absolute deviation and validated with 1000 bootstrap resamples to appraise the accuracy of the selected best model. Ultimately, Decision Curve Analysis (DCA) was employed to evaluate the net benefit of the model across various risk thresholds. The outer-validation group was utilized to evaluate the extrapolation and stability of the models. The performance metrics, including accuracy, precision, specificity, negative predictive value (NPV), recall, F1 score, and AUC were calculated. Furthermore, Calibration curves and DCA were constructed to provide a comprehensive evaluation of the models in the outer-validation group. Figure 1(B) illustrates the design of this study. In recent years, the Shapley Additive Explanation (SHAP) has been proposed to address the challenge of interpretability in various machine learning ”black box” models. SHAP is a machine learning interpretation method based on the concept of Shapley values from cooperative game theory. It employs an additive approach to calculate the contribution of each feature to the model’s prediction outcomes. This method provides not only local explanations for individual predictions, but also global explanations for the overall model behavior. Meanwhile，it also provides a suite of visualization tools to intuitively display the impact of each feature on individual data points and the overall feature importance distribution across the dataset[24]. In this study, “shap” Python package were utilized to compute the SHAP values of each features for the selected best model, followed by visualization. 3.1. Basic characteristics of patients This study included a total of 9,376 patients diagnosed with malignant appendiceal neoplasms. Of these, 1,100 (11.7%) had DM of neoplasms cells, while 8,276 (88.3%) did not have neoplasms cell dissemination beyond the primary site. Among patients with DM of neoplasms cells, 849 (77.2%) were aged 50 years or older, 636 (62.1%) were female, and 885 (80.5%) were white. In contrast, among patients without DM, 3,642 (44.0%) were aged 50 years or older, 4,761 (57.5%) were female, and 6,929 (83.7%) were white. Table 1 showed a comparison of appendiceal neoplasms via univariate logistic regression analysis. 3.2. Analysis of DM risk factors With respect to patient factors, there was significant difference in age, gender, race, histological type, grade, T stage, N stage, CEA level, tumor size and distant lymph node metastasis when compared DM patients to NDM patients, through the univariate logistic regression analysis (Table 1). Furthermore, multivariable logistic regression identified gender, histological type, grade, T stage, N stage, CEA level, and tumor size as risk factors for DM of neoplasms cells (Figure 2). Among these, male (OR = 0.569, 95% CI 0.472–0.686, p < 0.001) was found to be an independent protective factor against DM. When using patients with NETs as the reference, MAC (OR = 4.287, 95% CI 3.085–5.985, p < 0.001), NMAC (OR = 2.070, 95% CI 1.474–2.908, p < 0.001), and SRCC (OR = 2.780, 95% CI 1.721–4.489, p 0.05). Similarly, patients with Grade III (OR = 1.526, 95% CI 1.137–2.048, p = 0.005), T4 stage (OR = 14.232, 95% CI 8.880–22.809, p < 0.001), N1 stage (OR = 2.295, 95% CI 1.809–2.911, p < 0.001), N2 stage (OR = 4.117, 95% CI 2.968–5.712, p < 0.001), elevated CEA levels (OR = 2.818, 95% CI 2.146–3.701, p = 2 cm in diameter (OR = 2.065, 95% CI 1.588–2.684, p 0.05). 3.3. Model construction and evaluation All patients were randomly divided into a training group (n = 6563) and an inner-validation group (n = 2813) in a 7:3 ratio. No significant differences were observed between the two groups in terms of basic demographic and pathological characteristics, as shown in Table 2. A nomogram and four novel machine learning models, including RF, KNN, XGBoost, and SVM, were developed to predict the risk of DM in patients with malignant appendiceal neoplasms. The constructed Nomogram was shown in Figure3. The comparison of accuracy, precision, specificity, NPV, recall, F1 score, and AUC for the five models in the training group and inner-validation group is presented in Table 3. Among all five models, the AUC in the training group exceeded 0.7, while the AUC in the inner-validation group was also above 0.7. Hereinto, the performance of XGBoost was preferable to other four models with AUC of 0.9917 in training group. In inner-validation group, the performance of XGBoost was also the highest with AUC of 0.9738. The XGBoost model also achieved favorable scores in accuracy, precision, specificity, NPV, recall, and F1 in both datasets. The RF was second only to the appeal model with AUC of 0.9914 in training group and 0.9713 in inner-validation group，while scored observably in other indicators. However, the XGBoost model slightly outperformed the RF model in AUC across both datasets. Therefore, XGBoost was selected as the most optimal model among all five. The ROC of four novel machine learning for both data sets as shown in Fig.4. The predictive ability of the constructed XGBoost model was then assessed through Calibration curves. The Calibration curve showed a negligible discrepancy between the actual and predicted probabilities of DM in both training and inner-validation groups (Figure 5A, B). The findings suggest that the selected XGBoost model demonstrates high precision in forecasting DM in patients with appendiceal malignant neoplasms. To assess the clinical practicability of the model, we introduced the DCA curve. Throughout the entire spectrum of practical risk thresholds (spanning 0 to 0.9), the XGBoost model consistently demonstrated superior overall net benefit also in both data sets (Figure 6A, B). 52 Patients diagnosed with appendiceal malignancy neoplasms from 2018 to 2024 at the First Affiliated Hospital of Shantou University Medical College of China was enrolled for outer-validation group. Among them, 34 (65.4%) were female, 16 (30.8%) were aged <50 years, 45 (86.5%) were diagnosed with MAC, 6 (11.5%) were diagnosed with NMAC, and 1 (1.9%) were SRCC. Regarding tumor grade, 5 (9.6%) were Grade II, and 9 (17.3%) were Grade III. For tumor stage, 17 (32.7%) were at T2 stage, 6 (11.5%) at T3 stage, and 11 (21.2%) at T4 stage. Additionally, 2 (3.8%) had N1 stage, 4 (7.7%) had N2 stage, and 9 (17.3%) showed elevated CEA levels. Tumor size was ≥2 cm in 26 (50%) cases, 2 (3.8%) had distant lymph node metastasis, and 12 (23.1%) eventually developed DM of neoplasms cells. Through the above selected XGBoost model, the AUC was 0.8792 in outer-validation group (Figure 7A). Analogously, the model demonstrated superior generalization performance with an accuracy of 0.8654, precision of 0.7273, specificity of 0.925, NPV of 0.9024, recall of 0.6667, and an F1 score of 0.6957. Additionally, the model exhibited a slight difference between actual and predicted probabilities in the calibration curve and demonstrated superior overall benefit in the DCA curve (Figure 7B, C). The SHAP algorithm was applied to perform interpretability analysis on the selected XGBoost model, and a SHAP summary plot was generated. The summary plot visualized the impact of clinical features on the output of the XGBoost model (Figure 8A, B). Figure 8A illustrates the distribution of SHAP values for all clinical features: each dot represents a sample under the corresponding feature. The position of the dot indicates the SHAP value of the feature, with the value’s sign and magnitude representing the direction and extent of the feature’s influence on the model’s output. The more linearly separable the red and blue dots are, the more important the clinical feature was. Figure 8B presents a bar chart where clinical features are ranked in descending order based on their mean absolute SHAP values. Features ranked higher contribute more significantly to the prediction of DM. Based on the SHAP summary plot, SHAP dependency plots for the top 5 clinical features were generated to further explain their influence on DM in neoplasms cells (Figure 9A–E). The SHAP dependency plots feature the SHAP value of a clinical characteristic on the vertical axis and the range of the characteristic on the horizontal axis. Each dot represents a sample for the corresponding feature. Within a specific range of the clinical feature, larger SHAP values indicate a greater contribution to the increased risk of DM. 4. Discussion Malignant appendiceal neoplasms account for approximately 0.5% of gastrointestinal tumors, with an incidence rate of about 1.7 per 10,000. Moreover, with changes in lifestyle and advancements in diagnostic techniques, the incidence of these neoplasms has been steadily increasing in recent years[1,2,15,25]. The incidence of DM in appendiceal neoplasms cells is relatively high. Studies have reported that approximately 10%–30% of patients with malignant appendiceal neoplasms develop DM of malignant cells[9,10]. Among these, peritoneal dissemination is the most common, followed by metastases to the liver, lungs, and bones, with a 5-year survival rate of less than 30%[12]. Unlike colorectal cancer, MAC accounts for approximately 90% of all appendiceal epithelial cancers and is associated with a poor prognosis. In comparison, NMAC, SRCC, and GCC have an even worse prognosis. Lymph node metastasis is also highly prevalent among patients with malignant appendiceal neoplasms. Studies have reported lymph node metastasis rates as high as 20%, with mucinous adenocarcinoma accounting for the highest proportion[26,27]. In certain circumstances, peritoneal metastases are more common than lymph node metastases. A 2017 study by Mehta reported that postoperative pathology indicated peritoneal metastases were more frequent than lymph node metastases (57% vs. 15%) in patients with poorly differentiated malignant appendiceal neoplasms[28]. Simple malignant appendiceal neoplasms can be cured through complete appendectomy. However, for patients with perforation, peritoneal dissemination, or metastases to other organs, right hemicolectomy combined with HIPEC is required. Although current evidence is lacking to demonstrate the benefits of chemotherapy in patients with advanced malignant appendiceal neoplasms, systemic therapy based on the treatment protocols for mucinous colorectal cancer can be considered for those with extensive metastases. The 2020 Chicago guidelines state that for advanced patients who cannot undergo complete resection or are not suitable for surgery, systemic therapy based on the treatment protocols for mucinous colorectal cancer can be used. Furthermore, for patients eligible for CRS and HIPEC, neoadjuvant or adjuvant chemotherapy can also be considered[16]. Whether in terms of treatment plan selection or preoperative preparation, the unknown status of DM in neoplasms cells poses a significant challenge for surgeons. Early or preoperative identification of DM not only helps surgeons decide on the most appropriate treatment plan, but also allows patients to receive timely therapy, thereby delaying disease progression and extending survival. Unfortunately, routine imaging techniques, including ultrasound and CT scans, often struggle to detect small metastatic lesions scattered within the peritoneum or other organs, and these metastases are more commonly discovered incidentally during surgery[16]. Therefore, early identification of high-risk patients is particularly important. Unfortunately, current research on malignant appendiceal neoplasms has focused more on evaluating and predicting lymph node metastasis risk, and there is a lack of studies addressing DM, including those in the peritoneum, liver, lungs, and bones, which are far from the primary site[4,10,27,29–35]. To the best of our knowledge, this study is the first to explore the risk factors for DM of malignant appendiceal neoplasms cells using a population-based database. Univariate logistic regression and multivariable logistic regression analyses both showed that gender, histological type, grade, T stage, N stage, CEA level, and tumor size are significant factors associated with DM of neoplasms cells. Fisher exact test was used since the variable of distant lymph nodal metastasis has a scale of 0 in a NDM group，which showed distant lymph nodal also a dependence risk factor. Age and race, with P values 0.05 in multivariable analysis, suggest that both variables may be associated with DM of neoplasms cells, but they are not independent risk factors. Patients with neoplasms cells disseminating from the primary site to the peritoneal cavity and other distant organs often experience a significant decline in survival rates. Although only a limited number of studies have investigated the risk factors for DM in malignant appendiceal neoplasms, several related studies support the findings of this research. A retrospective study by Xie X et al found that advance age, T stage, N stage, Grade and tumor size were correlated with poor survival, while gender and race weren’t statistical different[9]. In our study, age >=50 years old (OR=4.304, 95% CI 3.717-4.987, p<0.001) was identified as a significant risk factor in univariate logistic regression analysis. Additionally, T4 stage (OR=14.232, 95% CI 8.880-22.809, p<0.001), N1 stage (OR=2.295, 95% CI 1.809-2.911, p<0.001), N2 stage (OR=4.117, 95% CI 2.968-5.712, p<0.001), Grade III (OR=1.526, 95% CI 1.137-2.048, p=2 cm (OR=2.065, 95% CI 1.588-2.684, p<0.001) were identified as independent risk factors for DM in multivariable logistic regression analysis. These findings are consistent with the results reported by Xie et al, further supporting the reliability of our conclusions. Moreover，fifty three patients developed distant lymph node metastasis, all of whom were in the DM group, illustrating the relationship with the outcome in this study using Fisher’s exact test. Similar conclusions can also be found in the study by Shiota et al. Patients diagnosed with GCC merged with advanced T/N stage, high-grade tumors, and lymphovascular invasion were more likely to develop peritoneal metastasis of neoplasms cells[7]. A study of Shyu S et al displayed that Grade III accounted for 73% in total goblet cell adenocarcinoma patients with peritoneal metastasis. The survival period was considerably shortened whether in Grade III GCC patients (Grade III vs Grade I/II: 33 vs 98 months) or in Grade III MAC patients(Grade III vs Grade I/II: 49 vs 204/81 months)[8]. This study indicated that patients with Grade III neoplasms have a 1.5-fold increased risk of DM compared to those with Grade I neoplasms, while Grade II was not considered a risk factor. Tumor size was also a key characteristic whether for DM or prognosis. The growth rate of tumor size in appendiceal malignant neoplasms affects clinical prognosis[3]. Gahagan et al. reviewed the National Cancer Database, which included 3,402 patients with appendiceal adenocarcinoma. Their study revealed that larger tumor size was associated with higher disease progression rates. However, this study did not investigate the relationship between tumor size and staging systems with peritoneal seeding or metastasis to other distant organs[32]. The relationship between tumor size and DM in appendiceal cancer has not been extensively studied compared to other malignancies such as those in the biliary tract, stomach, small intestine, colorectum, pancreas, thyroid, and breast[36–41]. In our study, multivariable logistic regression analysis revealed that patients with a tumor size of >=2 cm had a 2.1-fold increased risk of DM compared to those with smaller size. This finding underscores the significant impact of tumor size on the metastatic potential of appendiceal malignancies. Different pathological types of appendiceal malignant neoplasms seem to behave differently in terms of DM and prognosis. To date, few studies have directly elucidated the relationship between different pathological types of appendiceal malignant neoplasms and the risk of DM. Patients with DM generally have poor outcomes. A study by Mo S et al. analyzed 7,170 patients with appendiceal malignant neoplasms from the SEER database, comparing clinical and pathological data to explore risk factors for disease progression. The study found that SRCC had the shortest median survival time at 29 months, followed by NMAC at 61 months, MAC at 87 months, GCC at 162 months, and NETs had the best prognosis with a median survival time of 396 months[1]. In our study, using NETs as the reference category, multivariable logistic regression analysis revealed that MAC was associated with a 4.287 fold increased risk of DM (95% CI 3.085–5.958, p < 0.001). NMAC had a 2.070 fold increased risk (95% CI 1.474–2.908, p < 0.001), and SRCC had a 2.780 fold increased risk (95% CI 1.721–4.489, p 0.05). These findings highlight the distinct metastatic potential of different pathological types of appendiceal malignant neoplasms. Compared with NETS, NECs and GCC, MAC, NMAC and SRCC were more likely to metastasize far away, which related to poor prognosis. From this point of view, the conclusions of this study and the above article do not seem to conflict. CEA is a tumor marker, and studies have shown that pre-treatment levels of CEA are associated with the metastasis of appendiceal malignant neoplasms. In some patients with progressive appendiceal mucinous adenocarcinoma, elevated levels of CEA are often observed. Patients with elevated CEA levels tend to have a poorer prognosis compared to those with normal levels[42]. Kyang LS et al. reported that elevated preoperative CEA levels were associated with a 6.5-fold higher risk of developing peritoneal disease after treatment[43]. Higher CEA levels were found in patients with appendiceal neoplasms who had poorer prognosis. Aziz O et al. reported that in patients with appendiceal cancer and peritoneal metastasis, those with elevated CEA levels had a 5-year survival rate of less than 20%, whereas patients with CEA levels below 6 ng/mL had a 5-year survival rate exceeding 60%. Meanwhile, patients with a Peritoneal Cancer Index (PCI) score of 7 or higher had a 5-year survival rate of approximately 30%, while those with a PCI score below 7 had a survival rate exceeding 80%[17]. This relationship was also seen for CEA and in the Netherlands cohort[44]. Analysis in our study suggested that elevating in CEA level is an independent risk factor for DM (OR=2.818, 95%CI 2.146-3.701, p<0.001). Currently, most conventional statistical methods used are regression models, which assume a linear relationship between variables and outcomes[45]. However, only a limited subset of variables and outcomes actually exhibit a linear relationship within the model. In recent years, significant advancements in data processing technologies, particularly artificial intelligence algorithms, have offered novel approaches for analyzing clinically relevant issues and constructing predictive models. achine learning algorithms have been extensively utilized to construct clinical prediction models across various research types. These algorithms offer several advantages, such as preventing overfitting and effectively handling imbalanced data [46]. Machine learning is capable of extracting valuable information from extensive datasets, utilizing this information to construct mathematical models, and subsequently validating the performance of these models in novel datasets[47]. Machine learning has found extensive application in the medical field, encompassing disease diagnosis, treatment planning, and prognosis assessment. As data volume increases and algorithms improve, machine learning is anticipated to assume an even more critical role in medicine. According to the SEER database, four novel machine learning models and Nomogram were built to predict the risk of DM in appendiceal malignant neoplasms patients. We further evaluated the five algorithm models through confusion matrix including accuracy, precision, specificity, NPV, recall, F1 score, and AUC value, among which XGBoost has a good prediction in training group (AUC = 0.9917), higher than the rest four models. Continuously, XGBoost performed best in inner-validation groups (AUC = 0.9738). The XGBoost consistently demonstrated reliability and superior overall net benefit in both data sets via Calibration curves analysis and DCA. This model is excellent and unmatched in predicting the risk of DM using the SEER database.此外，52 appendiceal malignancy neoplasms patients outside the SEER database were used to verify the robustness and generalization performance, which showed superiority in predicting the risk of DM(AUC=0.8792). Machine learning prediction models have emerged as valuable tools for clinical disease diagnosis and prognosis assessment. However, their application in clinical practice is often limited due to the ”black box” problem, which refers to the difficulty in understanding the internal workings and decision-making processes of these models. This lack of transparency can lead to challenges in identifying biases or errors, and may undermine trust from clinicians who rely on these models for critical decision-making. As a result, the development and implementation of interpretable machine learning models have become essential to address these concerns. The SHAP algorithm, based on cooperative game theory, was applied to interpret the selected XGBoost model. The results revealed that the six most influential clinical features contributing to the outcome, in descending order, were T stage, histology type, N stage, tumor size, CEA level, and Grade. The explanation was consistent with univariate logistic regression and multivariable logistic regression analysis, which further proved the reliability of the analysis and constructed XGBoost model. It is worth noting that in the 8th and 9th editions of the AJCC staging manual, peritoneal involvement by acellular mucin is designated as M1a, while peritoneal deposits containing mucinous epithelium are designated as pM1b. However, in 2016, the Peritoneal Surface Oncology Group International (PSOGI) introduced a consensus terminology that suggests using the term ”low-grade mucinous carcinoma peritonei” for grade 1 lesions and ”high-grade mucinous carcinoma peritonei” for grades 2 and 3 lesions. The term ”mucinous adenocarcinoma” should be specifically used for mucinous neoplasms that exhibit infiltrative growth and desmoplasia, or for clusters of tumor cells suspended in small pools of mucin[18]. In some other staging systems, such as the Union Internationale Contre le Cancer (UICC) stage and the Japanese Society for Cancer of the Colon and Rectum (JCCRC) stage, peritoneal involvement by acellular mucin is not yet incorporated into the M1 stage[15,48]. Studies have reported that in patients with appendiceal MAC, those with peritoneal involvement by mucin containing epithelial cells had a disease progression rate of 53% over a median follow-up period of 4 years, while those with only acellular mucin had a progression rate of 1%. The five-year mortality rate was 54% for the former and 0% for the latter[21]. Even after adequate treatment, the recurrence rate remains as high as 26–44%, and the median survival period is less than 36 months for those with appendiceal malignant neoplasms and peritoneal metastasis[43]. Baratti et al. compared 265 patients with pseudomyxoma peritonei (PMP) and reported that those with intraperitoneal acellular mucin had a 10-year survival rate of approximately 90% after treatment. In contrast, patients with low-grade appendiceal neoplasms and peritoneal implants had a 10-year survival rate of 63%, while those with high-grade neoplasms had a 10-year survival rate of 40.1%. Notably, patients with signet ring cell carcinoma and peritoneal implants had a 10-year survival rate of 0% after treatment[23]. Therefore, in our study, patients with intraperitoneal acellular mucin were not considered to have DM but were instead classified into the non-metastasis group. This classification aligns with the findings that patients with acellular mucin have significantly better outcomes compared to those with cellular involvement. This study is the first to leverage the large sample size characteristic of a public database to investigate the risk factors for DM in appendiceal malignant neoplasms and to construct an interpretable machine learning model for predicting the risk of DM. Additionally, an outer-validation cohort was utilized to guarantee the reliability, robustness, and extrapolation ability of the constructed model. This study also has a few limitations: 1) The outer-validation cohort in this study was derived from single-center data, which included a relatively small number of patients, all of whom were Asian. The most prevalent histological types in this cohort were MAC and NMAC. In contrast, the sample sizes for SRCC, GCC, NETs, and NECs were insufficient. Thus, additional patient data from various hospitals will be required to confirm the diagnostic effectiveness and generalizability of our model. 2) The SEER database is deficient in critical information, including tumor family history and tumor markers other than CEA, which might also serve as significant predictors and prognostic indicators of DM. In the future, it is possible to incorporate additional risk factors associated with metastasis. 5.Conclusion This study explored the risk factors for appendiceal malignant neoplasms distant metastasis. Furthermore, utilizing machine learning algorithms, a predictive model was constructed and validated, which leverages clinical characteristics to quantify the primary factors contributing to DM. Among them, T stage, histology type, N stage, tumor size, CEA level and Grade are the top six most important factors for distant metastasis in appendiceal malignant neoplasms patients. Compared to the traditional Nomogram model, the XGBoost algorithm demonstrates superior predictability. As a result, it is capable of offering personalized treatment options and facilitating a more efficient allocation of medical resources for patients. Acknowledgements The authors have nothing to report. Conflicts of Interest The authors declare no conflicts of interest. Data Availability Statement The data that support the findings of this study are available from the corresponding author upon reasonable request. Reference 1. Mo S, Zhou Z, Ying Z, Dai W, Xiang W, Han L, et al. Epidemiology of and prognostic factors for appendiceal carcinomas: a retrospective, population-based study. Int J Colorectal Dis. 2019;34:1915–24. 2. Marmor S, Portschy PR, Tuttle TM, Virnig BA. The rise in appendiceal cancer incidence: 2000-2009. J Gastrointest Surg. 2015;19:743–50. 3. Asano Y, Yamamoto N, Hayashi K, Takeuchi A, Miwa S, Igarashi K, et al. Impact of Tumor Growth Speed of Primary Lesions on the Clinical Outcomes of Appendicular Skeletal Metastases. Anticancer Research. 2022;42:229–36. 4. Wang D, Liu C, Yan T, Li C, Güngör C, Yang Q, et al. A Nomogram for Predicting Lymph Nodal Metastases in Patients with Appendiceal Cancers: An Analysis of SEER Database. J Invest Surg. 2021;34:924–30. 5. Arshi J, Peter AD, Liang Y, Hao Y. Confined Low Grade Appendiceal Mucinous Neoplasm With Coexisting Distant Metastasis. In Vivo. 2024;38:295. 6. Padmanabhan Nair Sobha R, Jensen CT, Waters R, Calimano-Ramirez LF, Virarkar MK. Appendiceal Neuroendocrine Neoplasms: A Comprehensive Review. Journal of Computer Assisted Tomography. 2024;48:545. 7. Shiota T, Murata K, Kishimoto M, Yao T, Noura S, Morita S, et al. Clinicopathological features of appendiceal goblet cell adenocarcinoma in Japan: a multicenter retrospective study. Surg Today. 2023;53:174–81. 8. Shyu S, Choudry H, Hall L, Pingpank J, Holtzman M, Bartlett D, et al. Clinicopathological analysis of appendiceal goblet cell adenocarcinoma with peritoneal metastasis: World Health Organization grade predicts survival following cytoreductive surgery with intraperitoneal chemotherapy. Histopathology. 2020;77:798–809. 9. Xie X, Zhou Z, Song Y, Li W, Diao D, Dang C, et al. The Management and Prognostic Prediction of Adenocarcinoma of Appendix. Sci Rep. 2016;6:39027. 10. Tsoukalas N, Galanopoulos M, Tsapakidis K, Karamitrousis E, Kamposioras K. Predicting Lymph Nodal Metastases in Patients with Appendiceal Cancers. J Invest Surg. 2021;34:931–2. 11. Mehta A, Mittal R, Chandrakumaran K, Carr N, Dayal S, Mohamed F, et al. Peritoneal Involvement Is More Common Than Nodal Involvement in Patients With High-Grade Appendix Tumors Who Are Undergoing Prophylactic Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy. Dis Colon Rectum. 2017;60:1155–61. 12. Uemura M, Qiao W, Fournier K, Morris J, Mansfield P, Eng C, et al. Retrospective study of nonmucinous appendiceal adenocarcinomas: role of systemic chemotherapy and cytoreductive surgery. BMC Cancer. 2017;17:331. 13. Fields AC, Lu P, Enzinger A, Goldberg J, Irani J, Bleday R, et al. Treatment patterns and outcomes in goblet cell carcinoid tumors of the appendix. Journal of surgical oncology. 2019;120:1096. 14. Lowenthal BM, Lin GY, Tipps AMP, Hosseini M. Adenocarcinoma Ex-Goblet Cell Carcinoid of the Appendix With Metastatic Peritoneal Spread to Meckel’s Diverticulum and Endometriosis. Int J Surg Pathol. 2017;25:623–8. 15. Köhler F, Matthes N, Rosenfeldt M, Kunzmann V, Germer C-T, Wiegering A. Neoplasms of the Appendix. Dtsch Arztebl Int. 2023;120:519–25. 16. Group CCW. The Chicago Consensus on peritoneal surface malignancies: Management of appendiceal neoplasms. Cancer. 2020;126:2525–33. 17. Aziz O, Jaradat I, Chakrabarty B, Selvasekar CR, Fulford PE, Saunders MP, et al. Predicting Survival After Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy for Appendix Adenocarcinoma. Dis Colon Rectum. 2018;61:795–802. 18. Umetsu SE, Kakar S. Staging of appendiceal mucinous neoplasms: challenges and recent updates. Hum Pathol. 2023;132:65–76. 19. Davison JM, Choudry HA, Pingpank JF, Ahrendt SA, Holtzman MP, Zureikat AH, et al. Clinicopathologic and molecular analysis of disseminated appendiceal mucinous neoplasms: identification of factors predicting survival and proposed criteria for a three-tiered assessment of tumor grade. Mod Pathol. 2014;27:1521–39. 20. Ballentine SJ, Carr J, Bekhor EY, Sarpel U, Polydorides AD. Updated staging and patient outcomes in low-grade appendiceal mucinous neoplasms. Mod Pathol. 2021;34:104–15. 21. Roxburgh CS, Fenig YM, Cercek A, Shia J, Rassam RM, Paty PB, et al. Outcomes of Low-Grade Appendiceal Mucinous Neoplasms with Remote Acellular Mucinous Peritoneal Deposits. Ann Surg Oncol. 2019;26:118–24. 22. Choudry HA, Pai RK, Shuai Y, Ramalingam L, Jones HL, Pingpank JF, et al. Impact of Cellularity on Oncologic Outcomes Following Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemoperfusion for Pseudomyxoma Peritonei. Ann Surg Oncol. 2018;25:76–82. 23. Baratti D, Kusamura S, Milione M, Bruno F, Guaglio M, Deraco M. Validation of the Recent PSOGI Pathological Classification of Pseudomyxoma Peritonei in a Single-Center Series of 265 Patients Treated by Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy. Ann Surg Oncol. 2018;25:404–13. 24. Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From Local Explanations to Global Understanding with Explainable AI for Trees. Nat Mach Intell. 2020;2:56–67. 25. Wang D, Ge H, Lu Y, Gong X. Incidence trends and survival analysis of appendiceal tumors in the United States: Primarily changes in appendiceal neuroendocrine tumors. PLoS One. 2023;18:e0294153. 26. Acs M, Gerken M, Zustin J, Blaj S, Isgandarova S, Piso P. Prolonged Survival in Peritoneal Metastatic Appendiceal Carcinoma Patients Treated With Combined Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy. J Surg Res. 2023;283:839–52. 27. Shannon AB, Goldberg D, Song Y, Paulson EC, Roses RE, Fraker DL, et al. Predictors of lymph node metastases in patients with mucinous appendiceal adenocarcinoma. J Surg Oncol. 2020;122:399–406. 28. Mehta A, Mittal R, Chandrakumaran K, Carr N, Dayal S, Mohamed F, et al. Peritoneal Involvement Is More Common Than Nodal Involvement in Patients With High-Grade Appendix Tumors Who Are Undergoing Prophylactic Cytoreductive Surgery and Hyperthermic Intraperitoneal Chemotherapy. Dis Colon Rectum. 2017;60:1155–61. 29. Ciarrocchi A, Pietroletti R, Carlei F, Amicucci G. Clinical Significance of Metastatic Lymph Nodes in the Gut of Patients with Pure and Mixed Primary Appendiceal Carcinoids. Dis Colon Rectum. 2016;59:508–12. 30. Groth SS, Virnig BA, Al-Refaie WB, Jarosek SL, Jensen EH, Tuttle TM. Appendiceal carcinoid tumors: Predictors of lymph node metastasis and the impact of right hemicolectomy on survival. J Surg Oncol. 2011;103:39–45. 31. Galanopoulos M, McFadyen R, Drami I, Naik R, Evans N, Luong TV, et al. Challenging the Current Risk Factors of Appendiceal Neuroendocrine Neoplasms: Can They Accurately Predict Local Lymph Nodal Invasion? Results from a Large Case Series. Neuroendocrinology. 2019;109:179–86. 32. Gahagan JV, Whealon MD, Phelan MJ, Mills S, Pigazzi A, Stamos MJ, et al. Lymph Node Positivity in Appendiceal Adenocarcinoma: Should Size Matter? J Am Coll Surg. 2017;225:69–75. 33. Kuhlen M, Kunstreich M, Pape U-F, Seitz G, Lessel L, Vokuhl C, et al. Lymph node metastases are more frequent in paediatric appendiceal NET ≥1.5 cm but without impact on outcome - Data from the German MET studies. Eur J Surg Oncol. 2024;50:108051. 34. Day RW, Chang Y-H, Stucky C-C, Gray R, Pockaj B, Wasif N. A Predictive Model for Nodal Metastases in Patients With Appendiceal Cancers. Ann Surg. 2021;274:155–61. 35. Mosquera C, Fitzgerald TL, Vora H, Grzybowski M. Novel nomogram combining depth of invasion and size can accurately predict the risk for regional nodal metastases for appendiceal neuroendocrine tumors (A-NET). J Surg Oncol. 2017;116:651–7. 36. Guo K, Lou Y, Zheng S. Predictors of Distant Metastasis and Prognosis in Newly Diagnosed T1 Intrahepatic Cholangiocarcinoma. Biomed Res Int. 2023;2023:6638755. 37. Yao L, Shao H, Zhang X, Huang X. A novel risk model for predicting peritoneal metastasis in colorectal cancer based on the SEER database. J Cancer Res Clin Oncol. 2023;149:15989–6000. 38. Kim TH, Han S-S, Park S-J, Lee WJ, Woo SM, Yoo T, et al. CA 19-9 level as indicator of early distant metastasis and therapeutic selection in resected pancreatic cancer. Int J Radiat Oncol Biol Phys. 2011;81:e743-748. 39. Asare EA, Silva-Figueroa A, Hess KR, Busaidy N, Graham PH, Grubbs EG, et al. Risk of Distant Metastasis in Parathyroid Carcinoma and Its Effect on Survival: A Retrospective Review from a High-Volume Center. Ann Surg Oncol. 2019;26:3593–9. 40. Ma W, Wang X, Xu G, Liu Z, Yin Z, Xu Y, et al. Distant metastasis prediction via a multi-feature fusion model in breast cancer. Aging (Albany NY). 2020;12:18151–62. 41. Yang K, Wu J, Xu T, Zhou Y, Liu W, Yang L. Machine learning to predict distant metastasis and prognostic analysis of moderately differentiated gastric adenocarcinoma patients: a novel focus on lymph node indicators. Front Immunol. 2024;15:1398685. 42. Wagner PL, Austin F, Sathaiah M, Magge D, Maduekwe U, Ramalingam L, et al. Significance of serum tumor marker levels in peritoneal carcinomatosis of appendiceal origin. Ann Surg Oncol. 2013;20:506–14. 43. Kyang LS, Alzahrani NA, Alshahrani MS, Rahman MK, Liauw W, Morris DL. Early recurrence in peritoneal metastasis of appendiceal neoplasm: Survival and prognostic factors. Eur J Surg Oncol. 2019;45:2392–7. 44. van Ruth S, Hart A a. M, Bonfrer JMG, Verwaal VJ, Zoetmulder F a. N. Prognostic value of baseline and serial carcinoembryonic antigen and carbohydrate antigen 19.9 measurements in patients with pseudomyxoma peritonei treated with cytoreduction and hyperthermic intraperitoneal chemotherapy. Ann Surg Oncol. 2002;9:961–7. 45. Bai B-L, Wu Z-Y, Weng S-J, Yang Q. Application of interpretable machine learning algorithms to predict distant metastasis in osteosarcoma. Cancer Med. 2023;12:5025–34. 46. Deo RC. Machine Learning in Medicine. Circulation. 2015;132:1920–30. 47. Scott IA. Demystifying machine learning: a primer for physicians. Intern Med J. 2021;51:1388–400. 48. Shida D, Kanemitsu Y, Hamaguchi T, Shimada Y. Introducing the eighth edition of the tumor-node-metastasis classification as relevant to colorectal cancer, anal cancer and appendiceal cancer: a comparison study with the seventh edition of the tumor-node-metastasis and the Japanese Classification of Colorectal, Appendiceal, and Anal Carcinoma. Jpn J Clin Oncol. 2019;49:321–8. Supplementary Material File (figure.docx) Download 2.31 MB File (table.docx) Download 620.22 KB Information & Authors Information Version history V1 Version 1 11 February 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Authors Affiliations Zhou jinhong 0009-0008-4410-8853 The First Affiliated Hospital of Shantou University Medical College View all articles by this author Xie xiaojun [email protected] The First Affiliated Hospital of Shantou University Medical College View all articles by this author Metrics & Citations Metrics Article Usage 228 views 111 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Zhou jinhong, Xie xiaojun. A novel and interpretable risk model for predicting distant metastasis in appendiceal malignant neoplasms: A retrospective study based on the SEER database. Authorea . 11 February 2025. DOI: https://doi.org/10.22541/au.173927358.81987754/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.173927358.81987754/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a024a6285c49300f',t:'MTc3OTg4MTAyMg=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();