Multimodal machine learning for staging laparoscopy: a combined image analysis and morphologic tool for the discrimination of peritoneal metastasis.

Staging laparoscopy (SL) plays a crucial role in staging abdominal malignancies, particularly those of gastrointestinal origin, including esophageal, gastric, colorectal, biliary tract, pancreatic, and ovarian cancer [ 1 – 6 ] . SL enables the detection of both gross and occult peritoneal metastasis (PM), allowing for biopsy, informed diagnostic and surgical planning [ 7 – 9 ] . SL improves detection compared to preoperative imaging; however, accurately assessing the full extent of PM remains challenging [ 6 , 7 , 9 ] . Preoperative imaging modalities are limited in sensitivity due to poor soft tissue contrast and small bowel movement, highlighting the need for improved diagnostic strategies [ 10 – 12 ] . A key challenge of SL is the macroscopic differentiation between malignant and benign lesions or scar tissue in suspected cases of PM[ 13 ]. Tumor recognition is subjective and influenced by intra-patient variability, surgeon expertise, tumor characteristics, and response to systemic treatment[ 8 ]. The advent of computer vision (CV) and artificial intelligence (AI) has introduced innovative diagnostic approaches in medical images. [ 14 – 17 ] AI-based CV models, particularly those integrating machine learning (ML), present promising avenues for improving diagnostic accuracy in complex scenarios [ 18 , 19 ] . Specifically, ML can be integrated into CV to address the limitations of clinician decision-making, especially when dealing with complex data[ 20 ]. ML models have demonstrated high diagnostic accuracy in distinguishing between benign and malignant skin lesions, aiding dermatologists in clinical practice [ 18 , 21 – 23 ] . Recent advancements, such as the proposal by Schnelldorfer et al for a computer-assisted staging laparoscopy (CASL) system, highlight the potential of AI-based technologies to enhance surgeon capabilities during SL, particularly for PM detection[ 24 ]. Recent efforts, such as the Artificial Intelligence Laparoscopic Exploration System (AiLES), developed for real-time segmentation of intra-abdominal metastases during gastric cancer surgery, highlight the growing role of AI in lesion detection[ 25 ]. However, existing ML models in the medical field primarily focus on image-based data from a single modality, which limits their diagnostic scope[ 14 ]. Although morphologic features like nodularity and transparency are promising correlates of PM, they lack reliability as standalone diagnostic predictors[ 13 ]. No study has leveraged a multimodal approach for intraoperative PM identification. Integrating surgeon-provided morphologic assessments with image-based data could provide a comprehensive view of peritoneal lesions, potentially improving diagnostic accuracy during SL. In alignment with current definitions and methodological frameworks for multimodal machine learning (MML), this study aimed to bridge this gap by developing and validating an MML model to enable reliable discrimination of peritoneal lesions during SL procedures [ 14 , 20 ] .

This study was approved by the Institutional Review Board (approval reference: BC09947) and conducted in accordance with the principles of the Declaration of Helsinki. The study is reported using the TRIPOD AI and TITAN reporting guidelines (Supplemental Digital Content, Data S2, available at: http://links.lww.com/JS9/F75 ) [ 26 , 27 ] and registered in the Open Science Framework (OSF) registry. All participants provided written informed consent for data acquisition, analysis, and publication. Videos from the prospectively maintained institutional database were screened between February 2020 and March 2023. The inclusion criteria were adults aged ≥18 years with abdominal malignancy who underwent SL for PM staging. Recording of the biopsy procedure during the surgery was required as an inclusion criterion. Exclusion criteria included age <18 years, lack of informed consent, and absence of recorded biopsy or pathology reports. Demographic data were extracted from the institutional database and verified through a chart review. As part of standard institutional practice, multiple biopsies were obtained from distinct peritoneal quadrants, including the right and left hypochondria and the right and left iliac fossae or flanks. Additional biopsies were obtained from other locations based on individual cases. Each biopsy was intraoperatively labeled with the specific location from which it was obtained, enabling the corresponding pathology report to be directly linked to the surgical phase. Pathology samples and reports for each biopsy were retrieved and re-reviewed by a second pathologist. This approach provides a robust framework for the analysis and correlation of surgical and pathological data. The dataset was split once at the patient level into training, validation, and test sets, and the same split was used for all models. All lesions and associated image frames from a given patient were assigned exclusively to one subset to avoid data leakage and allow proper evaluation on unseen patients. The videos were converted into a standardized MPEG-4 format and uploaded to the web-based CV platform Encord, developed by Cord Technologies Limited, London, UK. Temporal annotations of the surgical biopsy phase were performed by two trained annotators who labeled the frames as benign or malignant based on the corresponding pathology data. The surgical phase was defined as 300 frames preceding biopsy forceps closure. Within each temporal annotation, 20 still frames were randomly extracted using a Python-based script to ensure an unbiased selection. This protocol was applied consistently across all annotated frames. Frames in which the tumor was not visible or obscured were excluded. The corresponding still images reflected the surgical phase, with the lesion’s position and zoom determined by the surgeon’s camera during the biopsy procedure. No fixed imaging standard (e.g., lesion alignment or zoom) was enforced as the images followed the natural progression of the surgical act. Deep learning (DL) models were developed for binary classification of malignant versus benign lesions using 8-fold cross-validation for training and evaluation on an independent test set. Three oncologic surgeons, with clinical and research expertise in PM and its treatment, devised a structured checklist (Supplemental Digital Content, Data S1, available at: http://links.lww.com/JS9/F75 ) for the morphologic evaluation of each lesion based on the concept of expert-augmented ML[ 28 ]. The checklist was adapted from the work of Schnelldorfer et al and included assessments of shape, symmetry, surface texture, nodularity, borders, color spectrum, color transition, transparency, spatial relationship with vessels, and neovasculature[ 13 ]. Each feature was defined in a table of definitions containing sample images. Using this checklist, two oncologic surgeons independently evaluated the anonymized videos, and one still frame per lesion. Both surgeons were blinded to the pathology and patient information. Discrepancies were resolved through consensus meetings with a third surgeon. Six ML models – logistic regression, decision tree, random forest, gradient boosting, artificial neural network (ANN), and support vector machine – were developed to predict malignancy based on morphologic features. In particular, the ML models were constructed using 5-fold cross-validation, with hyperparameters optimized through a grid search, and subsequently evaluated on an independent test set. The MML model was constructed by integrating raw laparoscopic image data with structured morphologic features derived from expert annotations. This approach follows current definitions and methodological frameworks for multimodal learning, in which heterogeneous yet complementary data sources are combined through hybrid fusion strategies to enhance classification performance. In this setting, low-level visual features and high-level semantic descriptors were jointly utilized to generate a comprehensive representation of lesion characteristics, consistent with the frameworks proposed by Lu et al [ 29 ] and Zhu and Yuan [ 30 ] . After completing the training of the previous models, an MML model was developed to distinguish between malignant and benign lesions. The MML model integrates the predictions from the best-performing image-based DL model and the best-performing morphology-based ML model by combining imaging data with tabular morphologic features. Six models (logistic regression, decision tree, random forest, gradient boosting, ANN, and support vector machine) were trained using 5-fold cross-validation and optimized through hyperparameter tuning. The best-performing model from this set was selected as the final MML model. Thirteen oncologic surgeons independently evaluated each biopsied lesion in the test set to assess the predictive performance of the developed MML model. All participating surgeons were experts in surgical oncology at a high-volume academic center and routinely performed SL in their clinical practice. They were blinded to the pathology results and patient data. The evaluations were conducted in settings designed to mimic the operating room environment. Each surgeon reviewed a single frame per lesion and classified it as either malignant or benign. The flowchart of the study is shown in Figure 1 . Figure 1. Overview of model development process. Overview of model development process. Descriptive statistics were used to summarize the results. Categorical variables were presented as counts, percentages were provided for categorical variables, while continuous data were presented as medians and interquartile ranges. The predictive performance of the models was evaluated using the F 1-score, accuracy, precision, recall, and Brier score, which are commonly used metrics in ML- and DL-based CV to assess the effectiveness of a model[ 31 ]. The predictive discrimination ability of each model was described using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. An AUC value between 0.9 and 1.0 was defined as outstanding discriminative power, 0.8 and 0.9 as excellent, 0.7 and 0.8 as acceptable, and 0.5 and 0.7 as poor[ 32 ]. Bootstrap resampling was used to estimate 95% confidence intervals (CIs) for the AUC values. Sensitivity analyses were performed. A fixed classification threshold of 0.5 was applied across all models to enable consistent evaluation and comparison with expert performance. No post hoc threshold optimization was performed to preserve transparency and avoid inflating performance metrics. Calibration was summarized using the calibration and calibration-in-the-large slopes. A target value of 1.0 was set for the calibration slope, with values 1 reflecting estimated risks that are too moderate. A target value of 0.0 was set for the calibration-in-the-large, with positive values reflecting underestimation and negative values reflecting overestimation [ 33 , 34 ] . This was calculated by comparing the predicted probability of the lesions being malignant to the ground truth. Decision curve analysis was used to assess the clinical utility of each model [ 35 , 36 ] . The SHapley Additive exPlanations (SHAP) method was applied to investigate the contribution of each feature to the predictions made by the morphology-based model[ 37 ]. SHAP analysis was applied exclusively to the morphology-based model, which is built on structured and semantically interpretable features. All patients included had complete data for demographics, pathology reports, and video annotations, ensuring complete datasets for analysis without the need for imputation or adjustments for missing data. Statistical analyses and model construction were performed using SciPy 1.11.4. Statistical significance was set at P < 0.05.

The study cohort included 67 patients (females: n = 33, 49.3%; males: n = 34, 50.7%) aged 31–83 years, who underwent SL for PM diagnostic assessment. A total of 453 consecutive lesions (benign: n = 197, 42.8%; malignant: n = 256, 56.5%) were biopsied. Indications for SL were primarily gastric ( n = 23, 34.3%) and colorectal ( n = 18, 26.9%). Biopsies were mostly located in the parietal peritoneum of the left iliac fossa ( n = 105, 23.2%) and right hypochondria ( n = 91, 20.1%). Baseline cohort characteristics are shown in Table 1 . Table 1 Baseline cohort characteristics ( n = 67 patients) Characteristics N (%), median (IQR) Age (years) 67.2 (54.4–74.8) Gender (M) 34 (50.7%) BMI (kg/m 2 ) 23.4 (21.1–27.0) ASA, n (%)  I 0 (0%)  II 28 (41.8%)  III 38 (56.7%)  IV 1 (1.5%) Neoadjuvant chemotherapy (yes) 53 (79.1%) Indication, n (%)  Gastric cancer 23 (34.3%)  CRC 18 (26.9%)  Ovarian cancer 6 (9.0%)  PC 5 (7.5%)  Appendiceal cancer 4 (6.0%)  Cholangiocarcinoma 3 (4.5%)  HCC 2 (3.0%)  Esophageal cancer 2 (3.0%)  Others 4 (6.0%) Number of biopsies Test set ( n = 50)  Benign 197 (42.8%) 21  Malignant 256 (56.5%) 26 Malignant subtype a  Non-mucinous carcinomas 209 (81.6%) 24  Mucinous adenocarcinoma 26 (10.2%) 0  Serous carcinoma 12 (4.7%) 5  Pleiomorphic lobular carcinoma 8 (3.1%) 0  Neuroendocrine 1 (0.4%) 0 Benign lesion subtype a  Fibrotic 76 (38.5%) 6  Fibroadipose 56 (28.4%) 3  Inflammatory fibrotic 27 (13.7%) 0  Inflammatory fibroadipose 10 (5.1%) 1  Fibrotic with mesothelial hyperplasia 10 (5.1%) 4  Scar tissue 10 (5.1%) 7  Fibromyomatous 3 (1.5%) 0  Necrotic tissue 2 (1.0%) 0  Fibroadipose with mesothelial hyperplasia 1 (0.5%) 0 Fibromyomatous with mesothelial hyperplasia 1 (0.5%) 0  Adipose 1 (0.5%) 0 Tumor grade a  G3 137 (53.5%)  G2 46 (18.0%)  G1 73 (28.5%) Biopsies per patient ( n ) 4 (3–4) Biopsy location, n (%)  Left iliac fossa 105 (23.2%)  Right hypochondrium 91 (20.1%)  Right iliac fossa 87 (19.2%)  Left hypochondrium 80 (17.7%)  Pelvic cavity 22 (5.5%)  Right flank 19 (4.2%)  Left flank 18 (4.0%)  Anterior abdominal wall 14 (3.1%)  Falciform ligament 6 (1.3%)  Liver surface 4 (0.9%)  Omentum 4 (0.9%)  Others 3 (0.7%) BMI: body mass index; ASA: American Society of Anesthesiologists score; CRC = colorectal cancer; PC: pancreatic cancer; HCC: hepatocellular carcinoma. a Based on the pathology report. Baseline cohort characteristics ( n = 67 patients) BMI: body mass index; ASA: American Society of Anesthesiologists score; CRC = colorectal cancer; PC: pancreatic cancer; HCC: hepatocellular carcinoma. Based on the pathology report. The dataset was randomly divided into training and test sets comprising 54 and 13 patients, respectively. A dataset of 5214 images (2044 benign and 3170 malignant) from 453 biopsied lesions was used to construct image-based models. The training resulted in the construction of 12 image-based models spanning six distinct types: DenseNet121, EfficientNet, MobileNetV2, MobileNetV3, ResNet, and ShuffleNetV2. Two variants were used for EfficientNet and MobileNetV3, and five variants were used for ResNet. These image-based models were trained using 100 epochs with validation loss-based early stopping and halting training if the validation loss did not improve for nine consecutive epochs. Training set images were randomly sampled to create a validation set for monitoring the training progress and tuning the two different learning rates: 0.001 and 0.0005. A fixed batch size of 16 was used. The best-performing image-based model was ResNet152, with an F 1-score and a weighted F 1-score of 0.77 and 0.67, respectively, and an AUC of 0.72 (95% CI, 0.54–0.87) for the test set. The accuracy, precision, and recall were 0.70, 0.76, and 0.78, respectively. The sensitivity was 0.78, and the specificity was 0.56. The positive predictive value (PPV) and negative predictive value (NPV) were 0.76 and 0.59, respectively. The calibration and calibration-in-the-large slopes were 1.65 (95% CI, 0.77–4.07) and −0.26 (95% CI, −1.60–1.08), respectively. These values indicate acceptable discriminative power and calibration of the best-performing image-based model. The performance results of all trained models are provided in Supplemental Digital Content, Data S3, available at: http://links.lww.com/JS9/F75 . The same biopsied lesions and train-test split were used to develop the morphology-based models. The morphology-based models consisted of 10 morphologic variables, including 27 morphologic characteristics. The distribution of the morphologic variables is provided in Supplemental Digital Content, Data S4, available at: http://links.lww.com/JS9/F75 . Six ML models were trained to predict PM based on the morphologic features provided by the experts. The median Cohen’s Kappa value across all morphologic features was 0.4331, indicating moderate inter-rater agreement. These models included decision tree, random forest, gradient boosting, support vector machine, logistic regression, and ANN. A summary of the performance results is presented in Table 2 . The ROC and precision-recall curves for each morphology-based model are presented in Supplemental Digital Content, Data S5, available at: http://links.lww.com/JS9/F75 . The best-performing morphology-based model was the ANN, with an AUC of 0.86 (95% CI, 0.71–0.95) on the test set. The calibration and calibration-in-the-large slopes of the ANN were 1.03 (95% CI, 0.68–1.31) and 0.09 (95% CI, −0.11–0.33), respectively. This indicates excellent discriminative power and calibration of the predictive morphology-based model. The SHAP values determined the weight of each variable in predicting malignant versus benign lesions. Figure 2 A illustrates the SHAP summary plot for the ANN, which depicts the relative importance of each feature. The vertical ordering in the SHAP plot represents the magnitude of these values. Positive SHAP values (clusters of points above the zero line) signify a stronger contribution to predicting PM, whereas negative SHAP values (below the zero line) indicate a lower predicted probability of PM. In the plot, red and blue denote opposing directions of influence: red represents features associated with a higher predicted probability of PM, while blue represents features associated with a lower predicted probability (likely benign). For example, neovasculature and markedly nodular lesions are characterized by positive red clusters with highly positive SHAP values, whereas smaller negative blue clusters have slightly negative SHAP values. As depicted in Figure 2 B, the presence of neovasculature (SHAP value = 0.054) and markedly nodular lesions (SHAP value = 0.040) were the features most indicative of malignancy. Conversely, flat lesions (SHAP value = 0.057), absence of marked contours (SHAP value = 0.041), polychromatic appearance (SHAP value = 0.038), and absence of neovascularization (SHAP value = 0.033) were associated with benign lesions. These characteristics suggest that neovasculature and markedly nodular lesions tend to increase the predicted probability of PM, indicating a higher likelihood of malignancy. Conversely, flat lesions displayed a prominent positive blue cluster and a negative red cluster, suggesting that flat lesions are associated with a lower predicted probability of PM, indicating a benign tendency. Figure 2. (A) SHAP plot; and (B) mean SHAP values of the best-performing morphology-based model. Table 2 Performance of morphology-based models on the test set Model Accuracy Precision Recall F 1-score Weighted F 1-score AUC Sensitivity Specificity PPV NPV Logistic regression 0.72 0.92 0.63 0.74 0.76 0.79 0.63 0.89 0.91 0.57 Decision tree 0.64 0.79 0.59 0.67 0.63 0.77 0.59 0.72 0.79 0.50 Random forest 0.70 0.84 0.66 0.74 0.72 0.84 0.66 0.78 0.84 0.56 Gradient boosting 0.80 0.78 0.97 0.86 0.74 0.81 0.97 0.50 0.78 0.90 Support vector machine 0.70 0.90 0.59 0.72 0.74 0.75 0.59 0.89 0.90 0.55 Artificial neural network 0.74 0.86 0.72 0.78 0.76 0.86 0.72 0.78 0.85 0.61 (A) SHAP plot; and (B) mean SHAP values of the best-performing morphology-based model. Performance of morphology-based models on the test set The best MML model was random forest, obtaining an AUC of 0.88 (95% CI, 0.77–0.96) on the test set. The accuracy, precision, recall, F 1-score, and weighted F 1-score were 0.74, 0.86, 0.73, 0.79, and 0.72, respectively. The MML model achieved a sensitivity of 0.72, a specificity of 0.78, and corresponding PPV and NPV of 0.85 and 0.61, respectively. Thirteen oncologic surgeons evaluated each image in the test set, resulting in an average AUC of 0.78 (95% CI, 0.53–1.00). Brier scores were 0.1597 (95% CI, 0.1218–0.1969) for the MML model and 0.1925 (95% CI, 0.1412–0.2489) for the experts. Figure 3 presents the ROC curves for the best-performing image-based model (Fig. 3 A), the best-performing morphology-based model (Fig. 3 B), the MML model (Fig. 3 C), and the performance of surgeons (Fig. 3 D). The precision-recall curve is shown in Figure 4 A. The calibration and calibration-in-the-large slopes of the MML model were 0.97 (95% CI, 0.70–1.19) and 0.13 (95% CI, −0.04 to 0.34), respectively (Fig. 4 B). This indicates excellent discriminative power and calibration of the predictive model. The decision analysis curve is shown in Figure 4 C. Examples of the model outputs for the constructed models are shown in Figure 5 . Figure 3. (A) ROC curve of the best-performing image-based model; (B) best-performing morphology-based model; (C) Multimodal model; and (D) expert evaluations for the test set. Figure 4. (A) Precision-recall curve; (B) calibration plot; and (C) decision analysis curve of the Multimodal Model. Figure 5. Comparison of the best Multimodal Model predictions, expert surgeon assessments, and ground truth. (A) ROC curve of the best-performing image-based model; (B) best-performing morphology-based model; (C) Multimodal model; and (D) expert evaluations for the test set. (A) Precision-recall curve; (B) calibration plot; and (C) decision analysis curve of the Multimodal Model. Comparison of the best Multimodal Model predictions, expert surgeon assessments, and ground truth.

This novel MML model, developed and internally validated on a large cohort of biopsied PM, demonstrated excellent discriminative performance and calibration in distinguishing PM from benign lesions. The best-performing image-based and morphology-based models showed an AUC of 0.72 and 0.86, respectively. The final MML model, combining both approaches, achieved an AUC of 0.88, outperforming experts with an AUC of 0.78. While both the image-based and morphology-based components rely on visual perception, they represent distinct data modalities – raw pixel input versus structured semantic features – and their integration aligns with established definitions of MML in medical imaging [ 14 , 20 , 29 , 30 ] . The present model demonstrates how heterogeneous but complementary sources of visual information can be jointly leveraged to improve classification accuracy. Recent AI advancements within healthcare have highlighted the growing use of MML models to enhance diagnostic accuracy[ 14 ]. In laparoscopic and endoscopic surgery, recent multimodal AI advances include transformer-based models that integrate visual, audio, and textual inputs, as well as video-text generative approaches, enabling cross-modal understanding and automated surgical video documentation. [ 38 – 40 ] However, for a diagnostic model to be deemed reliable and implementable, it should achieve near-perfect sensitivity to minimize false negatives and ensure clinical reliability. Clinically integrated AI models for melanoma achieve a sensitivity and specificity of 0.95 and 0.84, respectively[ 41 ]. ML models for polyp classification reach AUC values of up to 0.95, with both sensitivity and specificity reported at 0.97, demonstrating clinical suitability[ 42 ]. In this context, the MML model demonstrated a sensitivity of 0.72, a specificity of 0.78, and PPV and NPV of 0.85 and 0.61, respectively, outperforming expert surgeons. Although the current MML model may not yet meet the standards for clinical implementation, its superior performance over surgeons highlights its potential as an intraoperative decision-support tool. Recent literature highlights AI’s potential role as a “co-pilot,” enhancing surgical workflow efficiency as a secondary reviewer[ 43 ]. This model could reduce unnecessary biopsies, optimize PM sampling, and aid in assessing tumor burden and resectability for cytoreductive surgery. Recognizing PM is a complex cognitive task influenced by several critical factors that must be addressed when differentiating it from benign lesions. First, PM morphology exhibits substantial heterogeneity due to tumor regression, fibrosis due to systemic treatment, and intricate visual and morphologic characteristics of peritoneal lesions [ 44 , 45 ] . In the present dataset, 79% of the population received neoadjuvant chemotherapy. Some benign lesions likely represent regressed metastatic disease, as effective chemotherapy can induce tumoral shrinkage without complete macroscopic resolution, leaving fibrotic or scar-like tissue behind. The inclusion of such lesions strengthens this model’s clinical relevance by allowing it to address the real-world diagnostic challenge of distinguishing treatment-induced fibrosis from true malignant deposits. Furthermore, this differentiation heavily relies on the operating surgeon’s expertise. Previous studies have shown that surgeons may underestimate disease extent, while histopathology offers more reliable assessments [ 46 , 47 ] . Schnelldorfer et al used a DL system, CASL, achieving an AUC of 0.78, surpassing surgeons alone (AUC 0.69) and improving further with combined input (AUC 0.79). This finding indicates the strong potential of CASL in intraoperative PM identification. Schnelldorfer’s model utilized a dataset with only one-third of pathology-confirmed malignancies. While the dataset used for training the CASL model utilized pathology-confirmed malignancies, its performance was evaluated on presumed PM in patients with curative resections based on carcinomatosis-free survival (CFS) rather than confirmed malignant lesions[ 24 ]. In contrast, this study’s dataset highlights the coexistence of benign and malignant lesions, suggesting that CASL evaluations may not fully capture disease extent. AI model performance depends heavily on ground truth quality, requiring rigorous control of training, testing, and validation, especially when pathology data are available[ 48 ]. Patients with limited PM responding well to treatment pose challenges due to tumor regression, complicating the use of CFS as a reliable measure for PM identification[ 49 ]. This study emphasized classification based solely on pathology-confirmed ground truth due to morphologic similarities between benign and malignant lesions. Furthermore, in the CASL model, validating surgeons evaluated only 10 PM images each, whereas the present study used a smaller cohort of oncologic surgeons independently assessing the entire test set. This comprehensive evaluation of the test set enhances the robustness of the findings and underscores the reliability of the proposed model, particularly through the integration of morphologic features and the application of an MML model not previously described in this research area. A recent study by Chen et al introduced the AiLES for real-time segmentation of intra-abdominal PM from gastric cancer. While AiLES demonstrates promising performance for lesion detection, particularly of small and occult metastases, it does not address the critical challenge of differentiating malignant from benign lesions, such as fibrosis or scar tissue. Furthermore, its development was limited to gastric cancer, whereas peritoneal lesions encountered during SL span a wide range of primary tumors and morphologies[ 45 ]. Expert-augmented AI has demonstrated its validity as a methodology for integrating human expertise to build robust and data-efficient ML predictive models[ 28 ]. AI-based image analysis studies have shown reliability in the morphologic characterization of tumoral lesions, but applications for intraoperative diagnostics remain limited[ 50 ]. Specifically, the importance and correlation of morphologic features with malignant lesions were demonstrated by Schnelldorfer et al in 2019 using a smaller dataset. Although their model achieved an AUC of 0.80, these features alone were not considered suitable for a comprehensive risk prediction model[ 13 ]. Recognizing the importance of these features, particularly those correlated with tumor pathophysiology such as neovasculature and nodularity, and considering the challenges faced by the CV model in detecting these features in visual data, the current study adopted the concept of expert-augmented ML to integrate these features with visual pixel-based data[ 28 ]. The morphology-based model validated these features’ importance, achieving an AUC of 0.86, likely aided by this study’s larger dataset. This suggests that a morphology-based approach alone is reliable for identifying PM. Combining a morphology model with an image-based model clearly improves predictive performance and enhances clinical utility. Neovasculature and nodularity, aligned with tumor pathophysiology, were the strongest PM correlates[ 51 ]. These features are particularly valuable during surgical procedures, aiding surgeons in differentiating benign lesions from PM. Whereas specific morphologic features demonstrate prominence, the combined feature set proves to be the most predictive of PM. This is vividly illustrated by the SHAP plot, which depicts the average marginal contribution of each feature in a straightforward graphical format, ensuring consistent evaluation and broader clinical applicability[ 52 ]. For neovasculature and markedly nodular lesions, dominant red clusters indicate a higher predicted PM probability, while smaller blue clusters suggest that these features may show lower values in certain benign instances, although their overall association with malignancy remains positive. This underscores that while certain features often indicate malignancy, they may still be variably present in benign cases. SHAP-derived insights highlight morphologic features most predictive of malignancy, helping surgeons interpret lesions intraoperatively. In fact, by highlighting specific morphologic features most predictive of malignancy, such as neovasculature and nodularity, they not only improve interpretability of the model’s output but also provide practical guidance to surgeons when evaluating intraoperative lesions. Interpretability is particularly important in clinical contexts where trust and accountability are essential. While the DL and MML models offer strong predictive performance, their internal representations typically remain a “black box.” As such, the morphology-based model remains a critical component of the MML framework, enhancing both interpretability and clinical confidence. Limitations of this study include the reliance on monocentric data from patients with proven abdominal malignancies. Future research should incorporate multicenter oncology datasets for broader validation to further enhance the diagnostic capability of the proposed model. Specific validation in patients with inflammatory or infectious peritoneal conditions such as endometriosis, peritonitis, or granulomatous disease will be essential to assess the model’s performance in clinically ambiguous cases. Notably, conditions such as peritonitis may substantially hinder accurate interpretation of peritoneal lesions and are therefore not ideal for inclusion in this first-phase model, but they represent important targets for future integration. Future efforts should include institutions with lower case volume and expertise in PM to confirm the model’s robustness and potential utility as a decision-support tool not only in tertiary oncology centers, but also across varied clinical settings where expert assessment may be limited. This could enhance diagnostic consistency and staging accuracy in broader practice. Nonetheless, this study’s dataset offers a well-balanced representation of benign and malignant peritoneal lesions, including scar tissue and treatment-related changes. While the MML model is not intended to replace histopathologic confirmation, it may support intraoperative decision-making by helping prioritize lesion sampling and assess tumor burden, especially in cases with mixed benign and malignant features, including those following chemotherapy. Performance variability across models may reflect differences in their ability to model non-linear interactions between visual and morphologic features. The ANN, for instance, demonstrated superior discrimination, likely due to its capacity to capture complex multimodal patterns. All models were evaluated using identical input data and standard metrics, including AUC, sensitivity, specificity, and accuracy. SHAP analysis further confirmed that the ANN successfully emphasized clinically meaningful morphologic traits, supporting its internal consistency and interpretability. This study’s frame-based model could be significantly enhanced by advancing to a real-time video-based approach with integrated detection capabilities, enabling intraoperative application. This MML model could aid decision-making for patients with PM who have responded well to systemic treatment and are being staged for either curative surgery or targeted therapies[ 53 ]. Furthermore, a broader cohort of experts could further validate and enhance the model’s significance. Heterogeneity in model performance across patient groups should be explored to evaluate robustness against data distribution shifts[ 54 ]. A subgroup analysis for hepatobiliary tumors was not feasible due to the limited number of cases. Supplemental Digital Content, Data S6, available at: http://links.lww.com/JS9/F75 and Supplemental Digital Content, Data S7, available at: ––– provide detailed patient demographics, intraoperative images, and overall survival of the five hepatobiliary patients included in this study. These interpretations are based on model tendencies derived from data analysis; clinical decisions should integrate a comprehensive patient evaluation, including clinical history, imaging, pathology, and other diagnostic tests to make informed choices. Future investigations should incorporate patient demographics and history into the MML model to provide a more comprehensive view of the diagnostic challenge. Although not yet deployed in real-time surgical settings, this MML model establishes a foundational framework for intraoperative decision support. By simulating how combined visual and morphologic information can enhance lesion classification, it addresses an unmet clinical need in SL. Further validation in prospective workflows, randomized controlled trials, and multi-institutional settings will be required to assess robustness and prepare for clinical deployment. A user-friendly graphical tool capable of instantly providing a judgment on the nature of these lesions, either through analysis of images, surgeon morphologic characteristics, or a combination of both modalities, is currently under development and represents a planned future step of this work.

This study introduced a novel MML model that can accurately differentiate between malignant and benign peritoneal lesions. By integrating image and morphologic data, this predictive tool has the potential to enhance surgical decision-making, reduce unnecessary biopsies, and improve overall patient management.