External validation of ultrasound-based models for discrimination between benign and malignant adnexal masses in Italy: the prospective multicenter IOTA phase 6 study

Objective To prospectively validate the performance of the Risk of Malignancy Index (RMI), International Ovarian Tumor Analysis (IOTA) Simple Rules Risk Model (SRRisk), IOTA Assessment of Different NEoplasias in the adneXa (ADNEX) and the IOTA two-step strategy in different types of ultrasound centers in Italy. Methods This is a multicenter prospective observational study including regional referral centers and district hospitals in Italy. Consecutive patients with an adnexal mass examined with ultrasound by an IOTA certified ultrasound examiner with different levels of experience were included, provided they underwent surgery < 180 days after the inclusion scan. Ultrasound examination was performed transvaginally or transrectally and/or transabdominally based on the characteristics of the women and masses. Reference standard was the histology of the adnexal mass following surgical removal. Discrimination (area under receiver operating characteristic curve, AUROC), calibration, and clinical utility were assessed to illustrate the diagnostic performance of the methods. The performance of the models was also evaluated in predefined subgroups based on menopausal status, type of center (oncology vs non-oncology) and ultrasound examiner’s experience: [5000 scans performed; European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) Level 1, Level 2, Level 3]. Results 1567 patients were recruited between May 2017 and March 2020 from 23 italian centers. After data cleaning and application of exclusion criteria, our study population consisted of 1431 patients in 21 italian centers (10 oncological and 11 non-oncological). Based on histology, 995/1431 (69.5%) tumors were benign and 436/1431 (30.5%) were malignant (115/1431, 8.0% borderline, 263/1431, 18.4% primary invasive, 58/1431, 4.1% metastatic tumors). For all IOTA models (SRRisk, ADNEX with and without CA125, two step strategy with and without CA125), the AUROC was between 0.91 (95% CI 0.88-0.93) and 0.92 (0.89-0.94). The AUROC was 0.85 (0.81-0.87) for RMI. The malignancy risk was slightly underestimated by all IOTA models, but least so by SRRisk. All IOTA models had higher net benefit than RMI at risk thresholds from 1% to 50%. AUROC was > 0.90 for all IOTA models in all subgroups, while it ranged from 0.84 to 0.90 for RMI. Conclusions SRRisk, ADNEX and the two step strategy with or without CA125 had similar and good ability to distinguish benign from malignant adnexal tumours in patients examined by either expert or non-expert ultrasound operators in Italy. Their discriminative performance and clinical utility was superior to that of RMI.