Reproducibility of subtle MRI findings suggestive of minimal endometriosis: agreement, Kappa paradox, and interpretive variability

Purpose To evaluate intraobserver and interobserver agreement in the interpretation of subtle MRI findings classified as minimal endometriosis, to assess the impact of the Kappa paradox in low-prevalence compartments, and to compare blinded rereads with the original prospective reports.

In this retrospective study, 100 pelvic MRI examinations (50 originally reported as minimal or incipient endometriosis and 50 originally reported as normal) were independently reviewed by three radiologists on two blinded occasions 4–8 weeks apart. Seven anatomical compartments were assessed. Agreement was measured with Cohen’s Kappa, Fleiss’ Kappa, PABAK, Gwet’s AC1, and positive/negative agreement. Original reports were used as a clinical reference for descriptive comparison, and exact McNemar tests assessed directional asymmetry of discordance.

Intraobserver agreement for any finding was fair (mean Kappa = 0.36, 95%CI 0.16–0.54; AC1 = 0.49), and interobserver agreement was also fair (Fleiss Kappa = 0.28; AC1 = 0.40). Agreement was consistently stronger for negative than for positive classifications. Ovarian findings showed the highest reproducibility, whereas very low-prevalence compartments showed a marked Kappa paradox. Multicategory analysis of uterosacral ligament laterality reduced agreement, mainly because unilateral-versus-bilateral disagreement remained frequent among positive cases. Comparison with original reports showed slight-to-moderate agreement (Kappa 0.18–0.46), with specificity (76–86%) consistently exceeding sensitivity (32–66%), indicating a more restricted threshold on blinded rereads, particularly for uterosacral ligament involvement.

Agreement for subtle MRI findings interpreted as minimal endometriosis was only fair and clearly lower than that usually reported for deep endometriosis. Low-prevalence compartments were strongly affected by the Kappa paradox, and the low-to-moderate discordance with original reports, driven primarily by lower sensitivity on blinded rereads, supports the need for clearer operational criteria and more standardized interpretation. Similar content being viewed by others Data availability The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.

Giudice LC, Kao LC (2004) Endometriosis. Lancet 364:1789–1799. https://doi.org/10.1016/S0140-6736(04)17403-5 Zondervan KT, Becker CM, Missmer SA (2020) Endometriosis. N Engl J Med 382:1244–1256. https://doi.org/10.1056/NEJMra1810764 Nnoaham KE, Hummelshoj L, Webster P, et al (2011) Impact of endometriosis on quality of life and work productivity: a multicenter study across ten countries. Fertil Steril 96:366–373.e8. https://doi.org/10.1016/j.fertnstert.2011.05.090 Feldman MK, VanBuren WM, Barnard H, Taffel MT, Kho RM (2020) Systematic interpretation and structured reporting for pelvic magnetic resonance imaging studies in patients with endometriosis: value added for improved patient care. Abdom Radiol (NY) 45:1608–1622. https://doi.org/10.1007/s00261-019-02182-1 Thomassin-Naggara I, Dolciami M, Chamie LP, et al (2025) ESUR consensus MRI for endometriosis: protocol, lexicon, and compartment-based analysis. Eur Radiol 35:7272–7286. https://doi.org/10.1007/s00330-025-11611-3 Nisenblat V, Bossuyt PM, Farquhar C, Johnson N, Hull ML (2016) Imaging modalities for the non-invasive diagnosis of endometriosis. Cochrane Database Syst Rev 2:CD009591. https://doi.org/10.1002/14651858.CD009591.pub2 Bazot M, Daraï E (2017) Diagnosis of deep endometriosis: clinical examination, ultrasonography, magnetic resonance imaging, and other techniques. Fertil Steril 108:886–894. https://doi.org/10.1016/j.fertnstert.2017.10.026 Saba L, Guerriero S, Sulcis R, et al (2012) MRI and “tenderness-guided” transvaginal ultrasonography in the diagnosis of recto-sigmoid endometriosis. J Magn Reson Imaging 35:352–360. https://doi.org/10.1002/jmri.22832 Chassang M, Novellas S, Bloch-Marcotte C, et al (2010) Utility of vaginal and rectal contrast medium in MRI for the detection of deep pelvic endometriosis. Eur Radiol 20:1003–1010. https://doi.org/10.1007/s00330-009-1627-8 Siegelman ES, Oliver ER (2012) MR imaging of endometriosis: ten imaging pearls. Radiographics 32:1675–1691. https://doi.org/10.1148/rg.326125518 Del Frate C, Girometti R, Pittino M, Del Frate G, Bazzocchi M, Zuiani C (2006) Deep retroperitoneal pelvic endometriosis: MR imaging appearance with laparoscopic correlation. Radiographics 26:1705–1718. https://doi.org/10.1148/rg.266065048 Rousset P, Florin M, Bharwani N, et al (2023) Deep pelvic infiltrating endometriosis: MRI consensus lexicon and compartment-based approach from the ENDOVALIRM group. Diagn Interv Imaging 104:95–112. https://doi.org/10.1016/j.diii.2022.09.004 Conte de Oliveira MD, Fernandes HS, Vasconcelos AL, et al (2023) Impact of a quality programme on overindication of surgeries for endometriosis and cholecystectomies. BMJ Open Qual 12:e002178. https://doi.org/10.1136/bmjoq-2022-002178 Practice Committee of the American Society for Reproductive Medicine (2014) Treatment of pelvic pain associated with endometriosis: a committee opinion. Fertil Steril 101:927–935. https://doi.org/10.1016/j.fertnstert.2014.02.012 Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46. https://doi.org/10.1177/001316446002000104 Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43:543–549. https://doi.org/10.1016/0895-4356(90)90158-L Byrt T, Bishop J, Carlin JB (1993) Bias, prevalence and kappa. J Clin Epidemiol 46:423–429. https://doi.org/10.1016/0895-4356(93)90018-V Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. https://doi.org/10.2307/2529310 Gwet KL (2008) Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 61:29–48. https://doi.org/10.1348/000711006X126600 Corwin MT, Gerscovich EO, Lamba R, Wilson M, McGahan JP (2014) Differentiation of ovarian endometriomas from hemorrhagic cysts at MR imaging: utility of the T2 dark spot sign. Radiology 271:126–132. https://doi.org/10.1148/radiol.13131394 Takeuchi M, Matsuzaki K, Harada M (2011) Adenomyosis: usual and unusual imaging manifestations, pitfalls, and problem-solving MR imaging techniques. Radiographics 31:99–115. https://doi.org/10.1148/rg.311105110 Thomassin-Naggara I, Lamrabet S, Crestani A, et al (2020) Magnetic resonance imaging classification of deep pelvic endometriosis: description and impact on surgical management. Hum Reprod 35:1589–1600. https://doi.org/10.1093/humrep/deaa103 Thomassin-Naggara I, Monroc M, Chauveau B, et al (2023) Multicenter external validation of the deep pelvic endometriosis index magnetic resonance imaging score. JAMA Netw Open 6:e2311686. https://doi.org/10.1001/jamanetworkopen.2023.11686 Harth S, Kaya HE, Zeppernick F, et al (2023) Application of the #Enzian classification for endometriosis on MRI: prospective evaluation of inter- and intraobserver agreement. Front Med (Lausanne) 10:1303593. https://doi.org/10.3389/fmed.2023.1303593 Tammaa A, Fritzer N, Lozano P, et al (2015) Interobserver agreement and accuracy of non-invasive diagnosis of endometriosis by transvaginal sonography. Ultrasound Obstet Gynecol 46:737–740. https://doi.org/10.1002/uog.14843 VanBuren W, Feldman M, Shenoy-Bhangle AS, et al (2024) Radiology state-of-the-art review: endometriosis imaging interpretation and reporting. Radiology 312:e233482. https://doi.org/10.1148/radiol.233482 Keckstein J, Saridogan E, Ulrich UA, et al (2021) The #Enzian classification: a comprehensive non-invasive and surgical description system for endometriosis. Acta Obstet Gynecol Scand 100:1165–1175. https://doi.org/10.1111/aogs.14099

The authors thank Lucas Damiani and Renato Hideo Nakagawa Santos (Instituto de Pesquisa, Hospital do Coração - HCor, São Paulo, Brazil) for their assistance with the methodology and statistical analysis. Author information Authors and Affiliations Contributions H.L.F. conceived and designed the study, collected data, performed the image analysis, conducted the statistical analysis, and drafted the manuscript. M.F.A. and M.S.C. independently performed the blinded image analyses used for intra- and interobserver agreement assessment and contributed to data collection. A.S. provided institutional resources and supervision. H.L.F. revised the final version. All authors reviewed and approved the submitted manuscript and agree to be accountable for the accuracy and integrity of the work. Corresponding author Ethics declarations Conflict of interest The authors declare no competing interests. Additional information Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information Below is the link to the electronic supplementary material. Rights and permissions Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. About this article Cite this article Leao Filho, H., Figueiredo Alves, M., Sousa de Castro, M. et al. Reproducibility of subtle MRI findings suggestive of minimal endometriosis: agreement, Kappa paradox, and interpretive variability. Abdom Radiol (2026). https://doi.org/10.1007/s00261-026-05661-4 Received: Accepted: Published: Version of record: DOI: https://doi.org/10.1007/s00261-026-05661-4