Interrater and Intrarater Reliability in the Diagnosis and Staging of Endometriosis

Karen C Schliep, Joseph B Stanford, Zhen Chen, Bo Zhang, Jessie K Dorais, Erica Boiman Johnstone, Erica Johnstone, Ahmad Hammoud, Ahmad O Hammoud, Michael W Varner, Germaine M Buck Louis, C Matthew Peterson
Obstetrics and gynecology · 2012 · vol. 120(1) , pp. 104–112 · doi:10.1097/aog.0b013e31825bc6cf · PMID:22914398 · W2077088582
article OA: green CC0 ⤵ 20 in-corpus citations
🔓 Open OA copy View on OpenAlex View on PubMed View at publisher
⚙ AI-generated summary by claude@2026-06, 2026-06-07 ⓘ

Expert gynecologic surgeons demonstrated substantial interrater reliability for endometriosis diagnosis and moderate reliability for ASRM staging, with near-perfect intrarater reliability for staging.

One-sentence paraphrase of the abstract; not a substitute for reading it. No clinical advice. How this works

⚙ AI-generated deep summary by claude@2026-06, 2026-06-07 ⓘ

This study evaluated interrater and intrarater reliability of endometriosis diagnosis and revised ASRM severity staging by eight gynecologic surgeons (four academic expert and four local specialized expert) reviewing anonymized operative digital images from a population-based sample of 148 women enrolled in the ENDO study. Surgeons showed substantial agreement for diagnosis (Fleiss κ=0.69) but only moderate agreement for revised ASRM staging (Fleiss κ=0.44), while intrarater reliability for experienced surgeons’ severity ratings versus computer-assisted, checklist-based staging was almost perfect (mean weighted κ=0.95). A key limitation explicitly noted is that while reliability was assessed, how disease-burden staging correlates with clinical outcomes remains to be developed, and some women had “indeterminate” ratings that were excluded. This paper is centrally about endometriosis — it quantifies how consistently surgeons can diagnose and stage endometriosis from operative images.

Read from the paper's body, not the abstract. Not a substitute for reading the paper. No clinical advice. How this works

Abstract

OBJECTIVE: To estimate the interrater and intrarater reliability of endometriosis diagnosis and severity of disease among gynecologic surgeons viewing operative digital images. METHODS: The study population comprised a random sample (n=148 [36%]) of women who participated in the Endometriosis: Natural History, Diagnosis and Outcomes study. Four academic expert and four local, specialized expert surgeons reviewed the images, diagnosed the presence or absence of endometriosis for each woman, and rated severity using the revised American Society for Reproductive Medicine (ASRM) criteria. Interrater-level and intrarater-level agreement were calculated for both endometriosis diagnosis and staging. RESULTS: The interrater reliability for endometriosis diagnosis among the eight surgeons was substantial: Fleiss κ=0.69 (95% confidence interval [CI] 0.64-0.74). Surgeons agreed on revised ASRM endometriosis staging criteria after experienced assessment in a majority of cases (mean 61%, range 52-75%) with moderate interrater reliability: Fleiss κ=0.44 (95% CI 0.41-0.47). The intrarater reliability for experienced assessment compared with computer-assisted revised ASRM staging was almost perfect (mean weighted κ=0.95, range 0.89-0.99). CONCLUSION: Substantial reliability was found for revised ASRM endometriosis diagnosis, whereas moderate reliability was observed for staging. Almost perfect reliability was observed for surgeons' rating of disease severity compared with computerized-assisted, checklist-based staging. Findings suggest that reliability in endometriosis diagnosis is not greatly altered by location or composition of surgeons, supporting the conduct of multisite studies or compilation of endometriosis data across clinical centers. Although surgeons appear to be skilled at assessing endometriosis stage intuitively, how staging of disease burden correlates with clinical outcomes remains to be developed.

My notes (saved in your browser only)

Outcome instruments

rASRM

Condition tags

mesh:D004715endometriosis

MeSH descriptors

Endometriosis Adult Endometriosis Female Humans Reproducibility of Results Severity of Illness Index

Citation neighborhood

Papers in the corpus that this work cites (lower rings, blue) and that cite this one (upper rings, green). Dot size scales with the paper's in-corpus citation count — bigger dot = more influential within the endo/adeno field. Click a dot to open that paper. [ expand to 2 hops ] — adds papers reached through this work's immediate citers/citees. Heavier; up to 60 extra dots.

References (27)

Cited by (20)

Source provenance

europepmc
last seen: 2026-06-04T01:30:01.192114+00:00
openalex
last seen: 2026-06-10T17:14:06.276822+00:00
pubmed
last seen: 2026-05-13T22:16:04.919516+00:00
License: CC0 · commercial use OK