Retrospective evaluation of gynecologic cytodiagnosis. I. Reproducibility using an experimental diagnostic scale

A system for the exchange of specimens between two independent laboratories ("A" and "B") was designed to enable them to review a consecutive series of one another's gynecologic smears. Another cytopathologist "refereed" diagnoses differing beyond agreed limits and examined all smears called carcinoma in situ or a more severe lesion by either laboratory. A ten-numeral gradient nomenclature scale was developed to code all cytodiagnoses for data processing. This analysis of the reproducibility of the experimental diagnostic system is based on all participants' examination of 100 selected specimens, half contributed by reviewing laboratory A and half by reviewing laboratory B, with the target value of each specimen established by the contributor. Intralaboratory reproducibility, expressed by a correlation coefficient (r), ranged from 0.78 to 0.93. After adjusting for straightforward squamous-cell lesions, r ranged from 0.88 to 0.91. Intralaboratory agreement +/- 1 scalar category, defined as a "working consensus," ranged from 77% to 84%. Interlaboratory correlation ranged from r = 0.74 to 0.94. Adjustment for squamous-cell lesions exerted less effect than it had on intralaboratory reproducibility. Interlaboratory agreement within one category ranged from 58% (laboratory B reviewing specimens contributed by laboratory A) to 92% (referee examining specimens contributed by laboratory B). Initially, laboratory A tended to classify a given specimen higher on the scale than did laboratory B and the referee, but as the study progressed, both laboratories tended to converge toward a more congruent diagnostic position, with referee greater than A greater than B. The level of discrimination attained by all participants validates use of this system to compare a larger series of diagnoses.