User Variability in Abstracting and Entering Printed Case Histories with QUICK MEDICAL REFERENCE (QMR)

The variability in abstracting and entering printed case histories into the Quick Medical Reference (QMR) diagnostic computer program was explored. Six study physicians who were familiar with the use of the QMR program abstracted and entered data from nine published NEJM CPC cases. Their entered findings were compared to those entered by an expert user who was a primary developer of the system. When subjects findings were compared to the expert's in a 2×2 table analysis, mean predictive value for positive findings was 0.70, and for negative findings 0.57. Mean Sensitivity was 0.59 for positive findings and 0.30 for negative ones. This suggests that there was good agreement between positive findings entered by subjects and those entered by the expert, though not all findings entered by the expert were included by the subjects. Greater variability was encountered in entering negative findings than in entering positive ones. Several reasons for the observed variability are discussed.