A Prospective Study Measuring Resident/Faculty Contour Concordance: A Potential Tool for Quantitative Assessment of Residents’ Performance in Contouring and Target Delineation

ABSTRACT Purpose/Objective(s) Accurate target delineation/contouring is essential for radiation treatment planning and radiotherapy efficacy. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. The purpose of this study was to track the concordance of radiation oncology residents’ contours with faculty physicians’ contours over the course of one year to assess for patterns. Materials/Methods Residents in PGY levels of 2-4 were asked to contour target volumes which were compared to the finalized, faculty physician-approved contours. Concordance between resident and faculty physician contours was determined by calculating the Jaccard Concordance Index (JCI), ranging from 0 or no agreement to 1 or complete agreement. Mutivariate mixed effect models were used to assess the association of JCI to the fixed effect of PGY level and its interactions with cancer type and other baseline characteristics. Post hoc means of JCI were compared between PGY levels after accounting for multiple comparisons using a Tukey’s method. Results In total, 958 structures from 314 patients collected during the 2020-2021 academic year were studied. The mean JCI was 0.77, 0.75, and 0.61 for the levels of PGY-4, PGY-3, and PGY-2, respectively. JCI score in PGY-2 was found lower than those of PGY-3 and PGY-4 respectively (P’s<0.001). No statistically significant difference of JCI score was found when comparing between PGY-3 and PGY-4 levels. The average JCI score was lowest (0.51) for primary head/neck cancers and highest (0.80) for gynecologic cancers. Conclusion Tracking and comparing the concordance of resident contours and faculty physician contours is an intriguing method of assessing resident performance in contouring and target delineation and could potentially serve as a quantitative metric in radiation oncology resident evaluation, which is lacking currently. However, additional study is necessary before this technique can be incorporated into residency assessments.