Program Evaluation Activities in Competence by Design: A Survey of Specialty/Subspecialty Program Directors

AbstractBackground The evaluation of Competence by Design (CBD) residency programs is essential for improving program effectiveness. There is limited literature on the evaluation of CBD programs. We investigated the extent to which program evaluation activities are occurring in CBD residency programs in Canada and the reasons why these programs are engaging or not engaging in them. Methods We surveyed program directors whose programs transitioned to CBD. We calculated descriptive statistics for the 22 closed-ended survey items. Results We obtained 149 responses (response rate 33.5%). Of the 149 respondents, 127 (85.2%) indicated that their programs do engage in evaluation while 22 (14.8%) indicated that their programs do not. Of the 127 whose programs do engage in evaluation, 29 (22.8%) reported that their programs frequently or always develop evaluation questions and 23 (18.1%) noted that their programs design evaluation proposals/plans. Reasons for engaging in evaluation included: to make decisions about the program, and to stimulate changes in educational practices. Reasons for not engaging in evaluation comprised: no knowledge on how to do it, no personnel to do evaluation, and no funding to do it. Conclusions While most CBD programs are doing program evaluation the quality of it is questionable.