Implementation and Evaluation of a Communication Coaching Program: A CFIR-Informed Qualitative Analysis Mapped onto a Logic Model

Implementation and Evaluation of a Communication Coaching Program: A CFIR-Informed Qualitative Analysis Mapped onto a Logic Model | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Implementation and Evaluation of a Communication Coaching Program: A CFIR-Informed Qualitative Analysis Mapped onto a Logic Model Rachel M. Jensen, Marzena Sasnal, Uyen T. Mai, James R. Korndorffer, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4372302/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 25 Apr, 2025 Read the published version in BMC Medical Education → Version 1 posted 10 You are reading this latest preprint version Abstract Background Coaching programs in graduate medical education have the potential to impact trainee development across multiple core competencies but require rigorous program evaluation to ensure effectiveness. We sought to qualitatively evaluate the implementation of a multi-departmental, faculty-led communication coaching program using a logic model framework. Methods Study participants were selected from four key stakeholder groups: resident coachees, faculty coaches, medical education leaders, and programmatic sponsors. 30–45 minute semi-structured interviews were conducted via Zoom, transcribed, and de-identified for the analysis. Interviews captured stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders’ involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The Consolidated Framework of Implementation Research (CFIR) guided the codebook development and data analysis. A combined inductive/deductive approach was used to develop a 20-item codebook, followed by a team-based thematic analysis. A strong intercoder agreement (Cohen’s kappa coefficient κ = 0.83) ensured coding consistency. The emerging themes were then mapped onto four domains of a logic model: Context, Inputs and Outputs, Outcomes, and Evaluation. Results 35 interviews were conducted between November 2021 and April 2022 with representation from all stakeholder groups, including 10 resident coachees (who received coaching), 10 faculty coaches (who served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (who designed and implemented program), and programmatic sponsors (who provided financial support). We mapped 8 emergent themes onto the critical domains of a logic model for program evaluation. For the domain of Context, themes included ( 1 ) gap in communication education and ( 2 ) patient-centeredness. For the domain of Inputs/Outputs, themes included ( 1 ) investment in the program and ( 2 ) perceived program value. For the domain of Outcomes, themes included ( 1 ) learning-focused outcomes and ( 2 ) patient-related outcomes. For the domain of Evaluation, themes included ( 1 ) defining success and ( 2 ) challenges with evaluation. Conclusions Mapping CFIR-informed themes onto a logic model for program evaluation presents a novel strategy for integrating program implementation and evaluation, both of which are essential to effective educational programming. These findings can be used to guide future programmatic modifications to better meet the needs of key stakeholders. Graduate Medical Education (GME) Communication Coaching Program Evaluation Program Implementation The Consolidated Framework of Implementation Research Logic Model Qualitative Interview Study Figures Figure 1 Figure 2 Figure 3 Background Communication is central to the role of a competent physician, yet this core competency is taught or assessed directly less often than competencies such as patient care and medical knowledge. 1 Lack of education training limits the potential for feedback and thoughtful self-reflection on communication skills, which are essential to a trainee’s personal-professional growth. 2 More recently, coaching programs at the graduate medical education (GME) level have been implemented to address the specific needs of trainees. 3–5 Coaching, which is distinct from mentorship or advising, involves a true partnership between a coach and a coachee. 6 A coaching framework is learner-centered and helps the coachee identify opportunities for improvement through facilitated self-reflection and goal setting. 7,8 Coaching programs have enormous potential to impact resident growth across multiple core competencies, including practice-based learning and improvement, professionalism, and interpersonal skills and communication. 3,5 Proficiency-focused efforts are critical to help learners progress along milestones in the current era of competency-based medical education. Despite the strong potential of coaching programs, it is unclear whether coaching implementation efforts are effectively meeting the needs of stakeholders and inducing true change in learners. 9 Effective programmatic change requires deliberate implementation and rigorous evaluation. There is an opportunity cost associated with any new implementation effort in GME; the time and effort that a coach or coachee puts into one endeavor inevitably means less time and effort elsewhere. For this reason, programs must make difficult choices and evolve to effectively meet stakeholders’ needs. 10,11 One common framework for programmatic implementation and evaluation is a logic model, which helps to balance community needs, program inputs and outputs, outcome measurements, and evaluation strategies. 12,13 A logic model can be used in an a priori fashion to assist with program design and implementation or in a post hoc setting for program evaluation. To evaluate outcomes related to a coaching intervention, programs often rely on survey-based quasi-experimental study designs; however, this approach may provide a limited view of a program’s impact and does not allow for exploration beyond the survey’s measured constructs. 9 Qualitative approaches, using individual interviews or focus groups, have the potential to provide a more robust program evaluation, especially for complex interventions with multiple stakeholders and moving parts. Therefore, we conducted a qualitative program evaluation of a faculty-led communication coaching program implemented at a single institution for Surgery and Neurology residents using semi-structured interviews with key program stakeholders. Methods Program Implementation Stanford University implemented a faculty-led communication-focused coaching program in the Departments of Surgery and Neurology in 2020 as previously described. 5 Briefly, the program was designed using Kern’s 6-step model of curriculum development and utilized the Consolidated Framework for Implementation Research in Formative Evaluation (CFIR) as an implementation framework. 14,15 The CFIR framework breaks down the implementation into 5 domains; see Fig. 1 for the key domains and a description of each. Notably, the program was designed with input from multiple stakeholder groups, including resident coachees, faculty coaches, medical education leaders, and programmatic sponsors, as well as collaborative efforts across multiple levels to ensure a rigorous plan for programmatic evaluation. Participants and Oversight We employed a key informant sampling strategy to purposively select study participants, including resident coachees, faculty coaches, medical education leaders, and programmatic sponsors. 16 Those four diverse stakeholder groups were, in various roles and functions, involved in the program’s design and implementation. Respondents were recruited via email sent by communication coaching directors (C.A.G. and A.K.N.) or a research analyst (M.S). Verbal consent was obtained from all participants, and the study was exempt from the IRB review as a quality improvement project. Semi-Structured Interviews CFIR-informed semi-structured interviews conducted between November 2021 and April 2022 aimed to capture stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders’ involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The interviews were conducted by a research analyst experienced in qualitative methods (M.S.), lasted 30–45 minutes, and took place via the Zoom (Zoom Video Communications Inc.) videoconference platform. Interviews were recorded, transcribed verbatim, and deidentified for analysis. Analytic Approach A rigorous team-based thematic analysis of the interview transcripts was conducted involving the following six steps: ( 1 ) familiarization with the data, ( 2 ) generating initial codes, ( 3 ) searching for themes, ( 4 ) reviewing themes, ( 5 ) defining and naming themes, and ( 6 ) producing the report. 17 Analytical procedures, including coding and assessing inter-coder agreement, were performed using NVivo qualitative software (Version Pro Enterprise, QSR International Pty Ltd, Massachusetts, USA, 2020). Trustworthiness during each phase of thematic analysis was established by various means, including prolonged engagement with data, peer debriefing, researcher triangulation, use of coding framework, themes and subthemes vetted by team members, team consensus on themes, and thick description of the context. 18 To develop a codebook, three team members (R.M.J., M.S., and U.T.M.) first inductively coded four interviews and then met multiple times to discuss emerging patterns, meanings, and how they fit into the CFIR framework. Then, R.M.J. deductively coded the same set of four interviews, using CFIR constructs as codes, and developed the first draft of the codebook. Next, M.S. and U.T.M. validated the codebook by applying it when independently coding the same four interviews. During this intensive analytical phase, coders frequently met to compare coding, discuss ambiguities, and make adaptions based on the findings, which resulted in the development of a 20-item CFIR-informed codebook. The codebook was then vetted by the whole analytics team (A.K.N., R.K.M., C.A.G., J.R.K., and A.M.M.). After the strong inter-rater agreement was reached (Cohen’s kappa coefficient κ = 0.83) between M.S., R.M.J, and U.T.M for the three newly coded interviews, which ensured coding consistency, the coders divided transcripts and coded the remaining interviews. 19 Throughout the coding and interpretation phases, coders frequently engaged in consensus discussion to identify emerging themes and resolve disagreements. The entire team subsequently reviewed and verified these findings. The themes that emerged from the CFIR-informed qualitative analysis, were then mapped onto four critical domains of a standard logic model: Context, Inputs and Outputs, Outcomes, and Evaluation. The Context domain describes contextual factors, priorities, and the program landscape as key features of implementation and evaluation. The Inputs and Outputs domain describes the resources invested in the program such as funding, time, skills, technology, and facilities, as well as the personal investment of program personnel and their individual motivation or incentive to engage in the program. This domain also includes the program's content or activities, including program execution and participation. The Outcomes domain addresses the perceived program outcomes and impact. The Evaluation domain focuses on the specifics of programmatic evaluation. 12,13 Results Participant characteristics Thirty-five stakeholders, including 10 resident coachees (received coaching), 10 faculty coaches (served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (designed and implemented program), and programmatic sponsors (provided financial support), participated in the interviews. Respondents were mainly employees and trainees of the Department of Neurology and Neurological Sciences ( 16 ) and the Department of Surgery ( 16 ) at Stanford; however, as shown in Table 1 , we also interviewed stakeholders from the Stanford Department of Pediatrics, Stanford School of Medicine, and Stanford Healthcare. Table 1 Participant characteristics (n = 35) Participants Department Total Neurology Surgery Other a Resident coachees 6 4 0 10 Faculty coaches 4 6 0 10 Medical education leaders 4 5 0 9 Programmatic sponsors 2 1 5 8 Total 16 16 5 37 b Note. a faculty leaders were from the Stanford Department of Pediatrics ( 2 ), Stanford School of Medicine ( 1 ), and Stanford Health Care ( 2 ); b one participant held various roles (faculty coach, medical education leader, and programmatic sponsor); therefore, the total in the table is greater than the total number of participants. Key Themes and Conceptual Frameworks Utilized Eight key themes were identified during the analysis. Figure 2 presents how those CFIR-informed themes were mapped onto the logic model. Table 2 includes illustrated quotations and the CFIR domains from which individual quotations were coded, along with their relationships to each of the logic model domains. Table 2 Representative Quotations and Associated CFIR Domains for Key Themes Logic Model Domain Theme Representative Quotations Associated CFIR Domains Context Gap in communication education - Learning communication skills was built into a lot of the rotations that we had … Just seeing physicians who are higher than me modeling family meetings… was… how I was able to learn to do it. (P3 – Coach Neurology) - It’s typically an unstructured hidden curriculum . (P12 – Medical Education Leader, Programmatic Sponsor Surgery) Intervention Characteristics Patient-centeredness - Patients are more satisfied … There’s an alliance that is formed when there is good communication. I think that not only are patients happier, but they get better care, and they are healthier, probably. (P4 – Coach Neurology) - There is a lot of therapeutic benefit to just having good rapport and communication with people. (P34 – Resident Surgery) Intervention Characteristics Inputs and Outputs Investment in program - You’ve got to have some… faculty development so that the coaches don’t feel like the blind leading the blind. (P2 – Coach Neurology) - Funding is extremely important. That comes from buy-in from the hospital, or programs, or chairs, because what makes the coaching program disseminate and sustain itself is a robust reimbursement structure. (P11 – Medical Education Leader Surgery) Intervention Characteristics, Inner setting, Characteristics of Individuals, Process Perceived program value • If it’s tailored to what [the residents] feel is important, then there will… be more buy in and engagement , and that could differ by what year the person is, maybe what specialty they’re in. (P26 – Resident Neurology) • There’s a huge value in the relationship building between trainees and faculty , which, I think is confidence building for trainees. It helps trainees… feel more established within the program in general and that they have a safety net in some ways. (P5 – Coach Surgery) Intervention Characteristics, Inner Setting Outcomes Learning or action-focused outcomes • Some of the material that we go through, that’s part of the coaching side of the material, is helpful in my own life, in my own clinic… I become a better communicator because of that. (P1 – Coach Neurology) • I think a lot of things I kind of just do… We’re very busy as residents, but I think it makes me… in line with self-reflection… it makes me more cognizant of the way I’m doing things, the way I’m phrasing things. (P34 – Resident Surgery) Characteristics of Individuals, Process Cultural or patient-related outcomes • It helps me provide better patient care if I’m better able to communicate with my patients. Obviously, if I am able to communicate more effectively with other teams, that also procures better care for the patients on my team. (P33 – Resident Surgery) • It changed the way that we approach feedback in the department, so that our learners are setting their own goals, they’re asking, they’re seeking feedback and it changed our culture from being a performance culture to being much more of a growth mindset . That has extended beyond our residents. I think part of it is just embracing that it can really revolutionize the feedback culture . (P19 – Programmatic Sponsor Pediatrics) Inner Setting and Outer Setting Evaluation Defining success • The most successful outcome is improving patients’ satisfaction scores … that’s the ultimate outcome. (P11 – Medical Education Leader Surgery) • I think, for myself, if seven years from now, by the time I’m graduating, I sense a change in the surgical culture , where people are notably nurturing each other, helping each other thrive, respectful and happy to be at work together, and are saying things where you can clearly see they’re inspired by each other, I think that would be a success. (P35 – Resident Surgery) Process Challenges with evaluation • It’s really difficult to get great outcomes data for a program like this in the sense of are our residents better communicators today than they were a year ago? That’s difficult to say… But I do know that the residents are happy with the program and on the program side, we certainly get a lot of good written feedback from the communication coaches that give a lot of insight. (P14 – Medical Education Leader Neurology) • …one of the holy grails is patient level metrics … but those are also very noisy metrics that have so many different things competing for their input that the signal to noise ratio could be really hard to separate… (P10 – Medical Education Leader Neurology) Process Domain 1: Context The themes “Gap in communication education” and “ Patient-centeredness” , which were mapped onto the Context domain of the logic model, emerged from the CFIR domain of process. 1.1. Gap in communication education : Participants reported that in their experiences with medical education, communication was infrequently taught or evaluated in a formal setting. Several participants described learning communication through direct or indirect observation rather than in a planned and explicit manner. The few who reported attending classes or receiving specific instruction focused on communication skills typically described a group setting without opportunities for individualized feedback. Thus, this communication coaching program was seen by interviewed stakeholders as a novel means of addressing an unmet need in medical education. 1.2. Patient-centeredness : Respondents emphasized the importance of patient needs as a critical motivator in improving their own communication and that of other healthcare providers. They suggested that communication was central to a healthy patient-physician relationship. Participants also highlighted the potential for enhanced communication to improve patient care through better patient-provider alliance formation, which could lead to secondary benefits in patient comprehension and adherence to recommended care. Domain 2: Inputs and Outputs The themes “Investment in program” and “Perceived program value” , which were mapped onto the Inputs and Outputs domain of the logic model, emerged from multiple CFIR domains, including intervention characteristics, inner setting, characteristics of individuals, and process. 2.1. Investment in program : Two subthemes were identified within this theme: ( 1 ) resource investment and ( 2 ) personal investment. For resource investment, our participants highlighted critical resources that were invested across multiple different layers of the program, including education, stakeholder engagement, financial support, and collaboration. For example, faculty development was utilized up-front to help prepare faculty coaches through education about coaching, providing feedback, and facilitating self-reflection. Participants also described the importance of thoughtfully engaging stakeholder groups at the planning stages when making decisions related to program resources and engagement strategies; for instance, residents were included in the interview process for selecting coaches. Financial support from and collaboration with programmatic sponsors was also described as an essential element of program success. Additionally, collaborative efforts such as the mentorship from the Department of Pediatrics, which had previously implemented a coaching program, and the collaboration with Stanford-Surgery Policy Improvement Research & Education Center for an upfront approach to program evaluation were recognized as critical investments from external sources. For personal investment, participants described the individual investment and motivations of program participants, particularly among the program’s leadership team. Their dedication to the coaching effort was thought to be critical to success, and they were frequently described as program champions because of their strong personal commitment to both communication and coaching. Personal motivations to participate were highly variable, but many participants highlighted the importance of communication as an under-addressed skill, an interest in getting more involved in teaching, or a desire for stronger resident/faculty relationships. 2.2. Perceived program value : Study participants referenced value perceptions across a continuum. Many respondents felt that addressing communication skills through an individualized coaching approach was an important adjunct to existing medical education strategies. They described having a dedicated communication coach as a uniquely valuable element of the program. Many also saw value in the program beyond the benefits to communication, highlighting particularly the value of relationship building between coach and coachee. By contrast, other participants indicated that despite the importance of developing communication skills, the rigid program structure and contrived nature of the program limited its impact and prevented thoughtful engagement. Some participants described time limitations during residency training and highlighted this as a primary challenge to effective engagement, hindering the opportunity to benefit from the program. Domain 3: Outcomes The themes “Learning or action-focused (short or mid-term) outcomes” and “Cultural or patient-related (long-term) outcomes” , which were mapped onto the Outcomes domain of the logic model, emerged from the CFIR domains of characteristics of individuals, process, inner setting, and outer setting. 3.1. Learning or action-focused (short or mid-term) outcomes : Faculty coaches and resident coachees described changes in their own communication-specific behaviors with patients and colleagues as a direct result of the learning that had taken place over the course of the program. Change took the form of increased awareness of their own challenges or limitations related to patient communication, and increased use of and comfort with communication frameworks to guide difficult patient conversations. Participants also recognized behavior change in their interpersonal interactions within healthcare teams and with their coaches/coachees. 3.2. Cultural or patient-related (long-term outcomes) : Respondents also referenced long-term outcomes, either observed or expected, including changes in culture and improved patient outcomes. Positive culture change was highlighted in multiple areas, including developing a healthier culture of feedback and creating a more nurturing environment at the department level. Domain 4: Evaluation The themes “Defining success” and “ Challenges with evaluation” , which were mapped onto the Evaluation domain of the logic model, emerged from the CFIR domain of process. 4.1 Defining success : Study participants described a wide array of potential definitions of programmatic success. While many respondents felt that patient-level data should be considered the “gold standard” of success, others suggested that success could also be measured by resident graduation readiness, faculty-specific metrics related to coaching program utilization, and even perceived department and institutional culture change. 4.2. Challenges with evaluation : Despite having a clear vision for a successful communication coaching program, participants also described a variety of challenges related to how to effectively measure success within this context. For example, participants perceived difficulty with obtaining outcomes-level data for a communication coaching program. They described challenges associated with using resident milestone evaluations for specific communication encounters. Additionally, they recognized that while improved patient outcomes would generate the most convincing outcomes data, there is considerable noise associated with patient-level metrics. Discussion Effective programmatic change at the GME level requires deliberate program implementation and rigorous program evaluation. In this study, we identified critical elements of the Stanford Neurology and Surgery Communication Coaching Program, considering program inputs, outputs, outcomes, and evaluation metrics, all within the context of our unique environment and individual stakeholder needs. We considered the implementation and evaluation of the coaching program in parallel by combining a commonly used implementation science framework, CFIR, with a common program evaluation method, the logic model. While program implementation and evaluation are distinct entities, the two go hand-in-hand and should ultimately build on each other in a cyclical fashion to make programs more effective over time and as community and stakeholder needs change. 9 Mapping the key themes identified in our analysis onto a logic model offered a more holistic description of all critical elements of the intervention and exposed areas where the program may not sufficiently meet implementation goals, and even offered suggestions for improvement. Themes that emerged from only one specific stakeholder group or one portion of the logic model may not present the full story of the program; however, in our study, multiple different perspectives contributed to the comprehensive nature of this evaluation, an essential feature of program evaluation. 9 One of the advantages of using the logic model in this way was its emphasis on the relationship of other domains to the program’s context or environment. 13 Our qualitative findings demonstrated a shared perception of a gap in communication education and an emphasis on the importance of communication from a patient perspective. These themes served as a foundation for program implementation, providing common ground for all stakeholder groups. Our findings were consistent with the known importance of a needs assessment in identifying programmatic priorities and specifically seeking to address the needs of the community. 20 The analysis also demonstrated extensive early program investment in time, funding, resources, and personnel. Although the inputs were robust, the evaluation revealed a wider range of participant experiences related to perceived program value, suggesting key differences in the degree of perceived benefit, engagement, and experience in the program. While the linear nature of a logic model has been cited as one of its limitations, 21 clear links between different elements of the model help illuminate discrepancies. Thus, the inequalities between inputs and outputs highlight a potential area for programmatic improvement to better align participant experiences with program objectives and inputs. The findings of our study also exposed a unique interplay between definitions of program success, outcomes, and challenges with evaluation. The highly varied descriptions of program success suggested distinct perceptions and experiences both by individual and stakeholder groups. This also introduced potential unintended or unexpected consequences of the program, which are essential to consider in any program evaluation. 13 Although the foundation for the program was firmly rooted in patient-centeredness and a gap in communication education, program participants described successful outcomes much more broadly – at the level of the patient, the resident, the faculty, and even the culture of the institution. We found that participants recognized outcomes and evaluation strategies at multiple Kirkpatrick levels and for various stakeholder groups (i.e., resident perceptions at Level 1, knowledge of communication strategies at Level 2, better non-coach faculty utilization of the coaching program at Level 3, and patient outcomes at Level 4). 22 Stakeholders also recognized challenges in the measurement of success according to established metrics, such as patient satisfaction scores and resident milestones. These findings ultimately informed a framework from which to consider interwoven concepts of program success, outcomes, and challenges with evaluation to align in medical education interventions (see Fig. 3 ). The challenges with existing mechanisms of evaluation further exposed the invaluable nature of the qualitative approach to participant-described outcomes. While it is understandably challenging to see an observed change in patient-level outcomes data, for instance, it brings depth and meaning to the program when participants describe their experience with change, such as the way the intervention has impacted their patient-level communication or interactions with their peers. The perceived definitions of success also indicate that there is room to consider other types of program evaluation metrics, such as perceptions of non-coach faculty, feedback culture, and other patient-level data. There are several study limitations that warrant further discussion. While 35 separate interviews were conducted, it is possible that some concepts and themes were not represented in this cohort or that findings may be specific to our institution. Participants also had varying degrees of involvement in the program; thus, their experiences may be specific to only some domains of the logic model. However, within each group of interview participants, the researchers felt that thematic saturation was adequately achieved. As well, we believe that our comprehensive program evaluation would be incomplete without input from key stakeholders across departments who can speak to different aspects of the program and various domains of the logic model. Additionally, while patients are a key stakeholder group in the program implementation, they were not included in the qualitative study. Patients have varying levels of contact with the coaching program and many interactions beyond those directly involved in the program. Thus, we determined that it would be too difficult to parse out the impact of the communication coaching program at the level of individual patients. Conclusions In conclusion, the mapping of key themes from this qualitative program evaluation onto the logic model allowed for a holistic review of the distinct yet related elements of the Stanford Neurology and Surgery Residency Communication Coaching Program. Our project has facilitated an iterative process of adjusting the program implementation efforts based on program evaluation findings. We have found this methodology to be a coherent way of linking different programmatic elements to expose strengths and areas for improvement, as well as highlight and measure intended and unintended program outcomes. We will continue to use this strategy to guide future program modifications to meet the changing needs and priorities of stakeholders. A similar methodology should be considered to link implementation and evaluation efforts for coaching programs beyond our institution and for other medical education programs at large. Abbreviations CFIR The Consolidated Framework of Implementation Research GME Graduate Medical Education Declarations Ethics approval and consent to participate: The study was reviewed by the Stanford University Institutional Review Board and deemed exempt. All methods were carried out in accordance with relevant guidelines and regulations. Informed consent was obtained from all study participants. Consent for publication: Not applicable. Availability of data and materials: The study’s data are stored securely through Stanford University. The data supporting this study’s findings are not publicly available to protect participant identity. However, upon reasonable request, they are available from the corresponding author. Competing interests: The authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and/or discussion reported in this paper. Funding: The authors wish to thank Alpa Vyas, Mysti Smith-Bentley, Dr. Justin Ko, David Entwistle, and the Stanford Health Care Donor Fund for their generous support of this program. Authors’ contributions: R.K.M., M.S., J.R.K., R.L.B., A.K.N., and C.A.G. contributed to the project design and implementation. R.M.J., M.S., U.T.M., and A.M.M. developed the approach to qualitative analysis. R.M.J., M.S., and U.T.M. conducted the analysis. R.M.J. and M.S. wrote the main manuscript text and prepared all tables and figures. All authors contributed to the interpretation of data in addition to editing and reviewing the manuscript. Acknowledgements: The authors wish to thank Christina Carter and Nicole Tomimatsu for their administrative support, in addition to all of the coaches and resident coachees for their dedication to communication skills training. References Park YS, Zar FA, Norcini JJ, Tekian A. Competency Evaluations in the Next Accreditation System: Contributing to Guidelines and Implications. Teach Learn Med. 2016;28(2):135–45. 10.1080/10401334.2016.1146607 . Bing-You R, Varaklis K, Hayes V, Trowbridge R, Kemp H, McKelvy D. The Feedback Tango: An Integrative Review and Analysis of the Content of the Teacher–Learner Feedback Exchange. Acad Med. 2018;93(4):657–63. 10.1097/ACM.0000000000001927 . Rassbach CE, Blankenburg R. A Novel Pediatric Residency Coaching Program: Outcomes After One Year. Acad Med. 2018;93(3):430–4. 10.1097/ACM.0000000000001825 . Palamara K, Chu JT, Chang Y, et al. Who Benefits Most? A Multisite Study of Coaching and Resident Well-being. J GEN INTERN MED. 2022;37(3):539–47. 10.1007/s11606-021-06903-5 . Sasnal M, Miller-Kuhlmann R, Merrell SB, et al. Feasibility and acceptability of virtually coaching residents on communication skills: a pilot study. BMC Med Educ. 2021;21(1):513. 10.1186/s12909-021-02936-w . Deiorio NM, Foster KW, Santen SA. Coaching a Learner in Medical Education. Acad Med. 2021;96(12):1758. 10.1097/ACM.0000000000004168 . Rassbach CE, Bogetz AL, Orlov N, et al. The Effect of Faculty Coaching on Resident Attitudes, Confidence, and Patient-Rated Communication: A Multi-Institutional Randomized Controlled Trial. Acad Pediatr. 2019;19(2):186–94. 10.1016/j.acap.2018.10.004 . Whitmore J. Coaching for Performance: GROWing Human Potential and Purpose - The Principles and Practice of Coaching and Leadership . 4th ed.; 2009. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE Guide 67. Med Teach. 2012;34(5):e288–99. 10.3109/0142159X.2012.668637 . Musick DW. A Conceptual Model for Program Evaluation in Graduate Medical Education. Acad Med. 2006;81(8):759. Vassar M, Wheeler DL, Davison M, Franklin J. Program Evaluation in Medical Education: An Overview of the Utilization-focused Approach. J Educ Eval Health Prof. 2010;7:1. 10.3352/jeehp.2010.7.1 . Melle EV. Using a Logic Model to Assist in the Planning, Implementation, and Evaluation of Educational Programs. Acad Med. 2016;91(10):1464. 10.1097/ACM.0000000000001282 . Frechtling JA. Logic Modeling Methods in Program Evaluation. Jossey-Bass/Wiley; 2007. p. xiii. Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum Development for Medical Education: A Six-Step Approach . Johns Hopkins University Press; 2015. Accessed August 9, 2022. https://jhu.pure.elsevier.com/en/publications/curriculum-development-for-medical-education-a-six-step-approach . Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. 10.1186/1748-5908-4-50 . Patton MQ. Qualitative Research & Evaluation Methods: Integrating Theory and Practice. SAGE; 2014. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101. 10.1191/1478088706qp063oa . Thematic Analysis: Striving to Meet the Trustworthiness Criteria - Lorelli S. Nowell, Jill M. Norris, Deborah E. White, Nancy J. Moules. 2017. Accessed April 19, 2023. https://journals.sagepub.com/doi/10.1177/1609406917733847 . Creswell JW. 30 Essential Skills for the Qualitative Researcher. SAGE; 2016. Grant J. Learning needs assessment: assessing the need. BMJ. 2002;324(7330):156–9. Patton MQ. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press; 2011. p. xxi. Kirkpatrick DL. Evaluating Training Programs: The Four Levels . First edition. San Francisco: Berrett-Koehler ; Emeryville, CA : Publishers Group West [distributor], [1994] ©1994; 1994. https://search.library.wisc.edu/catalog/999764622302121 . Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 25 Apr, 2025 Read the published version in BMC Medical Education → Version 1 posted Editorial decision: Revision requested 21 Feb, 2025 Reviews received at journal 20 Feb, 2025 Reviews received at journal 11 Feb, 2025 Reviewers agreed at journal 10 Feb, 2025 Reviewers agreed at journal 22 Jan, 2025 Reviewers invited by journal 18 Aug, 2024 Editor assigned by journal 30 Jul, 2024 Editor invited by journal 15 May, 2024 Submission checks completed at journal 15 May, 2024 First submitted to journal 05 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4372302","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":305002680,"identity":"1f23e32e-7393-4dd0-bd27-199cb24208cd","order_by":0,"name":"Rachel M. Jensen","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Rachel","middleName":"M.","lastName":"Jensen","suffix":""},{"id":305002681,"identity":"63990b01-18a8-40fa-bbdb-5aca41026c90","order_by":1,"name":"Marzena Sasnal","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Marzena","middleName":"","lastName":"Sasnal","suffix":""},{"id":305002682,"identity":"13d1e66f-745b-49ab-918e-848162a6ad99","order_by":2,"name":"Uyen T. Mai","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Uyen","middleName":"T.","lastName":"Mai","suffix":""},{"id":305002684,"identity":"e1f8a977-7f7c-41ad-9474-0b28592f2b84","order_by":3,"name":"James R. Korndorffer","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"James","middleName":"R.","lastName":"Korndorffer","suffix":""},{"id":305002685,"identity":"2a237179-1ae7-4784-8727-7d8b75ef74ea","order_by":4,"name":"Rebecca K. Miller-Kuhlmann","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Rebecca","middleName":"K.","lastName":"Miller-Kuhlmann","suffix":""},{"id":305002687,"identity":"4c92231d-45db-4127-b8d8-51f25a6fa1be","order_by":5,"name":"Arden M. Morris","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Arden","middleName":"M.","lastName":"Morris","suffix":""},{"id":305002690,"identity":"d0f592d0-d1f1-408c-84e0-68192fd8d7a6","order_by":6,"name":"Aussama K. Nassar","email":"","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Aussama","middleName":"K.","lastName":"Nassar","suffix":""},{"id":305002692,"identity":"ba334273-c899-41f4-bfaf-19cd58c8121d","order_by":7,"name":"Carl A. Gold","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA60lEQVRIie3PsQoCIRjAcUW4ybrVuHoHIYiiexjjwMYaG4OgRteCHiIImhWHlqJVaCmCppYo4qAlr6YWr7Yg/yAofD9UAHy+3wxKAOpQPPexXfZI8kw2Ayd9ZLf8GzKTn5JwtN6pa49U5puNunRZ3ClKpLbYQciqTXV5RaoLk6BozHhjIoOk6SLUvkWXhqS1MAhEhVRTKnEtcpLw+CJzodEds4yENzchHKizJTOQBNGL4MBJiLG3APuXsUlqTcw4Lemg2pg6SCg43Ke9uCKEOmwxi2lxOdibk4NkofdnoJzxLJh+MOTz+Xx/3ANXlE0sfoj5uQAAAABJRU5ErkJggg==","orcid":"","institution":"Stanford University School of Medicine","correspondingAuthor":true,"prefix":"","firstName":"Carl","middleName":"A.","lastName":"Gold","suffix":""}],"badges":[],"createdAt":"2024-05-05 15:31:03","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4372302/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4372302/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12909-025-07188-6","type":"published","date":"2025-04-25T15:57:10+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":57444480,"identity":"3138acce-09e2-4809-8a70-5773ef2ac5f7","added_by":"auto","created_at":"2024-05-30 18:56:23","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":3213519,"visible":true,"origin":"","legend":"\u003cp\u003eFive Domains Within the CFIR Framework\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNote. \u003c/strong\u003eFigure 1 depicts the 5 domains within the Consolidated Framework of Implementation Research (CFIR), which include intervention characteristics, inner setting, characteristics of individuals, process, and outer setting. A brief description of each domain within the context of the coaching program is also included.\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4372302/v1/cf6ac7d018439761b2d328ac.jpg"},{"id":57442788,"identity":"cc1c2f2b-cd0d-4e79-83b2-361e515ce401","added_by":"auto","created_at":"2024-05-30 18:48:23","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":1661576,"visible":true,"origin":"","legend":"\u003cp\u003eKey Themes Mapped onto a Logic Model for Coaching Program Evaluation\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNote.\u003c/strong\u003e Figure 2 illustrates each of the key themes (\u003cem\u003eitalics\u003c/em\u003e) identified in the qualitative analysis, mapped onto a distinct domain of a logic model of program evaluation.\u003c/p\u003e","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4372302/v1/11d42347a14b45acb67de1d2.jpg"},{"id":57442791,"identity":"ca86b389-c0a5-405a-881b-1591b55e8b80","added_by":"auto","created_at":"2024-05-30 18:48:24","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":2108186,"visible":true,"origin":"","legend":"\u003cp\u003eFramework for Success, Outcomes, and Challenges with Evaluation\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNote.\u003c/strong\u003e Figure 3 depicts the complex interplay between definitions of success, outcomes being measured, and evaluation challenges given the limitations of current outcomes metrics.\u003c/p\u003e","description":"","filename":"Figure3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4372302/v1/5e0a625a051b627b75fdafea.jpg"},{"id":81569561,"identity":"63991b2c-5317-48df-9ce8-bd059f634235","added_by":"auto","created_at":"2025-04-28 16:07:15","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":7951722,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4372302/v1/8b927510-8f7c-4ee5-af2c-9ca15aa2282e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Implementation and Evaluation of a Communication Coaching Program: A CFIR-Informed Qualitative Analysis Mapped onto a Logic Model","fulltext":[{"header":"Background","content":"\u003cp\u003eCommunication is central to the role of a competent physician, yet this core competency is taught or assessed directly less often than competencies such as patient care and medical knowledge.\u003csup\u003e1\u003c/sup\u003e Lack of education training limits the potential for feedback and thoughtful self-reflection on communication skills, which are essential to a trainee\u0026rsquo;s personal-professional growth.\u003csup\u003e2\u003c/sup\u003e More recently, coaching programs at the graduate medical education (GME) level have been implemented to address the specific needs of trainees.\u003csup\u003e3\u0026ndash;5\u003c/sup\u003e Coaching, which is distinct from mentorship or advising, involves a true partnership between a coach and a coachee.\u003csup\u003e6\u003c/sup\u003e A coaching framework is learner-centered and helps the coachee identify opportunities for improvement through facilitated self-reflection and goal setting.\u003csup\u003e7,8\u003c/sup\u003e Coaching programs have enormous potential to impact resident growth across multiple core competencies, including practice-based learning and improvement, professionalism, and interpersonal skills and communication.\u003csup\u003e3,5\u003c/sup\u003e Proficiency-focused efforts are critical to help learners progress along milestones in the current era of competency-based medical education.\u003c/p\u003e \u003cp\u003eDespite the strong potential of coaching programs, it is unclear whether coaching implementation efforts are effectively meeting the needs of stakeholders and inducing true change in learners.\u003csup\u003e9\u003c/sup\u003e Effective programmatic change requires deliberate implementation and rigorous evaluation. There is an opportunity cost associated with any new implementation effort in GME; the time and effort that a coach or coachee puts into one endeavor inevitably means less time and effort elsewhere. For this reason, programs must make difficult choices and evolve to effectively meet stakeholders\u0026rsquo; needs.\u003csup\u003e10,11\u003c/sup\u003e One common framework for programmatic implementation and evaluation is a logic model, which helps to balance community needs, program inputs and outputs, outcome measurements, and evaluation strategies.\u003csup\u003e12,13\u003c/sup\u003e A logic model can be used in an \u003cem\u003ea priori\u003c/em\u003e fashion to assist with program design and implementation or in a \u003cem\u003epost hoc\u003c/em\u003e setting for program evaluation.\u003c/p\u003e \u003cp\u003eTo evaluate outcomes related to a coaching intervention, programs often rely on survey-based quasi-experimental study designs; however, this approach may provide a limited view of a program\u0026rsquo;s impact and does not allow for exploration beyond the survey\u0026rsquo;s measured constructs.\u003csup\u003e9\u003c/sup\u003e Qualitative approaches, using individual interviews or focus groups, have the potential to provide a more robust program evaluation, especially for complex interventions with multiple stakeholders and moving parts. Therefore, we conducted a qualitative program evaluation of a faculty-led communication coaching program implemented at a single institution for Surgery and Neurology residents using semi-structured interviews with key program stakeholders.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eProgram Implementation\u003c/h2\u003e \u003cp\u003eStanford University implemented a faculty-led communication-focused coaching program in the Departments of Surgery and Neurology in 2020 as previously described.\u003csup\u003e5\u003c/sup\u003e Briefly, the program was designed using Kern\u0026rsquo;s 6-step model of curriculum development and utilized the Consolidated Framework for Implementation Research in Formative Evaluation (CFIR) as an implementation framework.\u003csup\u003e14,15\u003c/sup\u003e The CFIR framework breaks down the implementation into 5 domains; see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e for the key domains and a description of each. Notably, the program was designed with input from multiple stakeholder groups, including resident coachees, faculty coaches, medical education leaders, and programmatic sponsors, as well as collaborative efforts across multiple levels to ensure a rigorous plan for programmatic evaluation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eParticipants and Oversight\u003c/h2\u003e \u003cp\u003eWe employed a key informant sampling strategy to purposively select study participants, including resident coachees, faculty coaches, medical education leaders, and programmatic sponsors.\u003csup\u003e16\u003c/sup\u003e Those four diverse stakeholder groups were, in various roles and functions, involved in the program\u0026rsquo;s design and implementation. Respondents were recruited via email sent by communication coaching directors (C.A.G. and A.K.N.) or a research analyst (M.S). Verbal consent was obtained from all participants, and the study was exempt from the IRB review as a quality improvement project.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eSemi-Structured Interviews\u003c/h2\u003e \u003cp\u003eCFIR-informed semi-structured interviews conducted between November 2021 and April 2022 aimed to capture stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders\u0026rsquo; involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The interviews were conducted by a research analyst experienced in qualitative methods (M.S.), lasted 30\u0026ndash;45 minutes, and took place via the Zoom (Zoom Video Communications Inc.) videoconference platform. Interviews were recorded, transcribed verbatim, and deidentified for analysis.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eAnalytic Approach\u003c/h2\u003e \u003cp\u003eA rigorous team-based thematic analysis of the interview transcripts was conducted involving the following six steps: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) familiarization with the data, (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) generating initial codes, (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) searching for themes, (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e) reviewing themes, (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e) defining and naming themes, and (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e) producing the report.\u003csup\u003e17\u003c/sup\u003e Analytical procedures, including coding and assessing inter-coder agreement, were performed using NVivo qualitative software (Version Pro Enterprise, QSR International Pty Ltd, Massachusetts, USA, 2020). Trustworthiness during each phase of thematic analysis was established by various means, including prolonged engagement with data, peer debriefing, researcher triangulation, use of coding framework, themes and subthemes vetted by team members, team consensus on themes, and thick description of the context.\u003csup\u003e18\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eTo develop a codebook, three team members (R.M.J., M.S., and U.T.M.) first inductively coded four interviews and then met multiple times to discuss emerging patterns, meanings, and how they fit into the CFIR framework. Then, R.M.J. deductively coded the same set of four interviews, using CFIR constructs as codes, and developed the first draft of the codebook. Next, M.S. and U.T.M. validated the codebook by applying it when independently coding the same four interviews. During this intensive analytical phase, coders frequently met to compare coding, discuss ambiguities, and make adaptions based on the findings, which resulted in the development of a 20-item CFIR-informed codebook. The codebook was then vetted by the whole analytics team (A.K.N., R.K.M., C.A.G., J.R.K., and A.M.M.). After the strong inter-rater agreement was reached (Cohen\u0026rsquo;s kappa coefficient \u003cem\u003eκ\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.83) between M.S., R.M.J, and U.T.M for the three newly coded interviews, which ensured coding consistency, the coders divided transcripts and coded the remaining interviews.\u003csup\u003e19\u003c/sup\u003e Throughout the coding and interpretation phases, coders frequently engaged in consensus discussion to identify emerging themes and resolve disagreements. The entire team subsequently reviewed and verified these findings.\u003c/p\u003e \u003cp\u003eThe themes that emerged from the CFIR-informed qualitative analysis, were then mapped onto four critical domains of a standard logic model: Context, Inputs and Outputs, Outcomes, and Evaluation. The Context domain describes contextual factors, priorities, and the program landscape as key features of implementation and evaluation. The Inputs and Outputs domain describes the resources invested in the program such as funding, time, skills, technology, and facilities, as well as the personal investment of program personnel and their individual motivation or incentive to engage in the program. This domain also includes the program's content or activities, including program execution and participation. The Outcomes domain addresses the perceived program outcomes and impact. The Evaluation domain focuses on the specifics of programmatic evaluation.\u003csup\u003e12,13\u003c/sup\u003e\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eParticipant characteristics\u003c/h2\u003e \u003cp\u003eThirty-five stakeholders, including 10 resident coachees (received coaching), 10 faculty coaches (served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (designed and implemented program), and programmatic sponsors (provided financial support), participated in the interviews. Respondents were mainly employees and trainees of the Department of Neurology and Neurological Sciences (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) and the Department of Surgery (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) at Stanford; however, as shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, we also interviewed stakeholders from the Stanford Department of Pediatrics, Stanford School of Medicine, and Stanford Healthcare.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eParticipant characteristics (n\u0026thinsp;=\u0026thinsp;35)\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eParticipants\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eDepartment\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cem\u003eTotal\u003c/em\u003e\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNeurology\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSurgery\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eOther\u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResident coachees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003e10\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eFaculty coaches\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003e10\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMedical education leaders\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003e9\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eProgrammatic sponsors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003e8\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eTotal\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003e16\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cem\u003e16\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cem\u003e5\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cem\u003e37\u003c/em\u003e\u003csup\u003e\u003cem\u003eb\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"5\"\u003e\u003cb\u003eNote.\u003c/b\u003e \u003csup\u003ea\u003c/sup\u003e faculty leaders were from the Stanford Department of Pediatrics (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e), Stanford School of Medicine (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e), and Stanford Health Care (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e); \u003csup\u003eb\u003c/sup\u003eone participant held various roles (faculty coach, medical education leader, and programmatic sponsor); therefore, the total in the table is greater than the total number of participants.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003eKey Themes and Conceptual Frameworks Utilized\u003c/h2\u003e \u003cp\u003eEight key themes were identified during the analysis. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e2\u003c/span\u003e presents how those CFIR-informed themes were mapped onto the logic model. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e includes illustrated quotations and the CFIR domains from which individual quotations were coded, along with their relationships to each of the logic model domains.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRepresentative Quotations and Associated CFIR Domains for Key Themes\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLogic Model Domain\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTheme\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRepresentative Quotations\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAssociated CFIR Domains\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eContext\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGap in communication education\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- \u003cem\u003eLearning communication skills was\u003c/em\u003e \u003cb\u003ebuilt into a lot of the rotations that we had\u003c/b\u003e\u003cem\u003e\u0026hellip; Just seeing physicians who are higher than me\u003c/em\u003e \u003cb\u003emodeling\u003c/b\u003e \u003cem\u003efamily meetings\u0026hellip; was\u0026hellip; how I was able to learn to do it.\u003c/em\u003e (P3 \u0026ndash; Coach Neurology)\u003c/p\u003e \u003cp\u003e- \u003cem\u003eIt\u0026rsquo;s typically an\u003c/em\u003e \u003cb\u003eunstructured hidden curriculum\u003c/b\u003e. (P12 \u0026ndash; Medical Education Leader, Programmatic Sponsor Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eIntervention Characteristics\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePatient-centeredness\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- \u003cem\u003ePatients are more\u003c/em\u003e \u003cb\u003esatisfied\u003c/b\u003e\u003cem\u003e\u0026hellip; There\u0026rsquo;s an\u003c/em\u003e \u003cb\u003ealliance\u003c/b\u003e \u003cem\u003ethat is formed when there is good communication. I think that not only are patients happier, but they get better care, and they are healthier, probably.\u003c/em\u003e (P4 \u0026ndash; Coach Neurology)\u003c/p\u003e \u003cp\u003e- \u003cem\u003eThere is a lot of\u003c/em\u003e \u003cb\u003etherapeutic benefit\u003c/b\u003e \u003cem\u003eto just having good rapport and communication with people.\u003c/em\u003e (P34 \u0026ndash; Resident Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eIntervention Characteristics\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eInputs and Outputs\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eInvestment in program\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e- \u003cem\u003eYou\u0026rsquo;ve got to have some\u0026hellip;\u003c/em\u003e \u003cb\u003efaculty development\u003c/b\u003e \u003cem\u003eso that the coaches don\u0026rsquo;t feel like the blind leading the blind.\u003c/em\u003e (P2 \u0026ndash; Coach Neurology)\u003c/p\u003e \u003cp\u003e- \u003cb\u003eFunding\u003c/b\u003e \u003cem\u003eis extremely important. That comes from\u003c/em\u003e \u003cb\u003ebuy-in\u003c/b\u003e \u003cem\u003efrom the hospital, or programs, or chairs, because what makes the coaching program disseminate and sustain itself is a robust reimbursement structure.\u003c/em\u003e (P11 \u0026ndash; Medical Education Leader Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eIntervention Characteristics, Inner setting, Characteristics of Individuals, Process\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePerceived program value\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026bull; \u003cem\u003eIf it\u0026rsquo;s tailored to what [the residents] feel is important, then there will\u0026hellip; be more\u003c/em\u003e \u003cb\u003ebuy in and engagement\u003c/b\u003e, \u003cem\u003eand that could differ by what year the person is, maybe what specialty they\u0026rsquo;re in.\u003c/em\u003e (P26 \u0026ndash; Resident Neurology)\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cem\u003eThere\u0026rsquo;s a huge value in the\u003c/em\u003e \u003cb\u003erelationship building between trainees and faculty\u003c/b\u003e, \u003cem\u003ewhich, I think is confidence building for trainees. It helps trainees\u0026hellip; feel more\u003c/em\u003e \u003cb\u003eestablished\u003c/b\u003e \u003cem\u003ewithin the program in general and that they have a safety net in some ways.\u003c/em\u003e (P5 \u0026ndash; Coach Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eIntervention Characteristics, Inner Setting\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eOutcomes\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eLearning or action-focused outcomes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026bull; \u003cem\u003eSome of the material that we go through, that\u0026rsquo;s part of the coaching side of the material, is helpful in my own life, in my own clinic\u0026hellip; I become a\u003c/em\u003e \u003cb\u003ebetter communicator\u003c/b\u003e \u003cem\u003ebecause of that.\u003c/em\u003e (P1 \u0026ndash; Coach Neurology)\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cem\u003eI think a lot of things I kind of just do\u0026hellip; We\u0026rsquo;re very busy as residents, but I think it makes me\u0026hellip; in line with self-reflection\u0026hellip; it makes me\u003c/em\u003e \u003cb\u003emore cognizant\u003c/b\u003e \u003cem\u003eof the way I\u0026rsquo;m doing things, the way I\u0026rsquo;m phrasing things.\u003c/em\u003e (P34 \u0026ndash; Resident Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eCharacteristics of Individuals, Process\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCultural or patient-related outcomes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026bull; \u003cem\u003eIt helps me provide\u003c/em\u003e \u003cb\u003ebetter patient care\u003c/b\u003e \u003cem\u003eif I\u0026rsquo;m better able to communicate with my patients. Obviously, if I am able to communicate more effectively with other teams, that also procures better care for the patients on my team.\u003c/em\u003e (P33 \u0026ndash; Resident Surgery)\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cem\u003eIt changed the way that we approach feedback in the department, so that our learners are setting their own goals, they\u0026rsquo;re asking, they\u0026rsquo;re seeking feedback and it changed our culture from being a performance culture to being much more of a\u003c/em\u003e \u003cb\u003egrowth mindset\u003c/b\u003e. \u003cem\u003eThat has extended beyond our residents. I think part of it is just embracing that it can really\u003c/em\u003e \u003cb\u003erevolutionize the feedback culture\u003c/b\u003e. (P19 \u0026ndash; Programmatic Sponsor Pediatrics)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eInner Setting and Outer Setting\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eEvaluation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDefining success\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026bull; \u003cem\u003eThe most successful outcome is\u003c/em\u003e \u003cb\u003eimproving patients\u0026rsquo; satisfaction scores\u003c/b\u003e\u003cem\u003e\u0026hellip; that\u0026rsquo;s the ultimate outcome.\u003c/em\u003e (P11 \u0026ndash; Medical Education Leader Surgery)\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cem\u003eI think, for myself, if seven years from now, by the time I\u0026rsquo;m graduating, I sense a\u003c/em\u003e \u003cb\u003echange in the surgical culture\u003c/b\u003e, \u003cem\u003ewhere people are notably nurturing each other, helping each other thrive, respectful and happy to be at work together, and are saying things where you can clearly see they\u0026rsquo;re inspired by each other, I think that would be a success.\u003c/em\u003e (P35 \u0026ndash; Resident Surgery)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eProcess\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eChallenges with evaluation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026bull; \u003cem\u003eIt\u0026rsquo;s really\u003c/em\u003e \u003cb\u003edifficult to get great outcomes data\u003c/b\u003e \u003cem\u003efor a program like this in the sense of are our residents better communicators today than they were a year ago? That\u0026rsquo;s difficult to say\u0026hellip; But I do know that the residents are happy with the program and on the program side, we certainly get a lot of good written feedback from the communication coaches that give a lot of insight.\u003c/em\u003e (P14 \u0026ndash; Medical Education Leader Neurology)\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cem\u003e\u0026hellip;one of the holy grails is\u003c/em\u003e \u003cb\u003epatient level metrics\u003c/b\u003e\u003cem\u003e\u0026hellip; but those are also very\u003c/em\u003e \u003cb\u003enoisy\u003c/b\u003e \u003cem\u003emetrics that have so many different things competing for their input that the signal to noise ratio could be really hard to separate\u0026hellip;\u003c/em\u003e (P10 \u0026ndash; Medical Education Leader Neurology)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eProcess\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eDomain 1: Context\u003c/h2\u003e \u003cp\u003eThe themes \u003cem\u003e\u0026ldquo;Gap in communication education\u0026rdquo;\u003c/em\u003e and \u0026ldquo;\u003cem\u003ePatient-centeredness\u0026rdquo;\u003c/em\u003e, which were mapped onto the Context domain of the logic model, emerged from the CFIR domain of process.\u003c/p\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e1.1. Gap in communication education\u003c/span\u003e: Participants reported that in their experiences with medical education, communication was infrequently taught or evaluated in a formal setting. Several participants described learning communication through direct or indirect observation rather than in a planned and explicit manner. The few who reported attending classes or receiving specific instruction focused on communication skills typically described a group setting without opportunities for individualized feedback. Thus, this communication coaching program was seen by interviewed stakeholders as a novel means of addressing an unmet need in medical education.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e1.2. Patient-centeredness\u003c/span\u003e: Respondents emphasized the importance of patient needs as a critical motivator in improving their own communication and that of other healthcare providers. They suggested that communication was central to a healthy patient-physician relationship. Participants also highlighted the potential for enhanced communication to improve patient care through better patient-provider alliance formation, which could lead to secondary benefits in patient comprehension and adherence to recommended care.\u003c/p\u003e\u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eDomain 2: Inputs and Outputs\u003c/h2\u003e \u003cp\u003eThe themes \u003cem\u003e\u0026ldquo;Investment in program\u0026rdquo;\u003c/em\u003e and \u003cem\u003e\u0026ldquo;Perceived program value\u0026rdquo;\u003c/em\u003e, which were mapped onto the Inputs and Outputs domain of the logic model, emerged from multiple CFIR domains, including intervention characteristics, inner setting, characteristics of individuals, and process.\u003c/p\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e2.1. Investment in program\u003c/span\u003e: Two subthemes were identified within this theme: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) resource investment and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) personal investment. For resource investment, our participants highlighted critical resources that were invested across multiple different layers of the program, including education, stakeholder engagement, financial support, and collaboration. For example, faculty development was utilized up-front to help prepare faculty coaches through education about coaching, providing feedback, and facilitating self-reflection. Participants also described the importance of thoughtfully engaging stakeholder groups at the planning stages when making decisions related to program resources and engagement strategies; for instance, residents were included in the interview process for selecting coaches. Financial support from and collaboration with programmatic sponsors was also described as an essential element of program success. Additionally, collaborative efforts such as the mentorship from the Department of Pediatrics, which had previously implemented a coaching program, and the collaboration with Stanford-Surgery Policy Improvement Research \u0026amp; Education Center for an upfront approach to program evaluation were recognized as critical investments from external sources. For personal investment, participants described the individual investment and motivations of program participants, particularly among the program\u0026rsquo;s leadership team. Their dedication to the coaching effort was thought to be critical to success, and they were frequently described as program champions because of their strong personal commitment to both communication and coaching. Personal motivations to participate were highly variable, but many participants highlighted the importance of communication as an under-addressed skill, an interest in getting more involved in teaching, or a desire for stronger resident/faculty relationships.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e2.2. Perceived program value\u003c/span\u003e: Study participants referenced value perceptions across a continuum. Many respondents felt that addressing communication skills through an individualized coaching approach was an important adjunct to existing medical education strategies. They described having a dedicated communication coach as a uniquely valuable element of the program. Many also saw value in the program beyond the benefits to communication, highlighting particularly the value of relationship building between coach and coachee. By contrast, other participants indicated that despite the importance of developing communication skills, the rigid program structure and contrived nature of the program limited its impact and prevented thoughtful engagement. Some participants described time limitations during residency training and highlighted this as a primary challenge to effective engagement, hindering the opportunity to benefit from the program.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eDomain 3: Outcomes\u003c/h2\u003e \u003cp\u003eThe themes \u003cem\u003e\u0026ldquo;Learning or action-focused (short or mid-term) outcomes\u0026rdquo;\u003c/em\u003e and \u003cem\u003e\u0026ldquo;Cultural or patient-related (long-term) outcomes\u0026rdquo;\u003c/em\u003e, which were mapped onto the Outcomes domain of the logic model, emerged from the CFIR domains of characteristics of individuals, process, inner setting, and outer setting.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e3.1. Learning or action-focused (short or mid-term) outcomes\u003c/span\u003e: Faculty coaches and resident coachees described changes in their own communication-specific behaviors with patients and colleagues as a direct result of the learning that had taken place over the course of the program. Change took the form of increased awareness of their own challenges or limitations related to patient communication, and increased use of and comfort with communication frameworks to guide difficult patient conversations. Participants also recognized behavior change in their interpersonal interactions within healthcare teams and with their coaches/coachees.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e3.2. Cultural or patient-related (long-term outcomes)\u003c/span\u003e: Respondents also referenced long-term outcomes, either observed or expected, including changes in culture and improved patient outcomes. Positive culture change was highlighted in multiple areas, including developing a healthier culture of feedback and creating a more nurturing environment at the department level.\u003c/p\u003e\u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eDomain 4: Evaluation\u003c/h2\u003e \u003cp\u003eThe themes \u003cem\u003e\u0026ldquo;Defining success\u0026rdquo;\u003c/em\u003e and \u0026ldquo;\u003cem\u003eChallenges with evaluation\u0026rdquo;\u003c/em\u003e, which were mapped onto the Evaluation domain of the logic model, emerged from the CFIR domain of process.\u003c/p\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e4.1 Defining success\u003c/span\u003e: Study participants described a wide array of potential definitions of programmatic success. While many respondents felt that patient-level data should be considered the \u0026ldquo;gold standard\u0026rdquo; of success, others suggested that success could also be measured by resident graduation readiness, faculty-specific metrics related to coaching program utilization, and even perceived department and institutional culture change.\u003c/p\u003e\u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003e4.2. Challenges with evaluation\u003c/span\u003e: Despite having a clear vision for a successful communication coaching program, participants also described a variety of challenges related to how to effectively measure success within this context. For example, participants perceived difficulty with obtaining outcomes-level data for a communication coaching program. They described challenges associated with using resident milestone evaluations for specific communication encounters. Additionally, they recognized that while improved patient outcomes would generate the most convincing outcomes data, there is considerable noise associated with patient-level metrics.\u003c/p\u003e\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eEffective programmatic change at the GME level requires deliberate program implementation and rigorous program evaluation. In this study, we identified critical elements of the Stanford Neurology and Surgery Communication Coaching Program, considering program inputs, outputs, outcomes, and evaluation metrics, all within the context of our unique environment and individual stakeholder needs. We considered the implementation and evaluation of the coaching program in parallel by combining a commonly used implementation science framework, CFIR, with a common program evaluation method, the logic model.\u003c/p\u003e \u003cp\u003eWhile program implementation and evaluation are distinct entities, the two go hand-in-hand and should ultimately build on each other in a cyclical fashion to make programs more effective over time and as community and stakeholder needs change.\u003csup\u003e9\u003c/sup\u003e Mapping the key themes identified in our analysis onto a logic model offered a more holistic description of all critical elements of the intervention and exposed areas where the program may not sufficiently meet implementation goals, and even offered suggestions for improvement. Themes that emerged from only one specific stakeholder group or one portion of the logic model may not present the full story of the program; however, in our study, multiple different perspectives contributed to the comprehensive nature of this evaluation, an essential feature of program evaluation.\u003csup\u003e9\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eOne of the advantages of using the logic model in this way was its emphasis on the relationship of other domains to the program\u0026rsquo;s context or environment.\u003csup\u003e13\u003c/sup\u003e Our qualitative findings demonstrated a shared perception of a gap in communication education and an emphasis on the importance of communication from a patient perspective. These themes served as a foundation for program implementation, providing common ground for all stakeholder groups. Our findings were consistent with the known importance of a needs assessment in identifying programmatic priorities and specifically seeking to address the needs of the community.\u003csup\u003e20\u003c/sup\u003e The analysis also demonstrated extensive early program investment in time, funding, resources, and personnel. Although the inputs were robust, the evaluation revealed a wider range of participant experiences related to perceived program value, suggesting key differences in the degree of perceived benefit, engagement, and experience in the program. While the linear nature of a logic model has been cited as one of its limitations,\u003csup\u003e21\u003c/sup\u003e clear links between different elements of the model help illuminate discrepancies. Thus, the inequalities between inputs and outputs highlight a potential area for programmatic improvement to better align participant experiences with program objectives and inputs.\u003c/p\u003e \u003cp\u003eThe findings of our study also exposed a unique interplay between definitions of program success, outcomes, and challenges with evaluation. The highly varied descriptions of program success suggested distinct perceptions and experiences both by individual and stakeholder groups. This also introduced potential unintended or unexpected consequences of the program, which are essential to consider in any program evaluation.\u003csup\u003e13\u003c/sup\u003e Although the foundation for the program was firmly rooted in patient-centeredness and a gap in communication education, program participants described successful outcomes much more broadly \u0026ndash; at the level of the patient, the resident, the faculty, and even the culture of the institution. We found that participants recognized outcomes and evaluation strategies at multiple Kirkpatrick levels and for various stakeholder groups (i.e., resident perceptions at Level 1, knowledge of communication strategies at Level 2, better non-coach faculty utilization of the coaching program at Level 3, and patient outcomes at Level 4).\u003csup\u003e22\u003c/sup\u003e Stakeholders also recognized challenges in the measurement of success according to established metrics, such as patient satisfaction scores and resident milestones. These findings ultimately informed a framework from which to consider interwoven concepts of program success, outcomes, and challenges with evaluation to align in medical education interventions (see Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe challenges with existing mechanisms of evaluation further exposed the invaluable nature of the qualitative approach to participant-described outcomes. While it is understandably challenging to see an observed change in patient-level outcomes data, for instance, it brings depth and meaning to the program when participants describe their experience with change, such as the way the intervention has impacted their patient-level communication or interactions with their peers. The perceived definitions of success also indicate that there is room to consider other types of program evaluation metrics, such as perceptions of non-coach faculty, feedback culture, and other patient-level data.\u003c/p\u003e \u003cp\u003eThere are several study limitations that warrant further discussion. While 35 separate interviews were conducted, it is possible that some concepts and themes were not represented in this cohort or that findings may be specific to our institution. Participants also had varying degrees of involvement in the program; thus, their experiences may be specific to only some domains of the logic model. However, within each group of interview participants, the researchers felt that thematic saturation was adequately achieved. As well, we believe that our comprehensive program evaluation would be incomplete without input from key stakeholders across departments who can speak to different aspects of the program and various domains of the logic model. Additionally, while patients are a key stakeholder group in the program implementation, they were not included in the qualitative study. Patients have varying levels of contact with the coaching program and many interactions beyond those directly involved in the program. Thus, we determined that it would be too difficult to parse out the impact of the communication coaching program at the level of individual patients.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eIn conclusion, the mapping of key themes from this qualitative program evaluation onto the logic model allowed for a holistic review of the distinct yet related elements of the Stanford Neurology and Surgery Residency Communication Coaching Program. Our project has facilitated an iterative process of adjusting the program implementation efforts based on program evaluation findings. We have found this methodology to be a coherent way of linking different programmatic elements to expose strengths and areas for improvement, as well as highlight and measure intended and unintended program outcomes. We will continue to use this strategy to guide future program modifications to meet the changing needs and priorities of stakeholders. A similar methodology should be considered to link implementation and evaluation efforts for coaching programs beyond our institution and for other medical education programs at large.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eCFIR\u0026nbsp; \u0026nbsp; \u0026nbsp;The Consolidated Framework of Implementation Research\u003c/p\u003e\n\u003cp\u003eGME \u0026nbsp; \u0026nbsp;Graduate Medical Education\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cem\u003eEthics approval and consent to participate:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe study was reviewed by the Stanford University Institutional Review Board and deemed exempt. All methods were carried out in accordance with relevant guidelines and regulations. Informed consent was obtained from all study participants.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eConsent for publication:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAvailability of data and materials:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe study’s data are stored securely through Stanford University. The data supporting this study’s findings are not publicly available to protect participant identity. However, upon reasonable request, they are available from the corresponding author.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eCompeting interests:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and/or discussion reported in this paper.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFunding:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe authors wish to thank Alpa Vyas, Mysti Smith-Bentley, Dr. Justin Ko, David Entwistle, and the Stanford Health Care Donor Fund for their generous support of this program.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAuthors’ contributions:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eR.K.M., M.S., J.R.K., R.L.B., A.K.N., and C.A.G. contributed to the project design and implementation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eR.M.J., M.S., U.T.M., and A.M.M. developed the approach to qualitative analysis.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eR.M.J., M.S., and U.T.M. conducted the analysis.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eR.M.J. and M.S. wrote the main manuscript text and prepared all tables and figures.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAll authors contributed to the interpretation of data in addition to editing and reviewing the manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eAcknowledgements:\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe authors wish to thank Christina Carter and Nicole Tomimatsu for their administrative support, in addition to all of the coaches and resident coachees for their dedication to communication skills training.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003ePark YS, Zar FA, Norcini JJ, Tekian A. Competency Evaluations in the Next Accreditation System: Contributing to Guidelines and Implications. Teach Learn Med. 2016;28(2):135\u0026ndash;45. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1080/10401334.2016.1146607\u003c/span\u003e\u003cspan address=\"10.1080/10401334.2016.1146607\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBing-You R, Varaklis K, Hayes V, Trowbridge R, Kemp H, McKelvy D. The Feedback Tango: An Integrative Review and Analysis of the Content of the Teacher\u0026ndash;Learner Feedback Exchange. Acad Med. 2018;93(4):657\u0026ndash;63. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000001927\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000001927\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRassbach CE, Blankenburg R. A Novel Pediatric Residency Coaching Program: Outcomes After One Year. Acad Med. 2018;93(3):430\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000001825\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000001825\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePalamara K, Chu JT, Chang Y, et al. Who Benefits Most? A Multisite Study of Coaching and Resident Well-being. J GEN INTERN MED. 2022;37(3):539\u0026ndash;47. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11606-021-06903-5\u003c/span\u003e\u003cspan address=\"10.1007/s11606-021-06903-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSasnal M, Miller-Kuhlmann R, Merrell SB, et al. Feasibility and acceptability of virtually coaching residents on communication skills: a pilot study. BMC Med Educ. 2021;21(1):513. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12909-021-02936-w\u003c/span\u003e\u003cspan address=\"10.1186/s12909-021-02936-w\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeiorio NM, Foster KW, Santen SA. Coaching a Learner in Medical Education. Acad Med. 2021;96(12):1758. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000004168\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000004168\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRassbach CE, Bogetz AL, Orlov N, et al. The Effect of Faculty Coaching on Resident Attitudes, Confidence, and Patient-Rated Communication: A Multi-Institutional Randomized Controlled Trial. Acad Pediatr. 2019;19(2):186\u0026ndash;94. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.acap.2018.10.004\u003c/span\u003e\u003cspan address=\"10.1016/j.acap.2018.10.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWhitmore J. \u003cem\u003eCoaching for Performance: GROWing Human Potential and Purpose - The Principles and Practice of Coaching and Leadership\u003c/em\u003e. 4th ed.; 2009.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFrye AW, Hemmer PA. Program evaluation models and related theories: AMEE Guide 67. Med Teach. 2012;34(5):e288\u0026ndash;99. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3109/0142159X.2012.668637\u003c/span\u003e\u003cspan address=\"10.3109/0142159X.2012.668637\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMusick DW. A Conceptual Model for Program Evaluation in Graduate Medical Education. Acad Med. 2006;81(8):759.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVassar M, Wheeler DL, Davison M, Franklin J. Program Evaluation in Medical Education: An Overview of the Utilization-focused Approach. J Educ Eval Health Prof. 2010;7:1. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3352/jeehp.2010.7.1\u003c/span\u003e\u003cspan address=\"10.3352/jeehp.2010.7.1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMelle EV. Using a Logic Model to Assist in the Planning, Implementation, and Evaluation of Educational Programs. Acad Med. 2016;91(10):1464. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000001282\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000001282\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFrechtling JA. Logic Modeling Methods in Program Evaluation. Jossey-Bass/Wiley; 2007. p. xiii.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThomas PA, Kern DE, Hughes MT, Chen BY. \u003cem\u003eCurriculum Development for Medical Education: A Six-Step Approach\u003c/em\u003e. Johns Hopkins University Press; 2015. Accessed August 9, 2022. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://jhu.pure.elsevier.com/en/publications/curriculum-development-for-medical-education-a-six-step-approach\u003c/span\u003e\u003cspan address=\"https://jhu.pure.elsevier.com/en/publications/curriculum-development-for-medical-education-a-six-step-approach\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDamschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/1748-5908-4-50\u003c/span\u003e\u003cspan address=\"10.1186/1748-5908-4-50\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePatton MQ. Qualitative Research \u0026amp; Evaluation Methods: Integrating Theory and Practice. SAGE; 2014.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBraun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77\u0026ndash;101. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1191/1478088706qp063oa\u003c/span\u003e\u003cspan address=\"10.1191/1478088706qp063oa\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThematic Analysis: Striving to Meet the Trustworthiness Criteria - Lorelli S. Nowell, Jill M. Norris, Deborah E. White, Nancy J. Moules. 2017. Accessed April 19, 2023. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://journals.sagepub.com/doi/10.1177/1609406917733847\u003c/span\u003e\u003cspan address=\"https://journals.sagepub.com/doi/10.1177/1609406917733847\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCreswell JW. 30 Essential Skills for the Qualitative Researcher. SAGE; 2016.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGrant J. Learning needs assessment: assessing the need. BMJ. 2002;324(7330):156\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePatton MQ. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press; 2011. p. xxi.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKirkpatrick DL. \u003cem\u003eEvaluating Training Programs: The Four Levels\u003c/em\u003e. First edition. San Francisco: Berrett-Koehler ; Emeryville, CA : Publishers Group West [distributor], [1994] \u0026copy;1994; 1994. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://search.library.wisc.edu/catalog/999764622302121\u003c/span\u003e\u003cspan address=\"https://search.library.wisc.edu/catalog/999764622302121\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Graduate Medical Education (GME), Communication Coaching, Program Evaluation, Program Implementation, The Consolidated Framework of Implementation Research, Logic Model, Qualitative Interview Study","lastPublishedDoi":"10.21203/rs.3.rs-4372302/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4372302/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eCoaching programs in graduate medical education have the potential to impact trainee development across multiple core competencies but require rigorous program evaluation to ensure effectiveness. We sought to qualitatively evaluate the implementation of a multi-departmental, faculty-led communication coaching program using a logic model framework.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eStudy participants were selected from four key stakeholder groups: resident coachees, faculty coaches, medical education leaders, and programmatic sponsors. 30\u0026ndash;45 minute semi-structured interviews were conducted via Zoom, transcribed, and de-identified for the analysis. Interviews captured stakeholders' perspectives on physicians' communication training needs, stakeholders perceived and actual roles, stakeholders\u0026rsquo; involvement in the program, factors influencing the implementation process, and strategies for programmatic improvement, sustainment, and spread. The Consolidated Framework of Implementation Research (CFIR) guided the codebook development and data analysis. A combined inductive/deductive approach was used to develop a 20-item codebook, followed by a team-based thematic analysis. A strong intercoder agreement (Cohen\u0026rsquo;s kappa coefficient \u003cem\u003eκ\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.83) ensured coding consistency. The emerging themes were then mapped onto four domains of a logic model: Context, Inputs and Outputs, Outcomes, and Evaluation.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003e 35 interviews were conducted between November 2021 and April 2022 with representation from all stakeholder groups, including 10 resident coachees (who received coaching), 10 faculty coaches (who served as coaches and underwent coaching-specific faculty development), 9 medical education leaders (who designed and implemented program), and programmatic sponsors (who provided financial support). We mapped 8 emergent themes onto the critical domains of a logic model for program evaluation. For the domain of Context, themes included (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) gap in communication education and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) patient-centeredness. For the domain of Inputs/Outputs, themes included (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) investment in the program and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) perceived program value. For the domain of Outcomes, themes included (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) learning-focused outcomes and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) patient-related outcomes. For the domain of Evaluation, themes included (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) defining success and (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) challenges with evaluation.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eMapping CFIR-informed themes onto a logic model for program evaluation presents a novel strategy for integrating program implementation and evaluation, both of which are essential to effective educational programming. These findings can be used to guide future programmatic modifications to better meet the needs of key stakeholders.\u003c/p\u003e","manuscriptTitle":"Implementation and Evaluation of a Communication Coaching Program: A CFIR-Informed Qualitative Analysis Mapped onto a Logic Model","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-30 18:48:19","doi":"10.21203/rs.3.rs-4372302/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-02-21T10:02:20+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-02-21T02:05:02+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-02-11T13:19:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"187025077827103128300037852296881703049","date":"2025-02-10T12:16:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"36394375050731828655825575939315923095","date":"2025-01-22T12:12:14+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-08-18T19:26:25+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-30T10:22:16+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-05-15T15:18:24+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-15T15:16:43+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2024-05-05T15:29:40+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e90073d3-8091-40d1-b28c-99a9b77554d0","owner":[],"postedDate":"May 30th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-04-28T15:59:54+00:00","versionOfRecord":{"articleIdentity":"rs-4372302","link":"https://doi.org/10.1186/s12909-025-07188-6","journal":{"identity":"bmc-medical-education","isVorOnly":false,"title":"BMC Medical Education"},"publishedOn":"2025-04-25 15:57:10","publishedOnDateReadable":"April 25th, 2025"},"versionCreatedAt":"2024-05-30 18:48:19","video":"","vorDoi":"10.1186/s12909-025-07188-6","vorDoiUrl":"https://doi.org/10.1186/s12909-025-07188-6","workflowStages":[]},"version":"v1","identity":"rs-4372302","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4372302","identity":"rs-4372302","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}