Developing Expert Consensus for Specialist e-Consult Response: A Delphi Study to Inform Graduate Medical Education

Developing Expert Consensus for Specialist e-Consult Response: A Delphi Study to Inform Graduate Medical Education | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Developing Expert Consensus for Specialist e-Consult Response: A Delphi Study to Inform Graduate Medical Education Kylee L. Martens, Daren Anderson, Thomas G. Deloughery, David A. Garcia, and 10 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8289170/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background To meet the demands of a growing number of specialty referrals, outpatient electronic consultations (e-consults) have emerged as an access strategy for specialist clinical input, yet no framework currently exists to standardize specialist response to e-consults. With the growing implementation of e-consult platforms throughout North America, it is essential to establish criteria for efficient, high-quality correspondence between specialists and primary care providers (PCP) and develop a formal e-consult curriculum to integrate into Accreditation Council for Graduate Medical Education (ACGME) subspecialty training programs. Methods We used a systematic, consensus-building methodology (modified Delphi) to develop expert recommendations for key elements of specialist response to e-consults. Panelists were purposefully chosen to balance clinical expertise, practice location, and setting. An initial synchronous meeting was held to create a set of items identified through comprehensive literature review using MEDLINE/PubMed, followed by two rounds of anonymous and iterative voting on the importance of each. Consensus was determined a priori as ≥ 80% of panelists agreeing that an item was essential. Results An expert group consisting of 14 clinicians (2 PCPs and 12 specialty providers: 7 hematology/oncology, 1 infectious diseases, 2 endocrinology, 1 gastroenterology, 1 cardiology) from across the U.S. and Canada was selected to form the Delphi panel. After two survey rounds, 8 essential items of specialist e-consult responses were identified and achieved consensus (≥ 80%). All final items that achieved consensus aligned with established ACGME core competencies. Conclusions An expert panel of PCP and medical specialists established consensus on a set of key components of effective specialist e-consult correspondence. These items highlight unique challenges in interprofessional communication and system-based practice that are specific to e-consults and emphasize the importance of formalized medical education curricula to build competence in providing e-consult services. Delphi panel electronic consultation specialist response graduate medical education Figures Figure 1 Background The number of outpatient subspecialty referrals has drastically risen over the past two decades ( 1 , 2 ), leading to higher health care spending, increased care fragmentation, and longer wait times for specialist consultation ( 3 , 4 ). This shift in practice stems in part from time constraints placed on primary care providers (PCPs), a rise in chronic illnesses and patient complexity, increasingly complex treatment algorithms, and physicians’ desire to reduce medical error with potential liability consequences ( 5 , 6 ). This current state of “referral madness” ( 7 ) has placed strain on both patients and healthcare providers alike, incentivizing strategies to improve communication and coordination of subspecialist care. To meet the demands of a growing number of specialty referrals, outpatient electronic consultations (e-consults) have emerged as a strategy for specialist consultation. Since their advent in the early 2000s, several studies have confirmed advantages to e-consults in reducing wait times, increasing specialty access, enhancing PCP education, and reducing costs through unnecessary travel and diagnostics ( 8 – 11 ). While e-consults have generally improved communication between referring and specialty providers, negative experiences have been reported, especially pertaining to the appropriateness of the consult question and non-specific advice and follow up plan delineated by the specialist ( 12 ). Additional concerns regarding the quality of specialist response and the impact on consultant workload have also been raised ( 13 ) ( 14 ). E-consults, when compared to the traditional face-to-face encounter or video-based telemedicine, rely heavily on effective PCP and specialist communication, highlighting a critical need to define and standardize an approach to e-consult correspondence to optimize health systems- and patient-related outcomes ( 15 , 16 ). The ECONSULT mnemonic is an example of a published framework used to guide referring providers on the appropriate elements of an e-consult request with the purpose of integrating into formalized medical training ( 17 ). However, there is currently no consensus regarding the key elements of specialist response. With the growing utilization of e-consults, it is increasingly essential to develop standardized guidelines of specialist response and implement into formalized medical training to improve the quality of inter-provider communication. Accordingly, we used a systematic consensus-building methodology (modified Delphi) to develop expert recommendations for a set of key elements of specialist response to outpatient e-consults. The goal was not only to help improve the quality of specialist to PCP correspondence but to also suggest a set of criteria for future integration into Accreditation Council for Graduate Medical Education (ACGME) subspecialty training programs to improve communication and systems-based practices. Methods Panelist Recruitment An expert group consisting of 14 clinicians was selected to form the modified Delphi panel consensus. A modified Delphi method is a structured and iterative process used to achieve high-quality consensus amongst a group of experts ( 18 ). The panel of experts was purposefully chosen to balance clinical expertise, practice location, and setting. All geographic regions in the United States (U.S.) and Canada (East, Central, and West) and practice settings (academic, private, and government healthcare systems) were included. Selection criteria for panelists included recognition as an expert in e-consults based on academic and/or clinical contributions. Potential participants were identified by the study team and recruited by e-mail. Of the 23 providers contacted, 13 international experts provided written informed consent to participate through project completion (DA, TGD, DAG, AJH, EK, CL, CM, EAM, SRO, VGV, JHW, JAF, and JJS). Participation in the panel was voluntary without explicit incentive. This study was deemed exempt by the Oregon Health & Science University Institutional Review Board prior to initiation. Initial Meeting & Questionnaire A literature search of MEDLINE/PubMed was conducted, with the assistance of a medical librarian, to identify studies addressing specialist response to outpatient e-consults published through January 31, 2025. Relevant articles were reviewed by the primary author (KLM, moderator) to inform the development of an initial set of competencies. Two initial synchronous video-conference meetings were conducted to discuss and confirm the outline of the consensus process, to collect feedback on the initial set of ideas collated from literature review, and to generate additional items felt to be essential elements of specialist correspondence to e-consults. Two or more rounds are required to build consensus using Delphi methodology ( 18 ). The moderator provided each expert with the items requiring consensus through an anonymous Qualtrics survey platform. Experts were asked their level of agreement (i.e., agree, disagree, and agree with modification) and modifications were presented as comments. Results from the prior round were aggregated and presented in subsequent rounds for review and re-vote. Consensus was determined a priori as ≥ 80% of panelists agreeing that an item was essential. Once final consensus was achieved, all items were presented for final review. Participant anonymity during the voting process was maintained throughout. Data Analysis Results from the iterative voting rounds were compiled using descriptive statistics. Each element was subsequently mapped to the 6 ACGME core competencies, including Patient Care (PC), Medical Knowledge (MK), Practice-Based Learning and Improvement (PLBI), Interpersonal and Communication Skills (ICS), Professionalism (P), and Systems-Based Practice (SBP) ( 19 ). Results Fourteen e-consult experts, including 2 PCPs and 12 specialty providers, represented a range of clinical expertise (7 hematology/oncology, 1 infectious diseases, 2 endocrinology, 1 gastroenterology, 1 cardiology; Table 1). Expert participants included 13 physicians and 1 physician assistant. The majority (n = 10) of panelists practice in an academic environment, with practice locations in both rural and urban environments spanning the U.S. and Canada. Three panelists serve in leadership roles for their e-consult program, and 6 panelists have five or more e-consult peer-reviewed publications (13, 20-23). Two initial synchronous meetings were held to navigate scheduling conflicts and allow for maximum expert panelist attendance. Ideas were openly generated by panelists and specific feedback was collected after review of the initial set of 10 items proposed by the moderator (Figure 1). Based on group discussion and feedback, the moderator revised 8 items requiring consensus. There was no panelist attrition between the initial group meeting and two subsequent survey rounds. Two rounds of asynchronous, anonymous survey voting followed these initial meetings. Following round 1 of voting, 7 of 8 items achieved expert consensus (≥ 80%) and 1 item required revisions based on panelist comments. Item 5 requiring revisions initially stated: “Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral would be indicated.” Based on expert panel review, the syntax was revised to state: “Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral or recontacting the specialist would be indicated/ necessary. ” After the second round of voting, this modified item achieved unanimous expert consensus (Table 2) resulting in a revised list of 8 items. Items that were deemed essential were then mapped to ACGME core competencies (Table 2). Most of the items aligned with the core competencies of ICS and SBP and highlight the unique attributes of e-consults that differ from traditional face-to-face referrals. Specifically, items 5-8, which include providing “a clear contingency plan”, communicating “in a professional and supportive tone”, delineating “if/how the referring provider can communicate with the subspecialist” and understanding “local context… and clear role delineation” emphasize the importance of clear communication and systems-based practice unique to the practice of responding to e-consults. Discussion The 8 items derived through expert consensus represent the first standardized set of criteria for specialist response to e-consults that align directly with pre-existing ACGME core competencies. These criteria provide an evidence-based framework for specialty providers to implement in their own e-consult programs to improve interprofessional communication and systems-based practice as it pertains to health care system navigation and patient safety. With increasing utilization of e-consults to facilitate patient access to subspecialty care, it is essential that specialty training programs adapt and incorporate formalized e-consult training as part of graduate medical education. A pervasive assumption in medical training is that informal learning through the traditional “see one, do one, teach one” model is sufficient for professional skill development and competence ( 24 ). However, it is increasingly recognized that intentional and formalized training of clinicians along their career lifespan is critical in order to adapt to changes in healthcare delivery ( 25 ). Informal and non-standardized approaches to now-common consultative practices, such as e-consults, can lead to insufficient skill development ( 26 ) with negative repercussions for both specialists and referring providers. Keely and Liddy emphasize that training and individualized feedback are essential to ensure timely, high-quality responses that promote engagement among specialists ( 27 ). Similarly, specialists have identified a need for formalized training to functionally improve efficiency and quality of response ( 28 ). Inadequate specialist training can also directly result in negative PCP experiences, particularly when communicated recommendations are unclear or generic ( 13 ). Several items generated by our expert consensus panel highlight unique facets of e-consults pertaining to interprofessional communication and role delineation. While items 1, 2, and 4 share similar overlap between traditional face-to-face referrals and e-consults, items 3, 5, and 7 acknowledge specific differences. Traditional face-to-face referrals assume that the specialist will take primary lead on arranging further diagnostic and therapeutic interventions. With e-consults, the specialist often defers that responsibility to the PCP, but in return must provide explicit instructions and a clear contingency plan based on the expected results ( 15 ). E-consult responses lacking specific guidance risk suboptimal care and a higher consult burden if repeat e-consults and face-to-face referrals are placed for additional clarification ( 13 , 14 ). Lastly, item 6 highlights the importance of a professional and supportive tone in specialist communication via e-consult. Several qualitative analyses have captured PCP perceptions of e-consult responses finding that direct recommendations strengthened relationships and learning, while impersonal and brusque communication was understandably viewed as antagonistic ( 29 – 31 ). Similarly, as patient-provider transparency grows in digital medical encounters, supportive, professional, and patient-centered language is another important consideration in the completion of e-consults ( 32 ). Our expert consensus panel found that e-consults align closely with the ACGME core competency of SBP. Historically, system-based approaches have not been an explicit part of the curriculum of most graduate medical training programs ( 33 ), however recent efforts to establish a cohesive health systems science curriculum in medical education are transforming the training environment. E-consults are highly relevant to the health systems science curriculum, exemplifying key domains such as clinical informatics, care coordination, high-value care, and quality improvement ( 34 ). Our standardized consensus for specialist response to e-consults can be seamlessly integrated into the health systems science framework to provide a clear learning pathway for medical trainees as they transition to the workforce. In a broader sense, focused curricula in e-consults can foster a sense of ownership for the systems in which medical professionals work and learn, achieving a key initiative in graduate medical education ( 35 ). In addition to contributing to training and exposure in the core competencies of SBP and ICS, e-consults also offer educational opportunities in PC, MK, and PBLI for medical trainees through case-based exchange in their desired subspecialty. Several educational benefits of e-consults, when compared to face-to-face referrals, include broadening exposure to a variety of conditions, purposefully selecting consults with greater educational impact, and reducing barriers of trainee participation through asynchronous interaction. Completion of e-consults by trainees can identify knowledge gaps, prompt literature and guideline review, and facilitate discussion and learning with supervising providers ( 16 ). Item 4 of our consensus criteria highlights how e-consults can serve as an educational tool and foster stronger relationships between PCP and specialty providers ( 36 ). While formalized telemedicine training programs have been piloted in both undergraduate and graduate medical education with positive reported educational outcomes, widespread application is still lacking ( 37 , 38 ). Ongoing efforts should focus on expanding implementation of a formalized e-consult curriculum in medical subspecialty training to augment clinical exposure through practice-based learning approaches. Our study has both strengths and limitations. Although purposefully designed, the nonrandom sampling of panelists potentially introduces bias into our set of expert consensus criteria. Despite our efforts to include a wide breadth of experts representing diverse clinical domains, practice locations, and settings, we acknowledge that not all specialties that offer e-consults were represented in our Delphi panel. Furthermore, the sample size of 14 panelists limits formal analysis of differences in the items generated. Despite this, our panelists represented a broad geographic range, various practice settings, and encompassed both primary care and 5 specialty domains. Lastly, while we acknowledge that formalized training for referring providers in placing effective e-consults is essential, addressing this was beyond the scope of our current study focused on specialty response. Conclusions We describe multidisciplinary expert consensus for specialist response to e-consults. These criteria contribute to our broader goal of training the current specialty workforce in practice- and systems-based approaches to improve care coordination and provide high-value care to patients in the 21st century. Implementation of formalized training programs is the essential next step to ensure provider preparedness and competence in delivering of high-quality care through e-consults. Declarations Data availability The data supporting the findings of this study are contained within the manuscript. Conflict of Interest The authors report no conflicts of interest. Contributions KLM, JAF, and JJS designed the study. KLM conducted the literature review, served as moderator for the modified Delphi panel, and wrote the manuscript. DA, TGD, DAG, AJH, EK, CL, CM, EAM, SRO, VGV, JHW, JAF, and JJS participated in all rounds of the Delphi process, revised the manuscript, made essential contributions, and critically reviewed and approved the final manuscript. Funding KLM receives funding through the Hemostasis & Thrombosis Research Society Mentored Research Award. Ethics, Consent to Participate, and Consent to Publish : This study was deemed exempt by the Oregon Health & Science University Institutional Review Board prior to initiation. The study was conducted in accordance with the principles of the Declaration of Helsinki. Written informed consent to participate was obtained from all participants of the expert panel. References Romaire MA. Use of Primary Care and Specialty Providers: Findings from the Medical Expenditure Panel Survey. J Gen Intern Med. 2020;35(7):2003–9. 10.1007/s11606-020-05773-7 . Epub 20200406. Barnett ML, Bitton A, Souza J, Landon BE. Ann Intern Med. 2021;174(12):1658–65. 10.7326/m21-1523 . Epub 20211102. Trends in Outpatient Care for Medicare Beneficiaries and Implications for Primary Care, 2000 to 2019. Agha L, Frandsen B, Rebitzer JB. Fragmented division of labor and healthcare costs: Evidence from moves across regions. J Public Econ. 2019;169:144–59. https://doi.org/10.1016/j.jpubeco.2018.11.001 . Snow K, Galaviz K, Turbow S. Patient Outcomes Following Interhospital Care Fragmentation: A Systematic Review. J Gen Intern Med. 2020;35(5):1550–8. 10.1007/s11606-019-05366-z . Epub 20191017. Djulbegovic B, Beckstead J, Nash DB. Human judgment and health care policy. Popul Health Manag. 2014;17(3):139 – 40. Epub 20140516. 10.1089/pop.2014.0027 . PubMed PMID: 24835387. Aliu O, Sun G, Burke J, Chung KC, Davis MM. Specialist participation in healthcare delivery transformation: influence of patient self-referral. Am J Manag Care. 2014;20(1):e22–6. PubMed PMID: 24669410. Djulbegovic B, Greenberg CS. Hematology referral madness syndrome. Blood Adv. 2023;7(21):6702–4. 10.1182/bloodadvances.2023011434 . PubMed PMID: 37729619; PubMed Central PMCID: PMC10641470. Chu CD, Dohan D, Estrella MM, Shlipak MG, Tuot DS. Primary care clinician perspectives on automated nephrology e-consults for diabetic kidney disease: a pre-implementation qualitative study. BMC Prim Care. 2024;25(1):197. 10.1186/s12875-024-02454-w . Epub 20240604. Liddy C, Drosinis P, Deri Armstrong C, McKellips F, Afkham A, Keely E. What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? A costing evaluation. BMJ Open. 2016;6(6):e010920. 10.1136/bmjopen-2015-010920 . Epub 20160623. Kwok J, Olayiwola JN, Knox M, Murphy EJ, Tuot DS. Electronic consultation system demonstrates educational benefit for primary care providers. J Telemed Telecare. 2018;24(7):465–72. 10.1177/1357633x17711822 . Epub 20170614. Vimalananda VG, Orlander JD, Afable MK, Fincke BG, Solch AK, Rinne ST, et al. Electronic consultations (E-consults) and their outcomes: a systematic review. J Am Med Inf Assoc. 2020;27(3):471–9. 10.1093/jamia/ocz185 . PubMed PMID: 31621847; PubMed Central PMCID: PMC7647247. Liddy C, Moroz I, Mihan A, Nawar N, Keely E. A Systematic Review of Asynchronous, Provider-to-Provider, Electronic Consultation Services to Improve Access to Specialty Care Available Worldwide. Telemed J E Health. 2019;25(3):184–98. 10.1089/tmj.2018.0005 . Epub 20180621. Ahmed S, Kelly YP, Behera TR, Zelen MH, Kuye I, Blakey R, et al. Utility, Appropriateness, and Content of Electronic Consultations Across Medical Subspecialties. Ann Intern Med. 2020;172(10):641–7. 10.7326/m19-3852 . Epub 20200414. Dosani T, Xiang J, Wang K, Deng Y, Connell NT, Connery D, et al. Impact of Hematology Electronic Consultations on Utilization of Referrals and Patient Outcomes in an Integrated Health Care System. JCO Oncol Pract. 2022;18(4):e564–73. 10.1200/op.21.00420 . Epub 20211216. Tran C, Archibald D, Humphrey-Murto S, Liddy C, Keely E. What makes a high-quality electronic consultation (eConsult)? A nominal group study. J Telemed Telecare. 2020;26(4):239–47. 10.1177/1357633x18822885 . Epub 20190204. Keely EJ, Archibald D, Tuot DS, Lochnan H, Liddy C. Unique Educational Opportunities for PCPs and Specialists Arising From Electronic Consultation Services. Acad Med. 2017;92(1):45–51. .0000000000001472. PubMed PMID: 28030423. Lee MS, Nambudiri V. Integrating Telemedicine Into Training: Adding Value to Graduate Medical Education Through Electronic Consultations. J Grad Med Educ. 2019;11(3):251–4. 10.4300/jgme-d-18-00754.1 . PubMed PMID: 31210850; PubMed Central PMCID: PMC6570439. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research(). Med Teach. 2017;39(1):14–9. 1245856. PubMed PMID: 27841062. ACGME. Milestones guidebook for residents and fellows. 2020 [June 12, 2025]. Available from: https://www.acgme.org/globalassets/pdfs/milestones/milestonesguidebookforresidentsfellows.pdf Tuot DS, Liddy C, Vimalananda VG, Pecina J, Murphy EJ, Keely E, et al. Evaluating diverse electronic consultation programs with a common framework. BMC Health Serv Res. 2018;18(1):814. 10.1186/s12913-018-3626-4 . Epub 20181024. Keely E, Guglani S, Mitchell E, Sethuram C, Afkham A, Liddy C. Specialists accessing specialty advice: Evaluating utilization, benefits, and impact of care of an e-consultation service. J Telemed Telecare. 2025;31(5):690–6. 10.1177/1357633x231211352 . Epub 20231107. Bifulco L, Grzejszczak L, Velez I, Angelocci T, Anderson D. A qualitative investigation of uninsured patient and primary care provider perspectives on specialty care eConsults. BMC Health Serv Res. 2023;23(1):1133. 10.1186/s12913-023-10086-6 . Epub 20231020. Medford RJ, Granger M, Pickering M, Lehmann CU, Mayorga C, King H. Implementation of Outpatient Infectious Diseases E-Consults at a Safety Net Healthcare System. Open Forum Infect Dis. 2022;9(7):ofac341. 10.1093/ofid/ofac341 . Epub 20220718. Doja A, Bould MD, Clarkin C, Eady K, Sutherland S, Writer H. The hidden and informal curriculum across the continuum of training: A cross-sectional qualitative study. Med Teach. 2016;38(4):410–8. 10.3109/0142159x.2015.1073241 . Epub 20150814. Collins RT 2nd, Sanford R. The Importance of Formalized, Lifelong Physician Career Development: Making the Case for a Paradigm Shift. Acad Med. 2021;96(10):1383–8. 10.1097/acm . .0000000000004191. PubMed PMID: 34074898. van de Wiel MW, Van den Bossche P, Janssen S, Jossberger H. Exploring deliberate practice in medicine: how do physicians learn in the workplace? Adv Health Sci Educ Theory Pract. 2011;16(1):81–95. 10.1007/s10459-010-9246-3 . Epub 20100918. Keely E, Liddy C. Specialist Participation in e-Consult and e-Referral Services: Best Practices. Telemed J E Health. 2021;27(1):17–9. 10.1089/tmj.2020.0023 . Epub 20200417. Peeters K, Giroldi E, Veldhuizen L, Abdelaziz K, Muris D, Cals J. General Practitioner Use of e-Consultation to Consult Hospital Specialists: Interview Study to Obtain Physician's Perceptions About Digital Interprofessional Communication. J Med Internet Res. 2023;25:e40318. Epub 20230328. doi: 10.2196/40318. PubMed PMID: 36976626; PubMed Central PMCID: PMC10131626. Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML. Primary Care Practitioners’ Perceptions of Electronic Consult Systems: A Qualitative Analysis. JAMA Intern Med. 2018;178(6):782–9. 10.1001/jamainternmed.2018.0738 . Anderson E, Vimalananda VG, Orlander JD, Cutrona SL, Strymish JL, Bokhour BG, et al. Implications of Electronic Consultations for Clinician Communication and Relationships: A Qualitative Study. Med Care. 2021;59(9):808–15. 10.1097/mlr.0000000000001575 . PubMed PMID: 34116530; PubMed Central PMCID: PMC8360667. Tran C, Archibald D, Humphrey-Murto S, Wood TJ, Dudek N, Liddy C, et al. eConsult Specialist Quality of Response (eSQUARE): A novel tool to measure specialist correspondence via electronic consultation. J Telemed Telecare. 2022;28(4):280–90. 10.1177/1357633x21998216 . Epub 20210303. Alpert JM, Dyer KE, Lafata JE. Patient-centered communication in digital medical encounters. Patient Educ Couns. 2017;100(10):1852–8. Epub 20170502. doi: 10.1016/j.pec.2017.04.019. PubMed PMID: 28522229; PubMed Central PMCID: PMC5573682. Varkey P, Karlapudi S, Rose S, Nelson R, Warner M. A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education. Acad Med. 2009;84(3):335–9. 10.1097/ACM.0b013e31819731fb . PubMed PMID: 19240440. Gonzalo JD, Chang A, Dekhtyar M, Starr SR, Holmboe E, Wolpaw DR. Health Systems Science in Medical Education: Unifying the Components to Catalyze Transformation. Acad Med. 2020;95(9):1362–72. 10.1097. /acm.0000000000003400. PubMed PMID: 32287080. Borkan JM, Hammoud MM, Nelson E, Oyler J, Lawson L, Starr SR, et al. Health systems science education: The new post-Flexner professionalism for the 21st century. Med Teach. 2021;43(sup2):S25–31. 1924366. PubMed PMID: 34291713. Liddy C, Abu-Hijleh T, Joschko J, Archibald D, Keely E. eConsults and Learning Between Primary Care Providers and Specialists. Fam Med. 2019;51(7):567–73. .2019.407574. PubMed PMID: 31287902. Kopp AR, Rikin S, Cassese T, Berger MA, Raff AC, Gendlina I. Medical student remote eConsult participation during the COVID-19 pandemic. BMC Med Educ. 2021;21(1):120. 10.1186/s12909-021-02562-6 . Epub 20210222. Jagolino AL, Jia J, Gildersleeve K, Ankrom C, Cai C, Rahbar M, et al. A call for formal telemedicine training during stroke fellowship. Neurology. 2016;86(19):1827–33. 10.1212/wnl.0000000000002568 . Epub 20160325. Tables Table 1 . Characteristics of Delphi Panel Characteristic n (%) Self-reported gender Woman Man 5 (36%) 9 (64%) Specialty Primary care Cardiology Endocrinology Gastroenterology Hematology/oncology Infectious diseases 2 (14%) 1 (7%) 2 (14%) 1 (7%) 7 (50%) 1 (7%) Academic Rank Assistant Professor Associate Professor Professor Clinical Faculty 4 (29%) 4 (29%) 4 (29%) 2 (14%) Clinical Practice Setting Academic Private practice/community Integrated/government healthcare 10 (71%) 1 (7%) 3 (21%) Region of Practice East Central West 7 (50%) 1 (7%) 6 (43%) # of e-consult publications in past 5 years 0 1 – 4 5 – 10 > 10 3 (21%) 5 (36%) 2 (14%) 4 (29%) Table 2 . Final Criteria for E-consult Specialist Response After Group Discussion and Iterative Voting Item Description % Consensus (≥ 80%) Round* ACGME Core Competency Briefly summarize patient-specific descriptives and pertinent workup (e.g., key labs, imaging studies, procedures, etc.), specifying the time period of data reviewed and any pertinent missing data 92% 1 PC Review the differential diagnosis and suspected etiology, if pertinent 100% 1 MK Communicate specific recommendations (e.g., additional tests, monitoring, and/or treatment, including duration and administration), and explicitly state what the specialist will order or arrange if applicable 92% 1 PC, ICS Include a brief rationale for recommendations applied to the clinical scenario to improve educational value and encourage guideline adherence (e.g., cite guidelines, relevant data, etc.) 85% 1 PBLI Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral or recontacting the specialist would be indicated/necessary 77% (Round 1); 100% (Round 2) 2 SBP, ICS Communicate in a professional and supportive tone, acknowledging the referring provider’s efforts and recognizing that the patient may review this communication 92% 1 P, ICS Delineate if/how the referring provider can communicate with the specialist, especially to ask an additional question or provide clarity about a patient (e.g., in basket message, chat function, repeat e-consult, etc.) 92% 1 SBP, ICS Understand the local context, including test and treatment availability and ordering, timeliness of e-consult completion, and clear role-delineation between referring and specialty providers 92% 1 SBP *Round during which item met consensus Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 19 Jan, 2026 Reviewers agreed at journal 09 Jan, 2026 Reviewers invited by journal 07 Jan, 2026 Editor assigned by journal 05 Jan, 2026 Editor invited by journal 12 Dec, 2025 Submission checks completed at journal 11 Dec, 2025 First submitted to journal 11 Dec, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8289170","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":570949445,"identity":"d83412af-f0a3-4756-b638-c086427e0e1e","order_by":0,"name":"Kylee L. Martens","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyElEQVRIiWNgGAWjYPCCBAZ+Bh4wi7GBCOUgRQkMkg0kazE4QKwW3fazzx983JGWuPl47+HPPAw2shsOENBidibdsHHmmZzEbWfOpUnzMKQZE9ZyII2xmbetInHbjRwzZh6Gw4mEtZx/xtj8F6hl84wcY6DD/hOh5QbQFsa2nMQNEjkGQIcdIEbLM8aZvW1pxjPOnDGTnGOQbDyTsMPSGD78bEuW7W/vMf7wpsJOto+QFhhwbABTBkQqBwF7EtSOglEwCkbBSAMAyutIuhQ48iEAAAAASUVORK5CYII=","orcid":"","institution":"Oregon Health \u0026 Science University","correspondingAuthor":true,"prefix":"","firstName":"Kylee","middleName":"L.","lastName":"Martens","suffix":""},{"id":570949450,"identity":"6cf0f5f8-9e15-4f98-9ae0-c9d367a09675","order_by":1,"name":"Daren Anderson","email":"","orcid":"","institution":"Weitzman Institute","correspondingAuthor":false,"prefix":"","firstName":"Daren","middleName":"","lastName":"Anderson","suffix":""},{"id":570949455,"identity":"c5bd120e-8381-4bbf-bad3-9867ca46ea7a","order_by":2,"name":"Thomas G. Deloughery","email":"","orcid":"","institution":"Oregon Health \u0026 Science University","correspondingAuthor":false,"prefix":"","firstName":"Thomas","middleName":"G.","lastName":"Deloughery","suffix":""},{"id":570949459,"identity":"027ffec4-9c4b-48d6-bb4f-daad67c1824c","order_by":3,"name":"David A. Garcia","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"A.","lastName":"Garcia","suffix":""},{"id":570949464,"identity":"a2b548bd-65ed-459e-a911-f6c6a05b6eba","order_by":4,"name":"Andrew J. Hale","email":"","orcid":"","institution":"University of Vermont Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Andrew","middleName":"J.","lastName":"Hale","suffix":""},{"id":570949467,"identity":"28d94c31-e259-4d80-aa0e-f38e7ca6b879","order_by":5,"name":"Erin Keely","email":"","orcid":"","institution":"University of Ottawa","correspondingAuthor":false,"prefix":"","firstName":"Erin","middleName":"","lastName":"Keely","suffix":""},{"id":570949470,"identity":"7cb1854d-ca4b-431b-b87e-0f78edd850b9","order_by":6,"name":"Clare Liddy","email":"","orcid":"","institution":"University of Ottawa","correspondingAuthor":false,"prefix":"","firstName":"Clare","middleName":"","lastName":"Liddy","suffix":""},{"id":570949474,"identity":"31afbebe-5f54-4c87-964b-dc37ffb57565","order_by":7,"name":"Christian Mayorga","email":"","orcid":"","institution":"Parkland Health \u0026 Hospital System","correspondingAuthor":false,"prefix":"","firstName":"Christian","middleName":"","lastName":"Mayorga","suffix":""},{"id":570949476,"identity":"e0717104-cad1-4fb5-b737-afbe0724a1dc","order_by":8,"name":"Elizabeth A. Miller","email":"","orcid":"","institution":"Oregon Health \u0026 Science University","correspondingAuthor":false,"prefix":"","firstName":"Elizabeth","middleName":"A.","lastName":"Miller","suffix":""},{"id":570949480,"identity":"10fa8768-a865-49b4-bc8c-5a617ae47660","order_by":9,"name":"Sven R. Olson","email":"","orcid":"","institution":"Oregon Health \u0026 Science University","correspondingAuthor":false,"prefix":"","firstName":"Sven","middleName":"R.","lastName":"Olson","suffix":""},{"id":570949482,"identity":"f2e0bc01-8a2e-4ded-80ec-74d690708803","order_by":10,"name":"Varsha G. Vimalananda","email":"","orcid":"","institution":"Bedford VA Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Varsha","middleName":"G.","lastName":"Vimalananda","suffix":""},{"id":570949488,"identity":"5dd9a9cd-15b7-4ea8-bde1-e044d1cb9341","order_by":11,"name":"Jason H. Wasfy","email":"","orcid":"","institution":"Mass General Brigham, Harvard Medical School","correspondingAuthor":false,"prefix":"","firstName":"Jason","middleName":"H.","lastName":"Wasfy","suffix":""},{"id":570949489,"identity":"f3be8b7b-71af-497c-b92f-eb5840108d82","order_by":12,"name":"Jason A. Freed","email":"","orcid":"","institution":"Beth Israel Deaconess Medical Center","correspondingAuthor":false,"prefix":"","firstName":"Jason","middleName":"A.","lastName":"Freed","suffix":""},{"id":570949492,"identity":"43ee6329-d937-43dd-a069-85ea754c0bf7","order_by":13,"name":"Joseph J. Shatzel","email":"","orcid":"","institution":"Oregon Health \u0026 Science University","correspondingAuthor":false,"prefix":"","firstName":"Joseph","middleName":"J.","lastName":"Shatzel","suffix":""}],"badges":[],"createdAt":"2025-12-05 15:38:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8289170/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8289170/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":100358748,"identity":"cdf7575f-6f6d-4868-acf4-f0a5bd775555","added_by":"auto","created_at":"2026-01-16 07:21:19","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":121661,"visible":true,"origin":"","legend":"","description":"","filename":"BMCMedEdManuscript.docx","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/8c1651c06cb58ec7dc8ae171.docx"},{"id":99881927,"identity":"56c3a958-46f3-4dc1-93f9-1c0a108eb8d5","added_by":"auto","created_at":"2026-01-09 11:28:31","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14228,"visible":true,"origin":"","legend":"","description":"","filename":"87f76167c4a149c88e2bf500888aca0b.json","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/c99f3091e8a73ee29ba24a05.json"},{"id":100358857,"identity":"5e3ef719-ebc1-4c13-bb55-c2b67fbf5c2c","added_by":"auto","created_at":"2026-01-16 07:21:30","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":107798,"visible":true,"origin":"","legend":"","description":"","filename":"87f76167c4a149c88e2bf500888aca0b1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/0cf861c15eb090ec016cdda7.xml"},{"id":99881929,"identity":"c2cf5daf-06bf-4ea0-a277-994e92d064d6","added_by":"auto","created_at":"2026-01-09 11:28:31","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":11635,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/f5ecd7c0369940b9e1417f83.png"},{"id":99881933,"identity":"0238f5bb-ccb1-46df-9a23-31d8bb42f8f9","added_by":"auto","created_at":"2026-01-09 11:28:31","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":104192,"visible":true,"origin":"","legend":"","description":"","filename":"87f76167c4a149c88e2bf500888aca0b1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/d1a07a13f2f790c23b5b737d.xml"},{"id":99881931,"identity":"53befc85-dbc8-455f-9f4d-5b85f2ada2aa","added_by":"auto","created_at":"2026-01-09 11:28:31","extension":"html","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":119826,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/1dbf263abda23316545719ed.html"},{"id":100358752,"identity":"b16fb9a3-df64-4db0-b3ea-c8301d46d0a5","added_by":"auto","created_at":"2026-01-16 07:21:19","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":25200,"visible":true,"origin":"","legend":"\u003cp\u003eDelphi Panel Flowchart for Item Generation\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/32ba04ce47baeac9288d5b20.png"},{"id":100377324,"identity":"ec53fbaa-5932-401a-a2ee-a6c7ace72931","added_by":"auto","created_at":"2026-01-16 08:47:42","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":650325,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8289170/v1/2ee02ba1-8432-4e5e-aab6-d1e3d8dffc2f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Developing Expert Consensus for Specialist e-Consult Response: A Delphi Study to Inform Graduate Medical Education","fulltext":[{"header":"Background","content":"\u003cp\u003eThe number of outpatient subspecialty referrals has drastically risen over the past two decades (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e), leading to higher health care spending, increased care fragmentation, and longer wait times for specialist consultation (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). This shift in practice stems in part from time constraints placed on primary care providers (PCPs), a rise in chronic illnesses and patient complexity, increasingly complex treatment algorithms, and physicians\u0026rsquo; desire to reduce medical error with potential liability consequences (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). This current state of \u0026ldquo;referral madness\u0026rdquo; (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e) has placed strain on both patients and healthcare providers alike, incentivizing strategies to improve communication and coordination of subspecialist care.\u003c/p\u003e \u003cp\u003eTo meet the demands of a growing number of specialty referrals, outpatient electronic consultations (e-consults) have emerged as a strategy for specialist consultation. Since their advent in the early 2000s, several studies have confirmed advantages to e-consults in reducing wait times, increasing specialty access, enhancing PCP education, and reducing costs through unnecessary travel and diagnostics (\u003cspan additionalcitationids=\"CR9 CR10\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). While e-consults have generally improved communication between referring and specialty providers, negative experiences have been reported, especially pertaining to the appropriateness of the consult question and non-specific advice and follow up plan delineated by the specialist (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). Additional concerns regarding the quality of specialist response and the impact on consultant workload have also been raised (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) (\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eE-consults, when compared to the traditional face-to-face encounter or video-based telemedicine, rely heavily on effective PCP and specialist communication, highlighting a critical need to define and standardize an approach to e-consult correspondence to optimize health systems- and patient-related outcomes (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). The ECONSULT mnemonic is an example of a published framework used to guide referring providers on the appropriate elements of an e-consult request with the purpose of integrating into formalized medical training (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). However, there is currently no consensus regarding the key elements of specialist response. With the growing utilization of e-consults, it is increasingly essential to develop standardized guidelines of specialist response and implement into formalized medical training to improve the quality of inter-provider communication.\u003c/p\u003e \u003cp\u003eAccordingly, we used a systematic consensus-building methodology (modified Delphi) to develop expert recommendations for a set of key elements of specialist response to outpatient e-consults. The goal was not only to help improve the quality of specialist to PCP correspondence but to also suggest a set of criteria for future integration into Accreditation Council for Graduate Medical Education (ACGME) subspecialty training programs to improve communication and systems-based practices.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003ePanelist Recruitment\u003c/h2\u003e \u003cp\u003eAn expert group consisting of 14 clinicians was selected to form the modified Delphi panel consensus. A modified Delphi method is a structured and iterative process used to achieve high-quality consensus amongst a group of experts (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). The panel of experts was purposefully chosen to balance clinical expertise, practice location, and setting. All geographic regions in the United States (U.S.) and Canada (East, Central, and West) and practice settings (academic, private, and government healthcare systems) were included. Selection criteria for panelists included recognition as an expert in e-consults based on academic and/or clinical contributions. Potential participants were identified by the study team and recruited by e-mail. Of the 23 providers contacted, 13 international experts provided written informed consent to participate through project completion (DA, TGD, DAG, AJH, EK, CL, CM, EAM, SRO, VGV, JHW, JAF, and JJS). Participation in the panel was voluntary without explicit incentive. This study was deemed exempt by the Oregon Health \u0026amp; Science University Institutional Review Board prior to initiation.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eInitial Meeting \u0026 Questionnaire\u003c/h3\u003e\n\u003cp\u003eA literature search of MEDLINE/PubMed was conducted, with the assistance of a medical librarian, to identify studies addressing specialist response to outpatient e-consults published through January 31, 2025. Relevant articles were reviewed by the primary author (KLM, moderator) to inform the development of an initial set of competencies. Two initial synchronous video-conference meetings were conducted to discuss and confirm the outline of the consensus process, to collect feedback on the initial set of ideas collated from literature review, and to generate additional items felt to be essential elements of specialist correspondence to e-consults.\u003c/p\u003e \u003cp\u003eTwo or more rounds are required to build consensus using Delphi methodology (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). The moderator provided each expert with the items requiring consensus through an anonymous Qualtrics survey platform. Experts were asked their level of agreement (i.e., agree, disagree, and agree with modification) and modifications were presented as comments. Results from the prior round were aggregated and presented in subsequent rounds for review and re-vote. Consensus was determined a priori as \u0026ge;\u0026thinsp;80% of panelists agreeing that an item was essential. Once final consensus was achieved, all items were presented for final review. Participant anonymity during the voting process was maintained throughout.\u003c/p\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eResults from the iterative voting rounds were compiled using descriptive statistics. Each element was subsequently mapped to the 6 ACGME core competencies, including Patient Care (PC), Medical Knowledge (MK), Practice-Based Learning and Improvement (PLBI), Interpersonal and Communication Skills (ICS), Professionalism (P), and Systems-Based Practice (SBP) (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eFourteen e-consult experts, including 2 PCPs and 12 specialty providers, represented a range of clinical expertise (7 hematology/oncology, 1 infectious diseases, 2 endocrinology, 1 gastroenterology, 1 cardiology; Table 1). Expert participants included 13 physicians and 1 physician assistant. The majority (n = 10) of panelists practice in an academic environment, with practice locations in both rural and urban environments spanning the U.S. and Canada. Three panelists serve in leadership roles for their e-consult program, and 6 panelists have five or more e-consult peer-reviewed publications (13, 20-23).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTwo initial synchronous meetings were held to navigate scheduling conflicts and allow for maximum expert panelist attendance. Ideas were openly generated by panelists and specific feedback was collected after review of the initial set of 10 items proposed by the moderator (Figure 1). Based on group discussion and feedback, the moderator revised 8 items requiring consensus. There was no panelist attrition between the initial group meeting and two subsequent survey rounds.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Two rounds of asynchronous, anonymous survey voting followed these initial meetings. Following round 1 of voting, 7 of 8 items achieved expert consensus (\u0026ge; 80%) and 1 item required revisions based on panelist comments. Item 5 requiring revisions initially stated: \u0026ldquo;Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral would be indicated.\u0026rdquo; Based on expert panel review, the syntax was revised to state: \u0026ldquo;Provide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral \u003cem\u003eor recontacting the specialist\u0026nbsp;\u003c/em\u003ewould be indicated/\u003cem\u003enecessary.\u003c/em\u003e\u0026rdquo; After the second round of voting, this modified item achieved unanimous expert consensus (Table 2) resulting in a revised list of 8 items.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eItems that were deemed essential were then mapped to ACGME core competencies (Table 2). Most of the items aligned with the core competencies of ICS and SBP and highlight the unique attributes of e-consults that differ from traditional face-to-face referrals. Specifically, items 5-8, which include providing \u0026ldquo;a clear contingency plan\u0026rdquo;, communicating \u0026ldquo;in a professional and supportive tone\u0026rdquo;, delineating \u0026ldquo;if/how the referring provider can communicate with the subspecialist\u0026rdquo; and understanding \u0026ldquo;local context\u0026hellip; and clear role delineation\u0026rdquo; emphasize the importance of clear communication and systems-based practice unique to the practice of responding to e-consults.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe 8 items derived through expert consensus represent the first standardized set of criteria for specialist response to e-consults that align directly with pre-existing ACGME core competencies. These criteria provide an evidence-based framework for specialty providers to implement in their own e-consult programs to improve interprofessional communication and systems-based practice as it pertains to health care system navigation and patient safety. With increasing utilization of e-consults to facilitate patient access to subspecialty care, it is essential that specialty training programs adapt and incorporate formalized e-consult training as part of graduate medical education.\u003c/p\u003e \u003cp\u003eA pervasive assumption in medical training is that informal learning through the traditional \u0026ldquo;see one, do one, teach one\u0026rdquo; model is sufficient for professional skill development and competence (\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e). However, it is increasingly recognized that intentional and formalized training of clinicians along their career lifespan is critical in order to adapt to changes in healthcare delivery (\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e). Informal and non-standardized approaches to now-common consultative practices, such as e-consults, can lead to insufficient skill development (\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e) with negative repercussions for both specialists and referring providers. Keely and Liddy emphasize that training and individualized feedback are essential to ensure timely, high-quality responses that promote engagement among specialists (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e). Similarly, specialists have identified a need for formalized training to functionally improve efficiency and quality of response (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e). Inadequate specialist training can also directly result in negative PCP experiences, particularly when communicated recommendations are unclear or generic (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eSeveral items generated by our expert consensus panel highlight unique facets of e-consults pertaining to interprofessional communication and role delineation. While items 1, 2, and 4 share similar overlap between traditional face-to-face referrals and e-consults, items 3, 5, and 7 acknowledge specific differences. Traditional face-to-face referrals assume that the specialist will take primary lead on arranging further diagnostic and therapeutic interventions. With e-consults, the specialist often defers that responsibility to the PCP, but in return must provide explicit instructions and a clear contingency plan based on the expected results (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e). E-consult responses lacking specific guidance risk suboptimal care and a higher consult burden if repeat e-consults and face-to-face referrals are placed for additional clarification (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). Lastly, item 6 highlights the importance of a professional and supportive tone in specialist communication via e-consult. Several qualitative analyses have captured PCP perceptions of e-consult responses finding that direct recommendations strengthened relationships and learning, while impersonal and brusque communication was understandably viewed as antagonistic (\u003cspan additionalcitationids=\"CR30\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e). Similarly, as patient-provider transparency grows in digital medical encounters, supportive, professional, and patient-centered language is another important consideration in the completion of e-consults (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eOur expert consensus panel found that e-consults align closely with the ACGME core competency of SBP. Historically, system-based approaches have not been an explicit part of the curriculum of most graduate medical training programs (\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e), however recent efforts to establish a cohesive health systems science curriculum in medical education are transforming the training environment. E-consults are highly relevant to the health systems science curriculum, exemplifying key domains such as clinical informatics, care coordination, high-value care, and quality improvement (\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e). Our standardized consensus for specialist response to e-consults can be seamlessly integrated into the health systems science framework to provide a clear learning pathway for medical trainees as they transition to the workforce. In a broader sense, focused curricula in e-consults can foster a sense of ownership for the systems in which medical professionals work and learn, achieving a key initiative in graduate medical education (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eIn addition to contributing to training and exposure in the core competencies of SBP and ICS, e-consults also offer educational opportunities in PC, MK, and PBLI for medical trainees through case-based exchange in their desired subspecialty. Several educational benefits of e-consults, when compared to face-to-face referrals, include broadening exposure to a variety of conditions, purposefully selecting consults with greater educational impact, and reducing barriers of trainee participation through asynchronous interaction. Completion of e-consults by trainees can identify knowledge gaps, prompt literature and guideline review, and facilitate discussion and learning with supervising providers (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). Item 4 of our consensus criteria highlights how e-consults can serve as an educational tool and foster stronger relationships between PCP and specialty providers (\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e). While formalized telemedicine training programs have been piloted in both undergraduate and graduate medical education with positive reported educational outcomes, widespread application is still lacking (\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e). Ongoing efforts should focus on expanding implementation of a formalized e-consult curriculum in medical subspecialty training to augment clinical exposure through practice-based learning approaches.\u003c/p\u003e \u003cp\u003eOur study has both strengths and limitations. Although purposefully designed, the nonrandom sampling of panelists potentially introduces bias into our set of expert consensus criteria. Despite our efforts to include a wide breadth of experts representing diverse clinical domains, practice locations, and settings, we acknowledge that not all specialties that offer e-consults were represented in our Delphi panel. Furthermore, the sample size of 14 panelists limits formal analysis of differences in the items generated. Despite this, our panelists represented a broad geographic range, various practice settings, and encompassed both primary care and 5 specialty domains. Lastly, while we acknowledge that formalized training for referring providers in placing effective e-consults is essential, addressing this was beyond the scope of our current study focused on specialty response.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eWe describe multidisciplinary expert consensus for specialist response to e-consults. These criteria contribute to our broader goal of training the current specialty workforce in practice- and systems-based approaches to improve care coordination and provide high-value care to patients in the 21st century. Implementation of formalized training programs is the essential next step to ensure provider preparedness and competence in delivering of high-quality care through e-consults.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data supporting the findings of this study are contained within the manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of Interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors report no conflicts of interest.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eContributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eKLM, JAF, and JJS designed the study. KLM conducted the literature review, served as moderator for the modified Delphi panel, and wrote the manuscript. DA, TGD, DAG, AJH, EK, CL, CM, EAM, SRO, VGV, JHW, JAF, and JJS participated in all rounds of the Delphi process, revised the manuscript, made essential contributions, and critically reviewed and approved the final manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eKLM receives funding through the Hemostasis \u0026amp; Thrombosis Research Society Mentored Research Award.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics, Consent to Participate, and Consent to Publish\u003c/strong\u003e: This study was deemed exempt by the Oregon Health \u0026amp; Science University Institutional Review Board prior to initiation. The study was conducted in accordance with the principles of the Declaration of Helsinki. Written informed consent to participate was obtained from all participants of the expert panel.\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eRomaire MA. Use of Primary Care and Specialty Providers: Findings from the Medical Expenditure Panel Survey. J Gen Intern Med. 2020;35(7):2003\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11606-020-05773-7\u003c/span\u003e\u003cspan address=\"10.1007/s11606-020-05773-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20200406.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarnett ML, Bitton A, Souza J, Landon BE. Ann Intern Med. 2021;174(12):1658\u0026ndash;65. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.7326/m21-1523\u003c/span\u003e\u003cspan address=\"10.7326/m21-1523\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20211102. Trends in Outpatient Care for Medicare Beneficiaries and Implications for Primary Care, 2000 to 2019.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAgha L, Frandsen B, Rebitzer JB. Fragmented division of labor and healthcare costs: Evidence from moves across regions. J Public Econ. 2019;169:144\u0026ndash;59. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jpubeco.2018.11.001\u003c/span\u003e\u003cspan address=\"10.1016/j.jpubeco.2018.11.001\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSnow K, Galaviz K, Turbow S. Patient Outcomes Following Interhospital Care Fragmentation: A Systematic Review. J Gen Intern Med. 2020;35(5):1550\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s11606-019-05366-z\u003c/span\u003e\u003cspan address=\"10.1007/s11606-019-05366-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20191017.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDjulbegovic B, Beckstead J, Nash DB. Human judgment and health care policy. Popul Health Manag. 2014;17(3):139\u0026thinsp;\u0026ndash;\u0026thinsp;40. Epub 20140516. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1089/pop.2014.0027\u003c/span\u003e\u003cspan address=\"10.1089/pop.2014.0027\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 24835387.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAliu O, Sun G, Burke J, Chung KC, Davis MM. Specialist participation in healthcare delivery transformation: influence of patient self-referral. Am J Manag Care. 2014;20(1):e22\u0026ndash;6. PubMed PMID: 24669410.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDjulbegovic B, Greenberg CS. Hematology referral madness syndrome. Blood Adv. 2023;7(21):6702\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1182/bloodadvances.2023011434\u003c/span\u003e\u003cspan address=\"10.1182/bloodadvances.2023011434\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 37729619; PubMed Central PMCID: PMC10641470.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChu CD, Dohan D, Estrella MM, Shlipak MG, Tuot DS. Primary care clinician perspectives on automated nephrology e-consults for diabetic kidney disease: a pre-implementation qualitative study. BMC Prim Care. 2024;25(1):197. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12875-024-02454-w\u003c/span\u003e\u003cspan address=\"10.1186/s12875-024-02454-w\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20240604.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiddy C, Drosinis P, Deri Armstrong C, McKellips F, Afkham A, Keely E. What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? A costing evaluation. BMJ Open. 2016;6(6):e010920. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1136/bmjopen-2015-010920\u003c/span\u003e\u003cspan address=\"10.1136/bmjopen-2015-010920\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20160623.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKwok J, Olayiwola JN, Knox M, Murphy EJ, Tuot DS. Electronic consultation system demonstrates educational benefit for primary care providers. J Telemed Telecare. 2018;24(7):465\u0026ndash;72. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/1357633x17711822\u003c/span\u003e\u003cspan address=\"10.1177/1357633x17711822\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20170614.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVimalananda VG, Orlander JD, Afable MK, Fincke BG, Solch AK, Rinne ST, et al. Electronic consultations (E-consults) and their outcomes: a systematic review. J Am Med Inf Assoc. 2020;27(3):471\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/jamia/ocz185\u003c/span\u003e\u003cspan address=\"10.1093/jamia/ocz185\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 31621847; PubMed Central PMCID: PMC7647247.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiddy C, Moroz I, Mihan A, Nawar N, Keely E. A Systematic Review of Asynchronous, Provider-to-Provider, Electronic Consultation Services to Improve Access to Specialty Care Available Worldwide. Telemed J E Health. 2019;25(3):184\u0026ndash;98. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1089/tmj.2018.0005\u003c/span\u003e\u003cspan address=\"10.1089/tmj.2018.0005\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20180621.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhmed S, Kelly YP, Behera TR, Zelen MH, Kuye I, Blakey R, et al. Utility, Appropriateness, and Content of Electronic Consultations Across Medical Subspecialties. Ann Intern Med. 2020;172(10):641\u0026ndash;7. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.7326/m19-3852\u003c/span\u003e\u003cspan address=\"10.7326/m19-3852\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20200414.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDosani T, Xiang J, Wang K, Deng Y, Connell NT, Connery D, et al. Impact of Hematology Electronic Consultations on Utilization of Referrals and Patient Outcomes in an Integrated Health Care System. JCO Oncol Pract. 2022;18(4):e564\u0026ndash;73. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1200/op.21.00420\u003c/span\u003e\u003cspan address=\"10.1200/op.21.00420\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20211216.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTran C, Archibald D, Humphrey-Murto S, Liddy C, Keely E. What makes a high-quality electronic consultation (eConsult)? A nominal group study. J Telemed Telecare. 2020;26(4):239\u0026ndash;47. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/1357633x18822885\u003c/span\u003e\u003cspan address=\"10.1177/1357633x18822885\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20190204.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKeely EJ, Archibald D, Tuot DS, Lochnan H, Liddy C. Unique Educational Opportunities for PCPs and Specialists Arising From Electronic Consultation Services. Acad Med. 2017;92(1):45\u0026ndash;51. .0000000000001472. PubMed PMID: 28030423.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee MS, Nambudiri V. Integrating Telemedicine Into Training: Adding Value to Graduate Medical Education Through Electronic Consultations. J Grad Med Educ. 2019;11(3):251\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.4300/jgme-d-18-00754.1\u003c/span\u003e\u003cspan address=\"10.4300/jgme-d-18-00754.1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 31210850; PubMed Central PMCID: PMC6570439.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHumphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research(). Med Teach. 2017;39(1):14\u0026ndash;9. 1245856. PubMed PMID: 27841062.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eACGME. Milestones guidebook for residents and fellows. 2020 [June 12, 2025]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.acgme.org/globalassets/pdfs/milestones/milestonesguidebookforresidentsfellows.pdf\u003c/span\u003e\u003cspan address=\"https://www.acgme.org/globalassets/pdfs/milestones/milestonesguidebookforresidentsfellows.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTuot DS, Liddy C, Vimalananda VG, Pecina J, Murphy EJ, Keely E, et al. Evaluating diverse electronic consultation programs with a common framework. BMC Health Serv Res. 2018;18(1):814. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12913-018-3626-4\u003c/span\u003e\u003cspan address=\"10.1186/s12913-018-3626-4\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20181024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKeely E, Guglani S, Mitchell E, Sethuram C, Afkham A, Liddy C. Specialists accessing specialty advice: Evaluating utilization, benefits, and impact of care of an e-consultation service. J Telemed Telecare. 2025;31(5):690\u0026ndash;6. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/1357633x231211352\u003c/span\u003e\u003cspan address=\"10.1177/1357633x231211352\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20231107.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBifulco L, Grzejszczak L, Velez I, Angelocci T, Anderson D. A qualitative investigation of uninsured patient and primary care provider perspectives on specialty care eConsults. BMC Health Serv Res. 2023;23(1):1133. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12913-023-10086-6\u003c/span\u003e\u003cspan address=\"10.1186/s12913-023-10086-6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20231020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMedford RJ, Granger M, Pickering M, Lehmann CU, Mayorga C, King H. Implementation of Outpatient Infectious Diseases E-Consults at a Safety Net Healthcare System. Open Forum Infect Dis. 2022;9(7):ofac341. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/ofid/ofac341\u003c/span\u003e\u003cspan address=\"10.1093/ofid/ofac341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20220718.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDoja A, Bould MD, Clarkin C, Eady K, Sutherland S, Writer H. The hidden and informal curriculum across the continuum of training: A cross-sectional qualitative study. Med Teach. 2016;38(4):410\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3109/0142159x.2015.1073241\u003c/span\u003e\u003cspan address=\"10.3109/0142159x.2015.1073241\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20150814.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCollins RT 2nd, Sanford R. The Importance of Formalized, Lifelong Physician Career Development: Making the Case for a Paradigm Shift. Acad Med. 2021;96(10):1383\u0026ndash;8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/acm\u003c/span\u003e\u003cspan address=\"10.1097/acm\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. .0000000000004191. PubMed PMID: 34074898.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003evan de Wiel MW, Van den Bossche P, Janssen S, Jossberger H. Exploring deliberate practice in medicine: how do physicians learn in the workplace? Adv Health Sci Educ Theory Pract. 2011;16(1):81\u0026ndash;95. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s10459-010-9246-3\u003c/span\u003e\u003cspan address=\"10.1007/s10459-010-9246-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20100918.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKeely E, Liddy C. Specialist Participation in e-Consult and e-Referral Services: Best Practices. Telemed J E Health. 2021;27(1):17\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1089/tmj.2020.0023\u003c/span\u003e\u003cspan address=\"10.1089/tmj.2020.0023\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20200417.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePeeters K, Giroldi E, Veldhuizen L, Abdelaziz K, Muris D, Cals J. General Practitioner Use of e-Consultation to Consult Hospital Specialists: Interview Study to Obtain Physician's Perceptions About Digital Interprofessional Communication. J Med Internet Res. 2023;25:e40318. Epub 20230328. doi: 10.2196/40318. PubMed PMID: 36976626; PubMed Central PMCID: PMC10131626.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML. Primary Care Practitioners\u0026rsquo; Perceptions of Electronic Consult Systems: A Qualitative Analysis. JAMA Intern Med. 2018;178(6):782\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jamainternmed.2018.0738\u003c/span\u003e\u003cspan address=\"10.1001/jamainternmed.2018.0738\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAnderson E, Vimalananda VG, Orlander JD, Cutrona SL, Strymish JL, Bokhour BG, et al. Implications of Electronic Consultations for Clinician Communication and Relationships: A Qualitative Study. Med Care. 2021;59(9):808\u0026ndash;15. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/mlr.0000000000001575\u003c/span\u003e\u003cspan address=\"10.1097/mlr.0000000000001575\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 34116530; PubMed Central PMCID: PMC8360667.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTran C, Archibald D, Humphrey-Murto S, Wood TJ, Dudek N, Liddy C, et al. eConsult Specialist Quality of Response (eSQUARE): A novel tool to measure specialist correspondence via electronic consultation. J Telemed Telecare. 2022;28(4):280\u0026ndash;90. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/1357633x21998216\u003c/span\u003e\u003cspan address=\"10.1177/1357633x21998216\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20210303.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlpert JM, Dyer KE, Lafata JE. Patient-centered communication in digital medical encounters. Patient Educ Couns. 2017;100(10):1852\u0026ndash;8. Epub 20170502. doi: 10.1016/j.pec.2017.04.019. PubMed PMID: 28522229; PubMed Central PMCID: PMC5573682.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVarkey P, Karlapudi S, Rose S, Nelson R, Warner M. A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education. Acad Med. 2009;84(3):335\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0b013e31819731fb\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0b013e31819731fb\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PubMed PMID: 19240440.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGonzalo JD, Chang A, Dekhtyar M, Starr SR, Holmboe E, Wolpaw DR. Health Systems Science in Medical Education: Unifying the Components to Catalyze Transformation. Acad Med. 2020;95(9):1362\u0026ndash;72. 10.1097. /acm.0000000000003400. PubMed PMID: 32287080.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBorkan JM, Hammoud MM, Nelson E, Oyler J, Lawson L, Starr SR, et al. Health systems science education: The new post-Flexner professionalism for the 21st century. Med Teach. 2021;43(sup2):S25\u0026ndash;31. 1924366. PubMed PMID: 34291713.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiddy C, Abu-Hijleh T, Joschko J, Archibald D, Keely E. eConsults and Learning Between Primary Care Providers and Specialists. Fam Med. 2019;51(7):567\u0026ndash;73. .2019.407574. PubMed PMID: 31287902.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKopp AR, Rikin S, Cassese T, Berger MA, Raff AC, Gendlina I. Medical student remote eConsult participation during the COVID-19 pandemic. BMC Med Educ. 2021;21(1):120. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12909-021-02562-6\u003c/span\u003e\u003cspan address=\"10.1186/s12909-021-02562-6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20210222.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJagolino AL, Jia J, Gildersleeve K, Ankrom C, Cai C, Rahbar M, et al. A call for formal telemedicine training during stroke fellowship. Neurology. 2016;86(19):1827\u0026ndash;33. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1212/wnl.0000000000002568\u003c/span\u003e\u003cspan address=\"10.1212/wnl.0000000000002568\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 20160325.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003e\u003cstrong\u003eTable 1\u003c/strong\u003e. Characteristics of Delphi Panel\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eCharacteristic\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003en (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSelf-reported gender\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eWoman\u003c/p\u003e\n \u003cp\u003eMan\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e5 (36%)\u003c/p\u003e\n \u003cp\u003e9 (64%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSpecialty\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003ePrimary care\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eCardiology\u003c/p\u003e\n \u003cp\u003eEndocrinology\u003c/p\u003e\n \u003cp\u003eGastroenterology\u003c/p\u003e\n \u003cp\u003eHematology/oncology\u003c/p\u003e\n \u003cp\u003eInfectious diseases\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e2 (14%)\u003c/p\u003e\n \u003cp\u003e1 (7%)\u003c/p\u003e\n \u003cp\u003e2 (14%)\u003c/p\u003e\n \u003cp\u003e1 (7%)\u003c/p\u003e\n \u003cp\u003e7 (50%)\u003c/p\u003e\n \u003cp\u003e1 (7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAcademic Rank\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAssistant Professor\u003c/p\u003e\n \u003cp\u003eAssociate Professor\u003c/p\u003e\n \u003cp\u003eProfessor\u003c/p\u003e\n \u003cp\u003eClinical Faculty\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e4 (29%)\u003c/p\u003e\n \u003cp\u003e4 (29%)\u003c/p\u003e\n \u003cp\u003e4 (29%)\u003c/p\u003e\n \u003cp\u003e2 (14%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eClinical Practice Setting\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eAcademic\u003c/p\u003e\n \u003cp\u003ePrivate practice/community\u003c/p\u003e\n \u003cp\u003eIntegrated/government healthcare\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e10 (71%)\u003c/p\u003e\n \u003cp\u003e1 (7%)\u003c/p\u003e\n \u003cp\u003e3 (21%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eRegion of Practice\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003eEast\u003c/p\u003e\n \u003cp\u003eCentral\u003c/p\u003e\n \u003cp\u003eWest\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e7 (50%)\u003c/p\u003e\n \u003cp\u003e1 (7%)\u003c/p\u003e\n \u003cp\u003e6 (43%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 318px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e# of e-consult publications in past 5 years\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003cp\u003e1 \u0026ndash; 4\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e5 \u0026ndash; 10\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026gt; 10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 98px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e3 (21%)\u003c/p\u003e\n \u003cp\u003e5 (36%)\u003c/p\u003e\n \u003cp\u003e2 (14%)\u003c/p\u003e\n \u003cp\u003e4 (29%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2\u003c/strong\u003e. Final Criteria for E-consult Specialist Response After Group Discussion and Iterative Voting\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eItem Description\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e% Consensus\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(\u0026ge; 80%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eRound*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eACGME Core Competency\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col\u003e\n \u003cli\u003eBriefly summarize patient-specific descriptives and pertinent workup (e.g., key labs, imaging studies, procedures, etc.), specifying the time period of data reviewed and any pertinent missing data\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e92%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003ePC\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"2\"\u003e\n \u003cli\u003eReview the differential diagnosis and suspected etiology, if pertinent\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e100%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003eMK\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"3\"\u003e\n \u003cli\u003eCommunicate specific recommendations (e.g., additional tests, monitoring, and/or treatment, including duration and administration), and explicitly state what the specialist will order or arrange if applicable\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e92%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003ePC, ICS\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"4\"\u003e\n \u003cli\u003eInclude a brief rationale for recommendations applied to the clinical scenario to improve educational value and encourage guideline adherence (e.g., cite guidelines, relevant data, etc.)\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e85%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003ePBLI\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"5\"\u003e\n \u003cli\u003eProvide a clear contingency plan based on expected results (e.g., if results are positive/negative, proceed with treatment X) and document when a face-to-face referral or recontacting the specialist would be indicated/necessary\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e77% (Round 1); 100% (Round 2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003eSBP, ICS\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"6\"\u003e\n \u003cli\u003eCommunicate in a professional and supportive tone, acknowledging the referring provider\u0026rsquo;s efforts and recognizing that the patient may review this communication\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e92%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003eP, ICS\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"7\"\u003e\n \u003cli\u003eDelineate if/how the referring provider can communicate with the specialist, especially to ask an additional question or provide clarity about a patient (e.g., in basket message, chat function, repeat e-consult, etc.)\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e92%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003eSBP, ICS\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 330px;\"\u003e\n \u003col start=\"8\"\u003e\n \u003cli\u003eUnderstand the local context, including test and treatment availability and ordering, timeliness of e-consult completion, and clear role-delineation between referring and specialty providers\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 114px;\"\u003e\n \u003cp\u003e92%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 72px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 108px;\"\u003e\n \u003cp\u003eSBP\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*Round during which item met consensus\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Delphi panel, electronic consultation, specialist response, graduate medical education","lastPublishedDoi":"10.21203/rs.3.rs-8289170/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8289170/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eTo meet the demands of a growing number of specialty referrals, outpatient electronic consultations (e-consults) have emerged as an access strategy for specialist clinical input, yet no framework currently exists to standardize specialist response to e-consults. With the growing implementation of e-consult platforms throughout North America, it is essential to establish criteria for efficient, high-quality correspondence between specialists and primary care providers (PCP) and develop a formal e-consult curriculum to integrate into Accreditation Council for Graduate Medical Education (ACGME) subspecialty training programs.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eWe used a systematic, consensus-building methodology (modified Delphi) to develop expert recommendations for key elements of specialist response to e-consults. Panelists were purposefully chosen to balance clinical expertise, practice location, and setting. An initial synchronous meeting was held to create a set of items identified through comprehensive literature review using MEDLINE/PubMed, followed by two rounds of anonymous and iterative voting on the importance of each. Consensus was determined \u003cem\u003ea priori\u003c/em\u003e as \u0026ge;\u0026thinsp;80% of panelists agreeing that an item was essential.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eAn expert group consisting of 14 clinicians (2 PCPs and 12 specialty providers: 7 hematology/oncology, 1 infectious diseases, 2 endocrinology, 1 gastroenterology, 1 cardiology) from across the U.S. and Canada was selected to form the Delphi panel. After two survey rounds, 8 essential items of specialist e-consult responses were identified and achieved consensus (\u0026ge;\u0026thinsp;80%). All final items that achieved consensus aligned with established ACGME core competencies.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eAn expert panel of PCP and medical specialists established consensus on a set of key components of effective specialist e-consult correspondence. These items highlight unique challenges in interprofessional communication and system-based practice that are specific to e-consults and emphasize the importance of formalized medical education curricula to build competence in providing e-consult services.\u003c/p\u003e","manuscriptTitle":"Developing Expert Consensus for Specialist e-Consult Response: A Delphi Study to Inform Graduate Medical Education","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-09 11:28:26","doi":"10.21203/rs.3.rs-8289170/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-01-19T20:24:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"1078545981808382168142957224604358024","date":"2026-01-09T13:59:08+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-01-07T12:08:14+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-05T09:25:16+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-12-12T07:09:03+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-12-11T23:50:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2025-12-11T23:46:46+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2f3ca996-3a57-4068-9421-63c9275b7b3c","owner":[],"postedDate":"January 9th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-01-09T11:28:26+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-09 11:28:26","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8289170","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8289170","identity":"rs-8289170","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}