Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years

Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years Jeffrey N. Love, David P. Way, Wendy C. Coates, Anne M. Messman, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8310627/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Growth in the number of academic medical journals over the past 25 years has increased the demand for quality peer reviewers. Shortages of qualified peer reviewers are partially attributable to a lack of formal training. We developed a formal mentored peer review (MPR) program to recruit and train peer reviewers. The objective of this paper is to evaluate the impact of this program on the sponsoring journal. Methods We compared the peer-reviews generated by mentored medical education fellows from emergency medicine fellowships throughout the United States to peer-reviews generated by our traditional peer reviewers on two key metrics: quality of the peer review and time to accept and then submit the review. Additionally, we report the number of participating education fellowships and their graduates who continued as peer reviewers once they completed their fellowships. Results Over 100 fellows were mentored by 43 fellowship directors who submitted 147 MPRs over a five-year period. We retained 21 MPR graduates and counted them as peer reviewers once they had completed their fellowships. Mentored peer reviews received significantly higher editorial ratings on review quality than traditional reviewers and independent reviews by previously participating fellowship graduates (p < .001). In addition, compared with traditional reviewers, MPRs reduced the time needed to obtain two reviewers by 6 weeks (p < .001). Discussion The MPR program contributed to improving the quality of peer reviews and increasing the size of the journal’s reviewer pool. Although the time to obtain two accepting reviewers decreased, there was no difference between MPRs and controls in submitting the accepted reviews. Peer reviews by graduates of the MPR program were found to be comparable in quality to our journal’s traditional peer reviewers. By contributing to the formal education of academic medical faculty related to peer review, such an MPR program has the potential for other journals, particularly those related to medical specialties, to meet the demand of academic medical journals for quality peer reviewers. Medical Education Scholarship Fellowship Training Program Evaluation Peer Review-Journal Mentorship Community of Practice Introduction Scholarly Publishing Figures Figure 1 BACKGROUND Peer review is critical to the integrity of the scientific process by improving the quality and curating the best possible submissions for publication. The number of journals in academic medicine has significantly increased over the years, demonstrating exponential growth in the digital age and further increasing the demand for peer reviews. [ 1 – 3 ] This demand, along with the lack of available formal training in peer review, has resulted in an insufficient number of qualified peer reviewers available to meet the needs of many journals. [ 4 – 9 ] The results are often substandard reviews and delays in editorial decision making, which frustrates authors who submit their work for publication and limits the journal’s ability to publish important studies in a timely fashion. [ 2 , 10 – 12 ] Many peer reviewers feel that they are ill-prepared for this role [ 13 – 15 ], whereas reviewer engagement has been further confounded by the perceived time commitment required for activities that are largely unrecognized and unrewarded. [ 15 – 19 ] Authors and journals have attempted to bridge this training gap by offering a range of lectures, workshops, just-in-time resources and published papers. [ 16 , 20 – 23 ] Despite these efforts, journal editors continue to struggle with limited pools of qualified reviewers. [ 10 – 12 ] To better prepare potential reviewers, there has been a call for more reliable processes to teach skills for performing quality peer reviews. [ 13 , 24 – 27 ] A number of authors have advocated for mentored peer reviews (MPRs) as an effective approach to developing the knowledge and skills required to successfully fulfill this role. [ 2 , 28 – 30 ] Preliminary reports from those who were coached or mentored through the peer review process have demonstrated mixed results. Two studies reported that participants generally enjoyed enhanced self-efficacy and a better understanding of research methodology but did not demonstrate improved peer review quality. [ 28 – 31 ] One journal-based program recently reported outcomes on the basis of a six-month curriculum that paired residents or fellows with experienced mentors. They reported via a preselected case-based study followed by mentorship that the quality of the postintervention mentee’s independent review of the same manuscript significantly improved over the mentee’s independent preintervention version. [ 32 ] While some journals are willing to provide MPR opportunities to novice reviewers, robust journal-based programs are uncommon. The need for additional administrative efforts, identifying qualified mentors, creating and validating a curriculum and establishing effective mentoring relationships tends to limit such efforts. [ 29 , 32 – 34 ] Consistent with the Council of Residency Directors in Emergency Medicine (CORD) mission to “Lead the advancement of emergency medicine education”, [ 35 ] a partnership was formed with the Western Journal of Emergency Medicine to create an annual Special Issue in Education Research & Practice (henceforth referred to as the “Special Issue”). In 2019, the editors of the Special Issue developed a curriculum for a novel, evidence based MPR program using Kern’s six-step approach and an educational experience grounded in Kolb’s Experiential Learning Cycle. [ 36 ] The objective of this study was to evaluate the impact of the MPR program over its first five years. We hypothesized that the program would improve both the quality and quantity of peer reviewers, thereby improving the editorial review process. The program evaluation included comparisons of review quality between MPR groups and our control group of traditional peer reviewers. Turnaround times of these groups were tracked from the initial review request to peer review submission. Finally, we evaluated the longer-term impact of the program through annual counts and the relative contributions of the participating MPR programs and program graduates. METHODS This prospective, observational program evaluation employed a quasi-experimental design with nonequivalent groups to establish a clear temporal relationship between the intervention and its outcomes. This evaluation was based on the 2020–2024 annual Special Issues editorial process. The George Washington University Institutional Review Board approved this study as exempt (NCR202845). Informed consent was given, and participation was voluntary. Efforts were made to ensure that the data and privacy were properly protected and managed. The MPR program engaged 35 emergency medicine (EM) education fellowships over the 5-year course of this investigation. These fellowships focused on developing expertise in medical education, pedagogy, instructional design and/or medical education research. Fellowships are either 1 or 2 years in duration, with the latter being more likely to have a scholarly orientation. Thirty-four of the 35 participating programs (97.1%) offered a 2-year fellowship, many including a master’s degree in education. The directors of these programs are highly qualified educators and researchers, whereas their fellows typically have little experience as authors or peer reviewers. [ 36 ] Three groups made up the pool of reviewers that were compared in this study: (1) our traditional reviewers (controls), (2) MPRs from participating fellowships and (3) graduates of our MPR program who subsequently provided independent reviews. Once a manuscript passed administrative review and was approved for formal review, the opportunity to peer review was assigned on a next in line basis regardless of subject matter or complexity. When MPRs were selected, the corresponding fellowship director was sent an invitation to participate by the coordinating author (JNL). If accepted, it was the fellow who was listed as the reviewer on the journal website to blind the editors of whether the review was an MPR or not. Time to Secure Two Peer Reviewers Per Manuscript The first time period tracked was the time from the initial two offers to review until two reviewers had accepted the opportunity. The Special Issue’s standard was to allow potential reviewers two weeks to respond with a reminder at one week. If no answer was received, another reviewer was selected. To compare traditional reviewers to MPRs, we measured only those instances in which two traditional or two MPRs received the initial offer. This approach was adopted as the most direct measurement of the impact of this time period on the editorial turnaround time as well as to gain a better understanding of any delays that took place. There were insufficient numbers of MPRs for this to occur randomly prior to 2023. Turnaround Time for Reviews (dup: abstract ?) The second interval was the time from when each accepting reviewer (independent or MPR) was provided access to the manuscript to the submission of the completed review. Completion of peer reviews was requested within 2 weeks of gaining access to the manuscript. Extensions were provided by decision editors as needed and counted toward measurement of this metric. This time period was tracked to determine if the collaborative interaction expected from the MPR process and subsequent submission of the reviews by the fellow would prolong this phase of the editorial process. Considering that independent reviews provided by traditional reviewers and fellowship graduates have to address this particular issue, these two groups were grouped together for this particular measurement. Review quality The quality of each submitted review was scored by one of two editors and one of eight section editors assigned to that manuscript. The final score for each review consisted of the means of the editor’s and the section editor’s scores. Scoring was guided by the Special Issue’s Holistic Editorial Scoring Rubric, which uses a 5-point scale with anchors for each point (5 = exceptional, 1 = unacceptable). Validity evidence supporting the use of this scale in the current population has previously been published. [ 37 ] Editors were blinded to each other’s scores. Overall review quality, as determined by mean scores and the percentage of reviews scoring > 4.0 (gold standard reviews) were compared between traditional, MPR and MPR graduates. For the latter group, comparisons were made to provide an understanding of this cadre’s ability to provide quality independent peer reviews following graduation from their fellowship. The process by which MPR graduates are evaluated for the potential to serve as independent reviews upon graduation has previously been described. [ 36 ] Data analysis When comparing traditional, MPR and MPR graduates scoring of review and turnaround times were analyzed via t-test and Levene’s test of equality of variances to assess whether the variances of two or more groups were equal. If a variance was found, Cohen’s D was applied as a standardized measure of the practical significance of the observed difference, with: 0-0.1 = no effect, 0.2–0.4 = small effect, 0.5–0.7 = intermediate effect, and > 0.8 = large effect. In addition, we tracked counts, frequencies, percentages and applied associated descriptive statistics related to demographics where appropriate. In addition, counts, frequencies, percentages and applied associated descriptive statistics related to demographics where appropriate. Temporal Changes in Reviewer Pool Composition The impact of the MPR program on the size of our reviewer pool was evaluated first by annually tracking the number of participating fellowship programs, MPRs and fellowship graduates providing independent peer reviews. The percentage of the total annual reviews represented by these groups was also tracked. RESULTS Time to Secure Two Peer Reviewers The first editorial time period assessed was from the time when the first offers were made until two review opportunities were accepted, when instances when the first two were MPRs versus two traditional reviewers were compared (Table 1 ). Table 1 Reviewer Type Number of Manuscripts Mean Turn-around Time Days (+/- SD) t df p es Traditional Peer-Reviews 17 60.24 (31.96) 4.95 23.825 < .001** 1.70* MPRs 18 17.22 (16.70) TOTAL 35 38.11 (33.10) Comparison between instances in which 2 traditional reviewers or 2 Mentored Peer Reviewers (in 2023 and 2024) were the first 2 offers to review made tracking the turn-around time from initial peer-review requests to that point in time when two offers were accepted. [Counts in the second column represent both the number of manuscripts tracked but also 2 accepted reviews.] **p < .001 level and Levene’s Test for Equality of Variances (es) was significant, so equal variances between groups was not assumed. (There is a significant effect for review rating between MPRs and traditional reviewers). *Cohen’s D effect sizes are interpreted as large (> .8) The difference between the means of the two groups was 43 days (p < .001). Issues that lead to delays in obtaining two acceptances are highlighted in Fig. 1 . Turnaround Time for Reviews The second time period evaluated was from the beginning when the reviewer gained access to the manuscript until the review was submitted to the journal. The mean turnaround time in days for traditional reviews (n = 305) was 9.4 days (SD +/-8.1) versus 10.4 days (SD +/-7.9) for MPRs (n = 138), a difference that was not significant (p = 0.210). Review quality Data over the 5-year period of the study (Table 2 ) revealed that the quality scores of the reviews submitted by the three groups were significantly different, with a large effect size (F = 61.24, df = 2,460, p < .001, es = 0.75). Table 2 Groups (N) Mean Review Score Std Deviation *S-N-K Post Hoc F df p es Traditional (292) 3.33 1.04 61.243 2; 460 < .001** 0.753* MPR Grads (30) 3.61 0.84 MPR (141) 4.13 0.82 Total (463) 463 1.03 Comparison of editorial scoring of reviews (1–5, 5 = exceptional) between traditional peer-reviewers, mentored peer-reviewers (MPRs) and graduates of the mentored-peer review program (MPR Grads) on their average editorial review rating. There is a significant difference at the p < .001* and the Levine’s Test of equality of Variances (es) was significant when comparing mentored peer-reviews from the other two groups. *Cohen’s D measurement of variance effect size was intermediate to large (.7-.8). The Student-Newman-Keuls (S-N-K) Post Hoc Test indicated that the traditional and program graduates are statistically similar at the p < .05 level The Student‒Newman‒Keuls (SNK) post hoc test indicated that the MPR group reviews scored significantly higher in quality than did the other two groups (traditional and MPR graduates). Furthermore, we observed no difference between the quality scores of traditional reviewers and those submitted by graduates of the MPR program. (Table 2 ) Reviewer Pool Size Forty-three fellowship directors served as mentors in the MPR program over this 5-year period. This is more than the number of participating programs due to turnover in the fellowship director position. A review of the Special Issues published since its inception in 2014 reveals that only two of these mentors (4.7%) served in the role of independent peer reviews for this journal prior to their involvement in the program. Over this study period, 102 fellows participated in this study through one or more mentored peer reviews. Table 3 provides the annual number of MPRs and MPR graduates providing independent peer reviews as well as their impact on the overall number of reviews and recognition as Gold Standard Reviewers. Table 3 Issue Year 2020 2021 2022 2023 2024 Education Fellowships 14 19 27 30 35 MPRs (% Total Reviews) 9.7% (16/165) 29.5% (38/129) 42.1% (32/83) 47.4% (27/57) 49.3% (34/69) MPRs (% of All Gold Standard Reviews) 50.0% (10/20) 35.3% (6/17) 81.0% (17/21) 82.4% (14/17) 54.5% (12/22) Graduates (Reviewers) 0 1 2 12 21 Graduates (% Total Reviews) 0.0% (0/165) 0.7% (1/129) 2.4% (2/83) 21.1% (12/57) 30.4% (21/69) Graduates (% of Gold Standards Reviewers) 0.0% (0/20) 5.0% (1/17) 4.8% (1/21) 11.8% (2/17) 18.2% (4/22) Data trends by year related to the mentored peer-review (MPR) program regarding enrolled participants and MPR graduates (Graduates) of the program serving as independent peer-reviewers in regards to participation, percentage of Gold Standard Reviews (Blinded editor’s scoring of > 4.0 on a 1–5 scale with 5 being excellent) and the percentage of total reviews annually provided to the Western Journal of Emergency Medicine Special Issue in Educational Research & Practice DISCUSSION The MPR program was initiated 5 years ago as a professional development opportunity for education fellows. Since that time, the program has had a significant effect on the Special Issue’s peer review process. First, MPRs reduced the turnaround time of peer reviews in editorial decision making. Second, they provided peer-reviews of significantly higher quality than our traditional reviewers did. Finally, by increasing the number of MPRs participating in programs and their graduates who subsequently provide independent reviews, the program has increased the size of the pool of available peer reviewers at a rate that continues to grow annually. There are multiple factors that authors consider when choosing a journal to submit their work to. Clearly, a journal’s reputation (and likely the impact factor), relevance to the topic of the manuscript and target audience are highly important. To ensure the timeliness of publication, the turnaround time from submission to a final decision and, ultimately, publication remain important determinants that editors are more likely to directly control. [ 38 – 39 ] Consequently, our finding that inviting 2 MPRs initially, as opposed to 2 traditional reviewers, reduced the time to obtain 2 acceptances-to-review by 6 weeks is potentially important. Consistent with our findings, a recently published preliminary report by a journal-based mentored peer review program demonstrated a reduction in ignored and declined peer review requests, thereby reducing turnaround times for reviews when comparing MPRs to traditional reviewers. [ 12 ] This disparity is likely the result of the different values these two groups place on this experience. Traditional reviewers have competing priorities that are often perceived as having a greater impact on their career, making it less likely that they accept an offer to review a manuscript. [ 13 – 15 , 17 – 19 ] In contrast, we reported previously that fellows value MPRs as professional development opportunities and that fellowship directors see this as an important addition to the training they provide. [ 36 ] Fellows generally have more time for a jointly performed exercise that supports their education, which may also be a contributing factor. However, once the opportunity to review was accepted, there was no difference between the two groups in the time period to submit the completed review, thereby demonstrating a commitment to the task at hand in both groups once they had committed to the review. On the basis of average review scores and the percentage of those who are recognized as “gold standard reviewers,” the quality of mentored peer reviews was significantly greater than that of our traditional reviewer pool. This finding runs contrary to the findings of a previous MPR study in which the sponsoring journal paired new reviewers with high-functioning senior reviewers. [ 33 ] The mentorship suggested by the sponsoring journal was to discuss the review with the mentees by “email or phone” followed by the mentor providing feedback to the mentee. After reviewing the literature, these authors hypothesized that the lack of improvement from this experience was likely due to an insufficiently focused and detailed mentoring process between individuals with no prior or ongoing relationships. Considering the novice status of most participating fellows in the current MPR program,[ 36 ] the quality of MPRs likely resulted from substantive mentorship in the context of a longitudinal mentoring relationship through fellowship training. It is also possible that the joint effort in creating a peer review contributed to the higher quality outcomes of the MPRs. Our MPR program has had a measurable impact on the availability and quality of peer reviews for the Special Issue. First, the number of participating education fellowship programs grew from 14–35 programs over the 5-year period since the program’s inception. Forty-one fellowship directors recognized the value of this skill development opportunity for their fellows and thus chose to participate in the Special Issue MPR program. This has resulted in a reliable cadre of high-quality reviewers for the Special Issue. Over the past 3 years, the MPRs have been responsible for nearly 50% of the peer reviews required by the Special Issue. Over this same time period, the number of manuscript submissions decreased, limiting our ability to expand opportunities to interested fellowships thereby limiting the ability to fully realize the potential value of this initiative to participants and the journal. A second way in which the MPR program was instrumental in developing independent peer reviews provided by fellowship graduates who previously participated in the program. The number of such reviewers has increased substantially beyond the third year of the program. On the basis of this growth, it appears that MPR graduates who provide independent reviews have the greatest potential for contributing to the future growth of the reviewer pool. It is also likely that they represent an untapped resource for future mentorship as they progress in their careers and assume leadership roles in medical education, such as post-graduate fellowship directors. The quality of independent peer reviews by fellowship graduates was found to be equivalent to that of traditional reviewers. Considering their background upon starting a fellowship [ 36 ], this performance demonstrates substantive growth in their skills and knowledge related to education scholarship and peer review as a result of the partnership with the editorial leadership of the Special Issue and the directors of their fellowship training. The MPR program was developed as an experience for early-career scholars who provided career development in peer review. Such experiences have the potential to be adopted by graduates and integrated into their own practices to improve scholarship in education. [ 40 ] Over the past 2 years, 6 prior fellow mentees have stepped forward to serve as mentors in the MPR program, primarily as fellowship faculty. These observations suggest that this may well serve as a scaling factor that further contributes to the pool of available highly competent peer reviewers over time. Although blinding of the editor’s score took place by having the fellow listed on the website, there was potential for the editors to identify the fellows and traditional reviewers. The medical education community in EM is such that it is not uncommon to recognize the names of other faculty members and learners from prior teaching/mentoring experiences. This raises the potential for bias on the basis of the recognition of both traditional reviewers and fellows involved in mentored peer reviews. It is also possible that the editor (JNL) who participated in invitations to fellowship directors, traditional reviewers and graduates to review could have recognized submitting reviewers. Hopefully, this risk was mitigated by the journal’s predetermined algorithm for assigning reviews. The success and thus generalizability of this journal-based program is largely dependent on integrating this experience into the curriculum of a specialty fellowship related to scholarship. This educational environment set the stage for quality mentorship and significant motivation of mentors/mentees. The absence of educational support, in this case, provided by fellowships, is a major obstacle to a successful journal-based mentored peer review program. [ 32 , 33 ] Although not measured in this study, there is no question that establishing and maintaining the MPR program requires a substantive investment in time and administrative support that may not be available to journals with limited staff or resources. [ 33 ] The scalability of the current approach to other journals will depend, in part, on the varying individual cultures, degree of access to a “fellow equivalent” pool and the available resources/commitment to the educational experience by mentors and the editorial board. CONCLUSIONS In conclusion, a commitment to the educational needs of academic-based fellowship training in education allowed this specialty-specific journal to develop a mentored peer review program that systematically increased the quality and size of the pool of peer reviewers while decreasing the turnaround time for editorial decision making. Declarations -This study received approval from the George Washington IRB as an exempt project (NCR202845). Informed consent was given, and participation was voluntary -Consent for publication: Not applicable -The datasets used and/or analyzed during this study are available from the corresponding author on reasonable request. -The authors declare that they have no competing interests. -This study was not support by funding. -Authors Contributions JNL participated in the conceptualization, data curation, formal analysis, investigation, methodology, project administration, visualization, developing the initial draft and written review and editing of the manuscript. DPW participated in the conceptualization, data curation, formal analysis, methodology, visualization, writing review and editing of the manuscript WCC participated in the conceptualization, methodology, visualization, writing review and editing of the manuscript. AMM participated in the conceptualization, data curation, investigation, writing review and editing of the manuscript. CM participated in the conceptualization, writing review and editing of the manuscript. DSA participated in the conceptualization, investigation, writing review and editing of the manuscript. JSI participated in the conceptualization, methodology, visualization, writing review and editing of the manuscript. -Acknowledgements; Not applicable References Ware M, Mabe M. The STM report: An overview of scientific and scholarly journal publishing. (2015). Kovanis M, Porcher R, Ravaud P, Trinquart L. The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PLoS ONE. 2016;11:e0166387. 10.1371/journal.pone.0166387 . Ghasemi A, Mirmiran P, Kashfi K, Bahadoran Z. Scientific publishing in biomedicine: A brief history of scientific journals. Int J Endocrinol Metab 2022;21:e131812. 10.5812/ijem-131812 Peterson CJ, Orticio C, Nugent K. The challenge of recruiting peer reviewers from one medical journal’s perspective. Proc (Bay Univ Med Cent). 2022;35:394. 10.1080/08998280.2022.2035189 . Hames I. Peer review is a rapidly evolving publishing landscape. In: Campbell R, Pentz E, Borthwick, editors. Academic and Professional Publishing. Chandos Publishing, Oxford; 2012. P. 15–52. org/10.1016/B978-1-84334-669-2.50002-0 DeLisi LE. Where have all the reviewers gone? Is the peer review concept in crisis? Psychiatric Res. 2022;310:114554. 10.1016/j.psychres.2022.114454 . Thomas SP. Current controversies regarding peer review in scholarly journals. Issue Ment Health Nurs. 2018;39:99–101. 10.1016/j.endonu.2010.02.003 . Fernandez-Llimos F. Peer review a nd publication delay. J Pharm Pract Epub 11-Nov-2019. 10.18549/PharmPract.2019.1.1502 Chloros GD, Giannoudis VP, Giannoudis PV. Peer-reviewing in surgical journals: Revolutionize or perish? Ann Surg. 2022;275:e82–90. 10.1097/SLA.0000000000004756 . Shomoossi N, Shomoossi Z, Rad M. Stressing the need for the pool of trained peer reviewer’s vs author’s suggested reviewers. J Med Ed Dev. 2023;16:78–9. Nicholas D, Watkinson A, Jamali HR, Herman E, Tenopir C, Volentine R, et al. Peer review: Still king in the digital age. Learn Publ. 2015;28:15–21. org/10.1087/20150104 . Kirk RB, Saravanan S, de Beer A, Lim JJ. The trainee as reviewer: Training the next generation of scholars. Med Educ 2025 Sep 9. 10.1111/medu.70039 Mulligan A, Hall L, Raphael E. Peer review in a changing world: An international study measuring the attitudes of researchers. J Am Soc Inf Sci Tec. 2013;64:132–61. org/10.1002/asi.22798 . Tite L, Schroter S. Why do peer reviewers decline to review? A survey. J Epidemiol Community Health. 2007;61:9–12. 10.1136/jech.2006.049817 . Snell L, Spencer J. Reviewers‘ perceptions of the peer review process for a medical education journal. Med Educ. 2005;39:90–7. org/10.1111/j.1365-2929.2004.02026.x . Watling C, Ginsburg S, Lingard L. Don’t be reviewer 2! Reflections on writing effective peer review comments. Perspect Med Educ. 2021;10:299–303. 10.1007/s40037-021-00670-z . DeMaria AN. What constitutes a great review? J Am Coll Cardiol. 2003;42:1314–5. Mayer D, Eastin C, Kane B, Lee S, Davis J, Chan TM. The importance of peer review skills and the value and necessity of training residents to ensure continued scientific excellence. AEM Educ Train 2024;May;8:S76–9. 10.1002/aet2.10940 Warne V. Rewarding reviewers-sense or sensibility? A Wiley study explained. Learn Publ. 2016;29:41–50. 10.1002/leap.1002 . Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006;99:178–82. 10.1177/014107680609900414 . Azer SA, Ramani S, Peterson R. Becoming a peer reviewer to medical education journals. Acad Med. 2012;34:698–704. 10.3109/0142159X.2012.687488 . Way DP, Bierer SB, Cianciolo AT, et al. Fundamentals of scholarly peer review: A workshop for health professions educators on practicing scholarly citizenship. MedEdPORTAL. 2021;17:11174. 10.15766/mep_2374-8265.11174 . Bruce R, Chaucin A, Trinquart L, Ravaud P, Boutron I. Impact of interventions to improve the quality of peer review of biomedical journals: A systematic review and meta-analysis. BMC Med. 2016;14:1–6. 10.1186/s12916-016-0631-5 . Ross JC, Martin A, Han SP. Finding our voice-Scholarship in Peer Review. Clin Teach. 2023;20(1). 10.1111/tct.13551 . Van Rooyen S. The evaluation of peer-review quality. Learned Publ. 2001;14:85–91. org/10.1087/095315101300059413 . Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: Randomized controlled trial. BMJ. 2004;328:1–5. .doi: org/10.1136/bmj.38023.700775.AE . Callaham ML, Wears RL, Waeckerle JF. Effect of attendance at a training session on peer reviewer quality and performance. Ann Emerg Med. 1998;32:318–22. 10.1016/s0196-0644(98)70007-1 . Wong VS, Strowd RE, Aragon-Garcia R, Moon YP, Ford B, Haut SR, Kass JS, Longdon ZN, Mays M, Milligan TA, Price RS. Mentored peer review of standardized manuscripts as a teaching tool for residents: a pilot randomized controlled multicenter study. Res Integr Peer Rev. 2017;2:1–9. 10.1186/s41073-017-0032-0 . Richards BF, Cardell EM, Chow CJ, Moore KB, Moorman KL, O’Connor M, Hart SE. Discovering the benefits of group peer review of submitted manuscripts. Teach Learn Med. 2020;32:104–9. 10.1080/10401334.2019.1657870 . Ilgen JS, Artino AR, Simpson D, Yarris LM, Chretien KC, Sullivan GM. Group peer review: The breakfast of champions. J Grad Med Educ. 2016;8:646–9. 10.4300/JGME-D-16-00531.1 . Sidalak D, Purdy E, Luckett-Gatopoulos S, et al. Coached peer review: developing the next generation of authors. Acad Med. 2016;92(2):201–4. 10.1097/ACM.0000000000001224 . Lyons-Warren AM, Aamodt WW, Pieper KM, Strowd RE. A Structured, journal-led peer-review mentoring program enhances peer review training. Res Integr Peer Rev. 2024;9:3–11. 10.1186/s41073-024-00143-x . Houry D, Green S, Callaham M. Does mentoring new peer reviewers improve review quality? A randomized controlled trial. BMC Med Educ. 2012;12:1–7. 10.1186/1472-6920-12-8 . Dumenco L, Engle DL, Goodell K, et al. Expanding group peer review: a proposal for medical education scholarship. Acad Med. 2017;92(2):147–9. 10.1097/ACM.0000000000001384 . Council of Residency Directors in Emergency Medicine. https://www.cordem.org/about-cord/mission--vision/mission--vision/ Accessed January 2nd, 2020. Love JN, Merritt C, Ilgen JS, Messman AM, Way DP, Ander DS, Coates WC. A Collaborative approach to mentored peer-reviews sponsored by the council of residency directors of emergency medicine. West J Emerg Med. 2024;25:111–6. 10.5811/westjem.61488 . Love JN, Messman AM, Ilgen JS, Coates WC, Merritt C, Ander DS, Way DP. Development and validation of a scoring rubric for editorial assessment of peer-review quality: A pilot study. West J Emerg Med. 2024;25:1–10. 10.5811/westjem.18432 . Björk BC, Solomon D. The publishing delay in scholarly peer-reviewed journals. J Informetr. 2013;7:14–23. Tenopir C, Dalton E, Fish A, Christian L, Jones M, Smith M. What motivates authors of scholarly articles? The importance of journal attributes and potential audience on publication choice. Publ. 2016;4:22. org/10.3390/publications4030022 . Eva KW. Altruism as enlightened self-interest: How helping others through peer review helps you. Med Educ. 2021;55:880–2. 10.1111/medu.14584 . Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8310627","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":571037549,"identity":"77826c7c-0e44-45e8-83bb-6f2481db580b","order_by":0,"name":"Jeffrey N. Love","email":"","orcid":"","institution":"Georgetown University","correspondingAuthor":false,"prefix":"","firstName":"Jeffrey","middleName":"N.","lastName":"Love","suffix":""},{"id":571037554,"identity":"55b70141-cc50-40e7-bc7b-9d5f4414bb78","order_by":1,"name":"David P. Way","email":"","orcid":"","institution":"The Ohio State University","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"P.","lastName":"Way","suffix":""},{"id":571037560,"identity":"44b07952-9d4c-4d85-a5bc-99d59c0ba6dd","order_by":2,"name":"Wendy C. Coates","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIiWNgGAWjYBACAwbmxgNAmrEfxEsoIEoLY8MBhgQGxpkNIC0GpGjZcADCJQzM2Q82HPj4Y5vs5vOrEz88MGCQ5xc7gF+LZU9iw8EZCbeNt914u1kC6DDDmbMTCDjsQGLDYZ6E24nbbpzdANKSYHCbkJbzDxsO/wFq2Tzj7OYfxGm5AbSFAahlA3/vNuJssZzxsOFgT9pt4xk3eLdZJBhIEPaLOX/ywQc/bG7L9vef3XzzR4WNPL80AS0IIAFWKUGschDgP0CK6lEwCkbBKBhJAABH41ITRzlo2wAAAABJRU5ErkJggg==","orcid":"","institution":"University of California, Los Angeles","correspondingAuthor":true,"prefix":"","firstName":"Wendy","middleName":"C.","lastName":"Coates","suffix":""},{"id":571037567,"identity":"67273fb3-2d4e-468c-ba10-88851383f485","order_by":3,"name":"Anne M. Messman","email":"","orcid":"","institution":"Wayne State University","correspondingAuthor":false,"prefix":"","firstName":"Anne","middleName":"M.","lastName":"Messman","suffix":""},{"id":571037575,"identity":"cf8f2a32-5994-4c1c-8e65-6e29f0c3cb56","order_by":4,"name":"Chris Merritt","email":"","orcid":"","institution":"Brown University","correspondingAuthor":false,"prefix":"","firstName":"Chris","middleName":"","lastName":"Merritt","suffix":""},{"id":571037580,"identity":"70a409e9-c875-4b21-b175-96f2e42c482c","order_by":5,"name":"Douglas S. Ander","email":"","orcid":"","institution":"Emory University","correspondingAuthor":false,"prefix":"","firstName":"Douglas","middleName":"S.","lastName":"Ander","suffix":""},{"id":571037584,"identity":"e62fb588-f051-4cab-a80d-a331d844473b","order_by":6,"name":"Jonathan S. Ilgen","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"prefix":"","firstName":"Jonathan","middleName":"S.","lastName":"Ilgen","suffix":""}],"badges":[],"createdAt":"2025-12-08 19:23:05","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8310627/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8310627/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99785598,"identity":"0d48ee8e-d932-48d5-a9c0-97a77b13b02d","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":85267,"visible":true,"origin":"","legend":"","description":"","filename":"ImpactofaMentoredPeerReviewProgramontheSponsoringJournalProgramEvaluationat5YearsBMCMedEd2025.docx","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/7de2e665712f247d5c841b31.docx"},{"id":99785601,"identity":"dedbeb0b-ded4-4401-9b74-69e027eaa48e","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9483,"visible":true,"origin":"","legend":"","description":"","filename":"2092148b564d4d769a1b9c977af578e0.json","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/09eef91549bf02f0f76b4864.json"},{"id":99785602,"identity":"86ded64b-c2f1-44c2-a044-e13a42c7cde9","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":103507,"visible":true,"origin":"","legend":"","description":"","filename":"2092148b564d4d769a1b9c977af578e01enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/875c8151575f04485b10ad30.xml"},{"id":99785604,"identity":"d2a8dd2d-3c64-486e-9237-5346dee266f4","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":43528,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/a26943ca05481dbf202b613d.png"},{"id":99798891,"identity":"b6848814-d5cf-4efd-99df-eed5631cbc77","added_by":"auto","created_at":"2026-01-08 13:49:00","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14605,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/feca88930525e107f87fe664.png"},{"id":99785600,"identity":"eb970e51-40f6-4593-a17a-2cce9016353c","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":101112,"visible":true,"origin":"","legend":"","description":"","filename":"2092148b564d4d769a1b9c977af578e01structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/456afa4025a0245002914bbf.xml"},{"id":99785605,"identity":"3df80e03-8f2b-406a-ab95-7a9af8d77b11","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"html","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":113716,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/4229bfc30a46c7a0c91c4369.html"},{"id":99785597,"identity":"7e33c117-8817-4702-a1e7-7ed3c71b98d7","added_by":"auto","created_at":"2026-01-08 11:28:38","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":43528,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of the times (days) and steps, in obtaining two acceptances when the first two offers were traditional versus two mentored peer-reviews MPRs. The examples used were that instance in each group that came closest to that groups mean time. (Traditional-57, MPR-15 days).\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/02e0b479824e11390af1d82a.png"},{"id":102529760,"identity":"57d7a8b4-9719-44ff-98b2-bc8b4540a0a5","added_by":"auto","created_at":"2026-02-12 16:11:34","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":645756,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8310627/v1/920f03cf-8b2a-4cb0-9cfd-2b689f820487.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years","fulltext":[{"header":"BACKGROUND","content":"\u003cp\u003ePeer review is critical to the integrity of the scientific process by improving the quality and curating the best possible submissions for publication. The number of journals in academic medicine has significantly increased over the years, demonstrating exponential growth in the digital age and further increasing the demand for peer reviews. [\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] This demand, along with the lack of available formal training in peer review, has resulted in an insufficient number of qualified peer reviewers available to meet the needs of many journals. [\u003cspan additionalcitationids=\"CR5 CR6 CR7 CR8\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] The results are often substandard reviews and delays in editorial decision making, which frustrates authors who submit their work for publication and limits the journal\u0026rsquo;s ability to publish important studies in a timely fashion. [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan additionalcitationids=\"CR11\" citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eMany peer reviewers feel that they are ill-prepared for this role [\u003cspan additionalcitationids=\"CR14\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], whereas reviewer engagement has been further confounded by the perceived time commitment required for activities that are largely unrecognized and unrewarded. [\u003cspan additionalcitationids=\"CR16 CR17 CR18\" citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] Authors and journals have attempted to bridge this training gap by offering a range of lectures, workshops, just-in-time resources and published papers. [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan additionalcitationids=\"CR21 CR22\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] Despite these efforts, journal editors continue to struggle with limited pools of qualified reviewers. [\u003cspan additionalcitationids=\"CR11\" citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eTo better prepare potential reviewers, there has been a call for more reliable processes to teach skills for performing quality peer reviews. [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan additionalcitationids=\"CR25 CR26\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e] A number of authors have advocated for mentored peer reviews (MPRs) as an effective approach to developing the knowledge and skills required to successfully fulfill this role. [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan additionalcitationids=\"CR29\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] Preliminary reports from those who were coached or mentored through the peer review process have demonstrated mixed results. Two studies reported that participants generally enjoyed enhanced self-efficacy and a better\u003c/p\u003e \u003cp\u003eunderstanding of research methodology but did not demonstrate improved peer review quality. [\u003cspan additionalcitationids=\"CR29 CR30\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] One journal-based program recently reported outcomes on the basis of a six-month curriculum that paired residents or fellows with experienced mentors. They reported via a preselected case-based study followed by mentorship that the quality of the postintervention mentee\u0026rsquo;s independent review of the same manuscript significantly improved over the mentee\u0026rsquo;s independent preintervention version. [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] While some journals are willing to provide MPR opportunities to novice reviewers, robust journal-based programs are uncommon. The need for additional administrative efforts, identifying qualified mentors, creating and validating a curriculum and establishing effective mentoring relationships tends to limit such efforts. [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan additionalcitationids=\"CR33\" citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eConsistent with the Council of Residency Directors in Emergency Medicine (CORD) mission to \u0026ldquo;Lead the advancement of emergency medicine education\u0026rdquo;, [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] a partnership was formed with the \u003cem\u003eWestern Journal of Emergency Medicine\u003c/em\u003e to create an annual \u003cem\u003eSpecial Issue in Education Research \u0026amp; Practice\u003c/em\u003e (henceforth referred to as the \u0026ldquo;Special Issue\u0026rdquo;). In 2019, the editors of the Special Issue developed a curriculum for a novel, evidence based MPR program using Kern\u0026rsquo;s six-step approach and an educational experience grounded in Kolb\u0026rsquo;s Experiential Learning Cycle. [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eThe objective of this study was to evaluate the impact of the MPR program over its first five years. We hypothesized that the program would improve both the quality and quantity of peer reviewers, thereby improving the editorial review process. The program evaluation included comparisons of review quality between MPR groups and our control group of traditional peer reviewers. Turnaround times of these groups were tracked from the initial review request to peer review submission. Finally, we evaluated the longer-term impact of the program through annual counts and the relative contributions of the participating MPR programs and program graduates.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003eThis prospective, observational program evaluation employed a quasi-experimental design with nonequivalent groups to establish a clear temporal relationship between the intervention and its outcomes. This evaluation was based on the 2020\u0026ndash;2024 annual Special Issues editorial process. The George Washington University Institutional Review Board approved this study as exempt (NCR202845). Informed consent was given, and participation was voluntary. Efforts were made to ensure that the data and privacy were properly protected and managed.\u003c/p\u003e \u003cp\u003eThe MPR program engaged 35 emergency medicine (EM) education fellowships over the 5-year course of this investigation. These fellowships focused on developing expertise in medical education, pedagogy, instructional design and/or medical education research. Fellowships are either 1 or 2 years in duration, with the latter being more likely to have a scholarly orientation. Thirty-four of the 35 participating programs (97.1%) offered a 2-year fellowship, many including a master\u0026rsquo;s degree in education. The directors of these programs are highly qualified educators and researchers, whereas their fellows typically have little experience as authors or peer reviewers. [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]\u003c/p\u003e \u003cp\u003eThree groups made up the pool of reviewers that were compared in this study: (1) our traditional reviewers (controls), (2) MPRs from participating fellowships and (3) graduates of our MPR program who subsequently provided independent reviews. Once a manuscript passed administrative review and was approved for formal review, the opportunity to peer review was assigned on a next in line basis regardless of subject matter or complexity. When MPRs were selected, the corresponding fellowship director was sent an invitation to participate by the coordinating author (JNL). If accepted, it was the fellow who was listed as the reviewer on the journal website to blind the editors of whether the review was an MPR or not.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eTime to Secure Two Peer Reviewers Per Manuscript\u003c/h2\u003e \u003cp\u003eThe first time period tracked was the time from the initial two offers to review until two reviewers had accepted the opportunity. The Special Issue\u0026rsquo;s standard was to allow potential reviewers two weeks to respond with a reminder at one week. If no answer was received, another reviewer was selected. To compare traditional reviewers to MPRs, we measured only those instances in which two traditional or two MPRs received the initial offer. This approach was adopted as the most direct measurement of the impact of this time period on the editorial turnaround time as well as to gain a better understanding of any delays that took place. There were insufficient numbers of MPRs for this to occur randomly prior to 2023.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eTurnaround Time for Reviews (dup: abstract ?)\u003c/h3\u003e\n\u003cp\u003eThe second interval was the time from when each accepting reviewer (independent or MPR) was provided access to the manuscript to the submission of the completed review. Completion of peer reviews was requested within 2 weeks of gaining access to the manuscript. Extensions were provided by decision editors as needed and counted toward measurement of this metric. This time period was tracked to determine if the collaborative interaction expected from the MPR process and subsequent submission of the reviews by the fellow would prolong this phase of the editorial process. Considering that independent reviews provided by traditional reviewers and fellowship graduates have to address this particular issue, these two groups were grouped together for this particular measurement.\u003c/p\u003e\n\u003ch3\u003eReview quality\u003c/h3\u003e\n\u003cp\u003eThe quality of each submitted review was scored by one of two editors and one of eight section editors assigned to that manuscript. The final score for each review consisted of the means of the editor\u0026rsquo;s and the section editor\u0026rsquo;s scores. Scoring was guided by the Special Issue\u0026rsquo;s Holistic Editorial Scoring Rubric, which uses a 5-point scale with anchors for each point (5\u0026thinsp;=\u0026thinsp;exceptional, 1\u0026thinsp;=\u0026thinsp;unacceptable). Validity evidence supporting the use of this scale in the current population has previously been published. [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] Editors were blinded to each other\u0026rsquo;s scores. Overall review quality, as determined by mean scores and the percentage of reviews scoring\u0026thinsp;\u0026gt;\u0026thinsp;4.0 (gold standard reviews) were compared between traditional, MPR and MPR graduates. For the latter group, comparisons were made to provide an understanding of this cadre\u0026rsquo;s ability to provide quality independent peer reviews following graduation from their fellowship. The process by which MPR graduates are evaluated for the potential to serve as independent reviews upon graduation has previously been described. [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]\u003c/p\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eData analysis\u003c/h2\u003e \u003cp\u003eWhen comparing traditional, MPR and MPR graduates scoring of review and turnaround times were analyzed via t-test and Levene\u0026rsquo;s test of equality of variances to assess whether the variances of two or more groups were equal. If a variance was found, Cohen\u0026rsquo;s D was applied as a standardized measure of the practical significance of the observed difference, with: 0-0.1\u0026thinsp;=\u0026thinsp;no effect, 0.2\u0026ndash;0.4\u0026thinsp;=\u0026thinsp;small effect, 0.5\u0026ndash;0.7\u0026thinsp;=\u0026thinsp;intermediate effect, and \u0026gt;\u0026thinsp;0.8\u0026thinsp;=\u0026thinsp;large effect. In addition, we tracked counts, frequencies, percentages and applied associated descriptive statistics related to demographics where appropriate. In addition, counts, frequencies, percentages and applied associated descriptive statistics related to demographics where appropriate.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eTemporal Changes in Reviewer Pool Composition\u003c/h3\u003e\n\u003cp\u003eThe impact of the MPR program on the size of our reviewer pool was evaluated first by annually tracking the number of participating fellowship programs, MPRs and fellowship graduates providing independent peer reviews. The percentage of the total annual reviews represented by these groups was also tracked.\u003c/p\u003e"},{"header":"RESULTS","content":"\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003eTime to Secure Two Peer Reviewers\u003c/h2\u003e \u003cp\u003eThe first editorial time period assessed was from the time when the first offers were made until two review opportunities were accepted, when instances when the first two were MPRs versus two traditional reviewers were compared (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e\u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eReviewer Type\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNumber of Manuscripts\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMean Turn-around Time\u003c/p\u003e \u003cp\u003eDays (+/- SD)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003et\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003edf\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003ep\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003ees\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTraditional Peer-Reviews\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e60.24 (31.96)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e4.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e23.825\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001**\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e1.70*\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMPRs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e17.22 (16.70)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTOTAL\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e38.11 (33.10)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eComparison between instances in which 2 traditional reviewers or 2 Mentored Peer Reviewers (in 2023 and 2024) were the first 2 offers to review made tracking the turn-around time from initial peer-review requests to that point in time when two offers were accepted. [Counts in the second column represent both the number of manuscripts tracked but also 2 accepted reviews.] **p\u0026thinsp;\u0026lt;\u0026thinsp;.001 level and Levene\u0026rsquo;s Test for Equality of Variances (es) was significant, so equal variances between groups was not assumed. (There is a significant effect for review rating between MPRs and traditional reviewers). *Cohen\u0026rsquo;s D effect sizes are interpreted as large (\u0026gt;\u0026thinsp;.8)\u003c/p\u003e \u003cp\u003eThe difference between the means of the two groups was 43 days (p\u0026thinsp;\u0026lt;\u0026thinsp;.001). Issues that lead to delays in obtaining two acceptances are highlighted in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eTurnaround Time for Reviews\u003c/h3\u003e\n\u003cp\u003eThe second time period evaluated was from the beginning when the reviewer gained access to the manuscript until the review was submitted to the journal. The mean turnaround time in days for traditional reviews (n\u0026thinsp;=\u0026thinsp;305) was 9.4 days (SD +/-8.1) versus 10.4 days (SD +/-7.9) for MPRs (n\u0026thinsp;=\u0026thinsp;138), a difference that was not significant (p\u0026thinsp;=\u0026thinsp;0.210).\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eReview quality\u003c/h2\u003e \u003cp\u003eData over the 5-year period of the study (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e) revealed that the quality scores of the reviews submitted by the three groups were significantly different, with a large effect size (F\u0026thinsp;=\u0026thinsp;61.24, df\u0026thinsp;=\u0026thinsp;2,460, p\u0026thinsp;\u0026lt;\u0026thinsp;.001, es\u0026thinsp;=\u0026thinsp;0.75).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e\u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"9\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGroups\u003c/p\u003e \u003cp\u003e(N)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMean\u003c/p\u003e \u003cp\u003eReview\u003c/p\u003e \u003cp\u003eScore\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eStd Deviation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e \u003cp\u003e*S-N-K Post Hoc\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eF\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003edf\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003ep\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003ees\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTraditional (292)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.04\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\" morerows=\"1\" rowspan=\"2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\" morerows=\"1\" rowspan=\"2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e61.243\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e2; 460\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;.001**\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003e0.753*\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMPR Grads\u003c/p\u003e \u003cp\u003e(30)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.61\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.84\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMPR\u003c/p\u003e \u003cp\u003e(141)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003cp\u003e(463)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e463\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.03\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"9\"\u003eComparison of editorial scoring of reviews (1\u0026ndash;5, 5\u0026thinsp;=\u0026thinsp;exceptional) between traditional peer-reviewers, mentored peer-reviewers (MPRs) and graduates of the mentored-peer review program (MPR Grads) on their average editorial review rating. There is a significant difference at the p\u0026thinsp;\u0026lt;\u0026thinsp;.001* and the Levine\u0026rsquo;s Test of equality of Variances (es) was significant when comparing mentored peer-reviews from the other two groups. *Cohen\u0026rsquo;s D measurement of variance effect size was intermediate to large (.7-.8). The Student-Newman-Keuls (S-N-K) Post Hoc Test indicated that the traditional and program graduates are statistically similar at the p\u0026thinsp;\u0026lt;\u0026thinsp;.05 level\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe Student‒Newman‒Keuls (SNK) post hoc test indicated that the MPR group reviews scored significantly higher in quality than did the other two groups (traditional and MPR graduates). Furthermore, we observed no difference between the quality scores of traditional reviewers and those submitted by graduates of the MPR program. (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e)\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eReviewer Pool Size\u003c/h2\u003e \u003cp\u003eForty-three fellowship directors served as mentors in the MPR program over this 5-year period. This is more than the number of participating programs due to turnover in the fellowship director position. A review of the Special Issues published since its inception in 2014 reveals that only two of these mentors (4.7%) served in the role of independent peer reviews for this journal prior to their involvement in the program.\u003c/p\u003e \u003cp\u003eOver this study period, 102 fellows participated in this study through one or more mentored peer reviews. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e provides the annual number of MPRs and MPR graduates providing independent peer reviews as well as their impact on the overall number of reviews and recognition as Gold Standard Reviewers.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e\u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIssue Year\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2020\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2021\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2022\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e2023\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003e2024\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEducation Fellowships\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e19\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e30\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e35\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMPRs (% Total Reviews)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e9.7% (16/165)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e29.5% (38/129)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e42.1% (32/83)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e47.4%\u003c/p\u003e \u003cp\u003e(27/57)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e49.3% (34/69)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMPRs (% of All Gold Standard Reviews)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e50.0% (10/20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e35.3% (6/17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e81.0% (17/21)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e82.4%\u003c/p\u003e \u003cp\u003e(14/17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e54.5% (12/22)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGraduates (Reviewers)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e21\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGraduates (% Total Reviews)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.0% (0/165)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.7% (1/129)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2.4% (2/83)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e21.1%\u003c/p\u003e \u003cp\u003e(12/57)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e30.4% (21/69)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGraduates (% of Gold Standards Reviewers)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.0% (0/20)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5.0%\u003c/p\u003e \u003cp\u003e(1/17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.8% (1/21)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e11.8%\u003c/p\u003e \u003cp\u003e(2/17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e18.2% (4/22)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"6\"\u003eData trends by year related to the mentored peer-review (MPR) program regarding enrolled participants and MPR graduates (Graduates) of the program serving as independent peer-reviewers in regards to participation, percentage of Gold Standard Reviews (Blinded editor\u0026rsquo;s scoring of \u0026gt;\u0026thinsp;4.0 on a 1\u0026ndash;5 scale with 5 being excellent) and the percentage of total reviews annually provided to the Western Journal of Emergency Medicine Special Issue in Educational Research \u0026amp; Practice\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eThe MPR program was initiated 5 years ago as a professional development opportunity for education fellows. Since that time, the program has had a significant effect on the Special Issue\u0026rsquo;s peer review process. First, MPRs reduced the turnaround time of peer reviews in editorial decision making. Second, they provided peer-reviews of significantly higher quality than our traditional reviewers did. Finally, by increasing the number of MPRs participating in programs and their graduates who subsequently provide independent reviews, the program has increased the size of the pool of available peer reviewers at a rate that continues to grow annually.\u003c/p\u003e \u003cp\u003eThere are multiple factors that authors consider when choosing a journal to submit their work to. Clearly, a journal\u0026rsquo;s reputation (and likely the impact factor), relevance to the topic of the manuscript and target audience are highly important. To ensure the timeliness of publication, the turnaround time from submission to a final decision and, ultimately, publication remain important determinants that editors are more likely to directly control. [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e] Consequently, our finding that inviting 2 MPRs initially, as opposed to 2 traditional reviewers, reduced the time to obtain 2 acceptances-to-review by 6 weeks is potentially important. Consistent with our findings, a recently published preliminary report by a journal-based mentored peer review program demonstrated a reduction in ignored and declined peer review requests, thereby reducing turnaround times for reviews when comparing MPRs to traditional reviewers. [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] This disparity is likely the result of the different values these two groups place on this experience. Traditional reviewers have competing priorities that are often perceived as having a greater impact on their career, making it less likely that they accept an offer to review a manuscript. [\u003cspan additionalcitationids=\"CR14\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan additionalcitationids=\"CR18\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] In contrast, we reported previously that fellows value MPRs as professional development opportunities and that fellowship directors see this as an important addition to the training they provide. [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e] Fellows generally have more time for a jointly performed exercise that supports their education, which may also be a contributing factor. However, once the opportunity to review was accepted, there was no difference between the two groups in the time period to submit the completed review, thereby demonstrating a commitment to the task at hand in both groups once they had committed to the review.\u003c/p\u003e \u003cp\u003eOn the basis of average review scores and the percentage of those who are recognized as \u0026ldquo;gold standard reviewers,\u0026rdquo; the quality of mentored peer reviews was significantly greater than that of our traditional reviewer pool. This finding runs contrary to the findings of a previous MPR study in which the sponsoring journal paired new reviewers with high-functioning senior reviewers. [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e] The mentorship suggested by the sponsoring journal was to discuss the review with the mentees by \u0026ldquo;email or phone\u0026rdquo; followed by the mentor providing feedback to the mentee. After reviewing the literature, these authors hypothesized that the lack of improvement from this experience was likely due to an insufficiently focused and detailed mentoring process between individuals with no prior or ongoing relationships. Considering the novice status of most participating fellows in the current MPR program,[\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e] the quality of MPRs likely resulted from substantive mentorship in the context of a longitudinal mentoring relationship through fellowship training. It is also possible that the joint effort in creating a peer review contributed to the higher quality outcomes of the MPRs.\u003c/p\u003e \u003cp\u003eOur MPR program has had a measurable impact on the availability and quality of peer reviews for the Special Issue. First, the number of participating education fellowship programs grew from 14\u0026ndash;35 programs over the 5-year period since the program\u0026rsquo;s inception. Forty-one fellowship directors recognized the value of this skill development opportunity for their fellows and thus chose to participate in the Special Issue MPR program. This has resulted in a reliable cadre of high-quality reviewers for the Special Issue. Over the past 3 years, the MPRs have been responsible for nearly 50% of the peer reviews required by the Special Issue. Over this same time period, the number of manuscript submissions decreased, limiting our ability to expand opportunities to interested fellowships thereby limiting the ability to fully realize the potential value of this initiative to participants and the journal. A second way in which the MPR program was instrumental in developing independent peer reviews provided by fellowship graduates who previously participated in the program. The number of such reviewers has increased substantially beyond the third year of the program. On the basis of this growth, it appears that MPR graduates who provide independent reviews have the greatest potential for contributing to the future growth of the reviewer pool. It is also likely that they represent an untapped resource for future mentorship as they progress in their careers and assume leadership roles in medical education, such as post-graduate fellowship directors.\u003c/p\u003e \u003cp\u003eThe quality of independent peer reviews by fellowship graduates was found to be equivalent to that of traditional reviewers. Considering their background upon starting a fellowship [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e], this performance demonstrates substantive growth in their skills and knowledge related to education scholarship and peer review as a result of the partnership with the editorial leadership of the Special Issue and the directors of their fellowship training.\u003c/p\u003e \u003cp\u003eThe MPR program was developed as an experience for early-career scholars who provided career development in peer review. Such experiences have the potential to be adopted by graduates and integrated into their own practices to improve scholarship in education. [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e] Over the past 2 years, 6 prior fellow mentees have stepped forward to serve as mentors in the MPR program, primarily as fellowship faculty. These observations suggest that this may well serve as a scaling factor that further contributes to the pool of available highly competent peer reviewers over time.\u003c/p\u003e \u003cp\u003eAlthough blinding of the editor\u0026rsquo;s score took place by having the fellow listed on the website, there was potential for the editors to identify the fellows and traditional reviewers. The medical education community in EM is such that it is not uncommon to recognize the names of other faculty members and learners from prior teaching/mentoring experiences. This raises the potential for bias on the basis of the recognition of both traditional reviewers and fellows involved in mentored peer reviews. It is also possible that the editor (JNL) who participated in invitations to fellowship directors, traditional reviewers and graduates to review could have recognized submitting reviewers. Hopefully, this risk was mitigated by the journal\u0026rsquo;s predetermined algorithm for assigning reviews.\u003c/p\u003e \u003cp\u003eThe success and thus generalizability of this journal-based program is largely dependent on\u003c/p\u003e \u003cp\u003eintegrating this experience into the curriculum of a specialty fellowship related to scholarship. This educational environment set the stage for quality mentorship and significant motivation of mentors/mentees. The absence of educational support, in this case, provided by fellowships, is a major obstacle to a successful journal-based mentored peer review program. [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e] Although not measured in this study, there is no question that establishing and maintaining the MPR program requires a substantive investment in time and administrative support that may not be available to journals with limited staff or resources. [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e] The scalability of the current approach to other journals will depend, in part, on the varying individual cultures, degree of access to a \u0026ldquo;fellow equivalent\u0026rdquo; pool and the available resources/commitment to the educational experience by mentors and the editorial board.\u003c/p\u003e"},{"header":"CONCLUSIONS","content":"\u003cp\u003eIn conclusion, a commitment to the educational needs of academic-based fellowship training in education allowed this specialty-specific journal to develop a mentored peer review program that systematically increased the quality and size of the pool of peer reviewers while decreasing the turnaround time for editorial decision making.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e-This study received approval from the George Washington IRB as an exempt project (NCR202845). \u0026nbsp;Informed consent was given, and participation was voluntary\u003c/p\u003e\n\u003cp\u003e-Consent for publication: \u0026nbsp;Not applicable\u003c/p\u003e\n\u003cp\u003e-The datasets used and/or analyzed during this study are available from the corresponding author on reasonable request.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e-The authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e-This study was not support by funding.\u003c/p\u003e\n\u003cp\u003e-Authors Contributions\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eJNL participated in the conceptualization, data curation, formal analysis, investigation, methodology, project administration, visualization, developing the initial draft and written review and editing of the manuscript.\u003c/li\u003e\n \u003cli\u003eDPW participated in the conceptualization, data curation, formal analysis, methodology, visualization, writing review and editing of the manuscript\u003c/li\u003e\n \u003cli\u003eWCC participated in the conceptualization, methodology, visualization, writing review and editing of the manuscript.\u003c/li\u003e\n \u003cli\u003eAMM participated in the conceptualization, data curation, investigation, writing review and editing of the manuscript.\u003c/li\u003e\n \u003cli\u003eCM participated in the conceptualization, writing review and editing of the manuscript.\u003c/li\u003e\n \u003cli\u003eDSA participated \u0026nbsp;in the conceptualization, investigation, writing review and editing of the manuscript.\u003c/li\u003e\n \u003cli\u003eJSI participated in the conceptualization, methodology, visualization, writing review and editing of the manuscript.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e-Acknowledgements; \u0026nbsp;Not applicable\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eWare M, Mabe M. The STM report: An overview of scientific and scholarly journal publishing. (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKovanis M, Porcher R, Ravaud P, Trinquart L. The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PLoS ONE. 2016;11:e0166387. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1371/journal.pone.0166387\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0166387\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGhasemi A, Mirmiran P, Kashfi K, Bahadoran Z. Scientific publishing in biomedicine: A brief history of scientific journals. Int J Endocrinol Metab 2022;21:e131812. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.5812/ijem-131812\u003c/span\u003e\u003cspan address=\"10.5812/ijem-131812\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePeterson CJ, Orticio C, Nugent K. The challenge of recruiting peer reviewers from one medical journal\u0026rsquo;s perspective. Proc (Bay Univ Med Cent). 2022;35:394. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1080/08998280.2022.2035189\u003c/span\u003e\u003cspan address=\"10.1080/08998280.2022.2035189\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHames I. Peer review is a rapidly evolving publishing landscape. In: Campbell R, Pentz E, Borthwick, editors. Academic and Professional Publishing. Chandos Publishing, Oxford; 2012. P. 15\u0026ndash;52. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.1016/B978-1-84334-669-2.50002-0\u003c/span\u003e\u003cspan address=\"10.1016/B978-1-84334-669-2.50002-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeLisi LE. Where have all the reviewers gone? Is the peer review concept in crisis? Psychiatric Res. 2022;310:114554. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.psychres.2022.114454\u003c/span\u003e\u003cspan address=\"10.1016/j.psychres.2022.114454\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThomas SP. Current controversies regarding peer review in scholarly journals. Issue Ment Health Nurs. 2018;39:99\u0026ndash;101. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.endonu.2010.02.003\u003c/span\u003e\u003cspan address=\"10.1016/j.endonu.2010.02.003\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFernandez-Llimos F. Peer review \u003cem\u003ea\u003c/em\u003end publication delay. J Pharm Pract Epub 11-Nov-2019. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.18549/PharmPract.2019.1.1502\u003c/span\u003e\u003cspan address=\"10.18549/PharmPract.2019.1.1502\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChloros GD, Giannoudis VP, Giannoudis PV. Peer-reviewing in surgical journals: Revolutionize or perish? Ann Surg. 2022;275:e82\u0026ndash;90. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/SLA.0000000000004756\u003c/span\u003e\u003cspan address=\"10.1097/SLA.0000000000004756\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShomoossi N, Shomoossi Z, Rad M. Stressing the need for the pool of trained peer reviewer\u0026rsquo;s vs author\u0026rsquo;s suggested reviewers. J Med Ed Dev. 2023;16:78\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNicholas D, Watkinson A, Jamali HR, Herman E, Tenopir C, Volentine R, et al. Peer review: Still king in the digital age. Learn Publ. 2015;28:15\u0026ndash;21. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.1087/20150104\u003c/span\u003e\u003cspan address=\"10.1087/20150104\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKirk RB, Saravanan S, de Beer A, Lim JJ. The trainee as reviewer: Training the next generation of scholars. Med Educ 2025 Sep 9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/medu.70039\u003c/span\u003e\u003cspan address=\"10.1111/medu.70039\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMulligan A, Hall L, Raphael E. Peer review in a changing world: An international study measuring the attitudes of researchers. J Am Soc Inf Sci Tec. 2013;64:132\u0026ndash;61. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.1002/asi.22798\u003c/span\u003e\u003cspan address=\"10.1002/asi.22798\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTite L, Schroter S. Why do peer reviewers decline to review? A survey. J Epidemiol Community Health. 2007;61:9\u0026ndash;12. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1136/jech.2006.049817\u003c/span\u003e\u003cspan address=\"10.1136/jech.2006.049817\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSnell L, Spencer J. Reviewers\u0026lsquo; perceptions of the peer review process for a medical education journal. Med Educ. 2005;39:90\u0026ndash;7. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.1111/j.1365-2929.2004.02026.x\u003c/span\u003e\u003cspan address=\"10.1111/j.1365-2929.2004.02026.x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWatling C, Ginsburg S, Lingard L. Don\u0026rsquo;t be reviewer 2! Reflections on writing effective peer review comments. Perspect Med Educ. 2021;10:299\u0026ndash;303. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1007/s40037-021-00670-z\u003c/span\u003e\u003cspan address=\"10.1007/s40037-021-00670-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeMaria AN. What constitutes a great review? J Am Coll Cardiol. 2003;42:1314\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMayer D, Eastin C, Kane B, Lee S, Davis J, Chan TM. The importance of peer review skills and the value and necessity of training residents to ensure continued scientific excellence. AEM Educ Train 2024;May;8:S76\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1002/aet2.10940\u003c/span\u003e\u003cspan address=\"10.1002/aet2.10940\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWarne V. Rewarding reviewers-sense or sensibility? A Wiley study explained. Learn Publ. 2016;29:41\u0026ndash;50. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1002/leap.1002\u003c/span\u003e\u003cspan address=\"10.1002/leap.1002\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSmith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006;99:178\u0026ndash;82. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/014107680609900414\u003c/span\u003e\u003cspan address=\"10.1177/014107680609900414\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAzer SA, Ramani S, Peterson R. Becoming a peer reviewer to medical education journals. Acad Med. 2012;34:698\u0026ndash;704. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3109/0142159X.2012.687488\u003c/span\u003e\u003cspan address=\"10.3109/0142159X.2012.687488\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWay DP, Bierer SB, Cianciolo AT, et al. Fundamentals of scholarly peer review: A workshop for health professions educators on practicing scholarly citizenship. MedEdPORTAL. 2021;17:11174. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.15766/mep_2374-8265.11174\u003c/span\u003e\u003cspan address=\"10.15766/mep_2374-8265.11174\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBruce R, Chaucin A, Trinquart L, Ravaud P, Boutron I. Impact of interventions to improve the quality of peer review of biomedical journals: A systematic review and meta-analysis. BMC Med. 2016;14:1\u0026ndash;6. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12916-016-0631-5\u003c/span\u003e\u003cspan address=\"10.1186/s12916-016-0631-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRoss JC, Martin A, Han SP. Finding our voice-Scholarship in Peer Review. Clin Teach. 2023;20(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/tct.13551\u003c/span\u003e\u003cspan address=\"10.1111/tct.13551\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVan Rooyen S. The evaluation of peer-review quality. Learned Publ. 2001;14:85\u0026ndash;91. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.1087/095315101300059413\u003c/span\u003e\u003cspan address=\"10.1087/095315101300059413\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: Randomized controlled trial. BMJ. 2004;328:1\u0026ndash;5. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e.doi: org/10.1136/bmj.38023.700775.AE\u003c/span\u003e\u003cspan address=\".doi: 10.1136/bmj.38023.700775.AE\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCallaham ML, Wears RL, Waeckerle JF. Effect of attendance at a training session on peer reviewer quality and performance. Ann Emerg Med. 1998;32:318\u0026ndash;22. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/s0196-0644(98)70007-1\u003c/span\u003e\u003cspan address=\"10.1016/s0196-0644(98)70007-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWong VS, Strowd RE, Aragon-Garcia R, Moon YP, Ford B, Haut SR, Kass JS, Longdon ZN, Mays M, Milligan TA, Price RS. Mentored peer review of standardized manuscripts as a teaching tool for residents: a pilot randomized controlled multicenter study. Res Integr Peer Rev. 2017;2:1\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s41073-017-0032-0\u003c/span\u003e\u003cspan address=\"10.1186/s41073-017-0032-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRichards BF, Cardell EM, Chow CJ, Moore KB, Moorman KL, O\u0026rsquo;Connor M, Hart SE. Discovering the benefits of group peer review of submitted manuscripts. Teach Learn Med. 2020;32:104\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1080/10401334.2019.1657870\u003c/span\u003e\u003cspan address=\"10.1080/10401334.2019.1657870\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIlgen JS, Artino AR, Simpson D, Yarris LM, Chretien KC, Sullivan GM. Group peer review: The breakfast of champions. J Grad Med Educ. 2016;8:646\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.4300/JGME-D-16-00531.1\u003c/span\u003e\u003cspan address=\"10.4300/JGME-D-16-00531.1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSidalak D, Purdy E, Luckett-Gatopoulos S, et al. Coached peer review: developing the next generation of authors. Acad Med. 2016;92(2):201\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000001224\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000001224\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLyons-Warren AM, Aamodt WW, Pieper KM, Strowd RE. A Structured, journal-led peer-review mentoring program enhances peer review training. Res Integr Peer Rev. 2024;9:3\u0026ndash;11. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s41073-024-00143-x\u003c/span\u003e\u003cspan address=\"10.1186/s41073-024-00143-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHoury D, Green S, Callaham M. Does mentoring new peer reviewers improve review quality? A randomized controlled trial. BMC Med Educ. 2012;12:1\u0026ndash;7. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/1472-6920-12-8\u003c/span\u003e\u003cspan address=\"10.1186/1472-6920-12-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDumenco L, Engle DL, Goodell K, et al. Expanding group peer review: a proposal for medical education scholarship. Acad Med. 2017;92(2):147\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000001384\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000001384\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCouncil of Residency Directors in Emergency Medicine. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cordem.org/about-cord/mission--vision/mission--vision/\u003c/span\u003e\u003cspan address=\"https://www.cordem.org/about-cord/mission--vision/mission--vision/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e Accessed January 2nd, 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLove JN, Merritt C, Ilgen JS, Messman AM, Way DP, Ander DS, Coates WC. A Collaborative approach to mentored peer-reviews sponsored by the council of residency directors of emergency medicine. West J Emerg Med. 2024;25:111\u0026ndash;6. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.5811/westjem.61488\u003c/span\u003e\u003cspan address=\"10.5811/westjem.61488\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLove JN, Messman AM, Ilgen JS, Coates WC, Merritt C, Ander DS, Way DP. Development and validation of a scoring rubric for editorial assessment of peer-review quality: A pilot study. West J Emerg Med. 2024;25:1\u0026ndash;10. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.5811/westjem.18432\u003c/span\u003e\u003cspan address=\"10.5811/westjem.18432\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBj\u0026ouml;rk BC, Solomon D. The publishing delay in scholarly peer-reviewed journals. J Informetr. 2013;7:14\u0026ndash;23.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTenopir C, Dalton E, Fish A, Christian L, Jones M, Smith M. What motivates authors of scholarly articles? The importance of journal attributes and potential audience on publication choice. Publ. 2016;4:22. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003eorg/10.3390/publications4030022\u003c/span\u003e\u003cspan address=\"10.3390/publications4030022\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEva KW. Altruism as enlightened self-interest: How helping others through peer review helps you. Med Educ. 2021;55:880\u0026ndash;2. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/medu.14584\u003c/span\u003e\u003cspan address=\"10.1111/medu.14584\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Medical Education Scholarship, Fellowship Training, Program Evaluation, Peer Review-Journal, Mentorship, Community of Practice Introduction, Scholarly Publishing","lastPublishedDoi":"10.21203/rs.3.rs-8310627/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8310627/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eGrowth in the number of academic medical journals over the past 25 years has increased the demand for quality peer reviewers. Shortages of qualified peer reviewers are partially attributable to a lack of formal training. We developed a formal mentored peer review (MPR) program to recruit and train peer reviewers. The objective of this paper is to evaluate the impact of this program on the sponsoring journal.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eWe compared the peer-reviews generated by mentored medical education fellows from emergency medicine fellowships throughout the United States to peer-reviews generated by our traditional peer reviewers on two key metrics: quality of the peer review and time to accept and then submit the review. Additionally, we report the number of participating education fellowships and their graduates who continued as peer reviewers once they completed their fellowships.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eOver 100 fellows were mentored by 43 fellowship directors who submitted 147 MPRs over a five-year period. We retained 21 MPR graduates and counted them as peer reviewers once they had completed their fellowships. Mentored peer reviews received significantly higher editorial ratings on review quality than traditional reviewers and independent reviews by previously participating fellowship graduates (p\u0026thinsp;\u0026lt;\u0026thinsp;.001). In addition, compared with traditional reviewers, MPRs reduced the time needed to obtain two reviewers by 6 weeks (p\u0026thinsp;\u0026lt;\u0026thinsp;.001).\u003c/p\u003e\u003ch2\u003eDiscussion\u003c/h2\u003e \u003cp\u003eThe MPR program contributed to improving the quality of peer reviews and increasing the size of the journal\u0026rsquo;s reviewer pool. Although the time to obtain two accepting reviewers decreased, there was no difference between MPRs and controls in submitting the accepted reviews. Peer reviews by graduates of the MPR program were found to be comparable in quality to our journal\u0026rsquo;s traditional peer reviewers. By contributing to the formal education of academic medical faculty related to peer review, such an MPR program has the potential for other journals, particularly those related to medical specialties, to meet the demand of academic medical journals for quality peer reviewers.\u003c/p\u003e","manuscriptTitle":"Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-08 11:28:33","doi":"10.21203/rs.3.rs-8310627/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7b9465b7-6262-4377-a83a-3cecfdfdb64b","owner":[],"postedDate":"January 8th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-02-12T16:10:45+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-08 11:28:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8310627","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8310627","identity":"rs-8310627","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}