Outreach Simulation for System Improvement: a Novel Advocacy and Reporting Process

Outreach Simulation for System Improvement: a Novel Advocacy and Reporting Process | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF advancing-simulation-practice Outreach Simulation for System Improvement: a Novel Advocacy and Reporting Process A. St-Onge-St-Hilaire, B. Lawton, L. Dodson, J. Acworth, D. Hufton, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6230704/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 01 Sep, 2025 Read the published version in Advances in Simulation → Version 1 posted 9 You are reading this latest preprint version Abstract Intro : Healthcare simulation programs measuring their value risk wasting resources in futile attempts to prove they impact patient outcomes. Simulation is one of many strategies used to enhance healthcare systems, and proving specific correlation with simulation will prove impossible in many circumstances. To maintain accountability but ensure feasibility, we argue simulation services need measurement processes that are robust, achievable and synergistic with their mission. In 2023, the STORK service in Queensland, Australia began measuring the impact of simulation on systems rather than patients , to define the extent to which their educational programs could impact system improvement. Methods : Translational simulation methodologies and quality improvement measures were embedded in an established educational course. We used simulation activities to diagnose environmental and system-level problems in participants’ workplaces throughout Queensland. Courses included dedicated time to discuss site-specific actionable solutions with participants, and identified local champions to implement quality improvement changes. By designing a novel electronic reporting process (Optimus PRIME Course Summary), we documented issues and solutions identified in regional healthcare facilities and ensured they reached key stakeholders. We audited our ability to improve these systems through follow up data collection via phone and emails with local educators across the state. Results : From 40 courses delivered across 37 facilities, 242 issues were identified, primarily related to drug safety and equipment management. At follow-up, 45.5% of the issues were resolved, with 44.6% still being addressed. Recommended resources were successfully implemented in 64% of sites. Conclusion : This process demonstrates that focusing on system-level changes can significantly enhance healthcare systems. The reporting framework provided a robust, achievable and synergistic method to measure simulation impact and influence change. Additionally, we share key lessons learned from the process to guide other simulation services in improving their own measurement strategies. Simulation Outreach Value Quality & Safety Improvement Figures Figure 1 Figure 2 INTRODUCTION Simulation programs aiming to correlate their work with improved clinical outcomes face an extraordinarily challenging task. Understanding simulation’s impact is important, but it’s often one of many simultaneous improvement processes occurring in any healthcare system. Isolating simulation’s impact may necessitate large data sets, time and resources, and still has limited likelihood of proving causation(1). Making matters worse, spending resources to ‘prove sim works’ may come at the expense of service provision. Conversely, avoiding measuring impact altogether sacrifices opportunities for accountability, reflexivity and service improvement. Measuring our impact matters, but needs to be feasible. To balance this tension between ‘what is desirable’ and ‘what is feasible’, simulation services need pragmatic processes to measure their impact that are robust, achievable and synergistic with their mission. As a simulation education service, we wanted to understand the impact of our work. Founded in 2014, the Simulation Training Optimising Resuscitation for Kids (STORK) team from Children’s Health Queensland delivers courses to clinicians across Queensland, Australia. Our program’s mission is “ensuring every child in Queensland has access to optimal resuscitative care”. During our first decade of practice we pursued this mission through extensive provision of resuscitation education, delivering 100-140 courses per year, to over 60 hospitals. This involved every Hospital and Health Service in a state more than twice the size of Texas, USA. Measuring our impact on actual paediatric resuscitations was desirable but not feasible given our widely distributed and heterogeneous participant group. Any improvement in statewide paediatric outcomes would be tempting to claim as validation of our impact but statistically dubious. To measure our impact statewide, we needed a feasible alternative separate to measuring patient outcome trends. Supported by recurrent funding from our organisation, our motivation to measure impact was intrinsically driven. Despite our educational reach and positive survey reactions, we noticed recurrent and persistent resuscitation challenges when returning to the same hospitals. We knew our educational programs were popular, but remained concerned about their limited impact on healthcare. While our simulations provided rich data about barriers to optimal care in regional settings, these insights weren’t consistently translated into action. Acceptance that “education was essential, but insufficient”(2) prompted us to embed quality improvement concepts within our second generation courses and pivot from hypothesising impact on resuscitated patients , to measuring impact on resuscitation systems . To keep our methods robust , we began to document, report and follow up on every latent safety threat identified on our courses. To keep the process achievable , we utilised familiar technology: a combination of sticky notes, Microsoft Forms and Office. To keep our process synergistic with our mission, we designed our reports for internal and external use, sharing with local hospitals. In doing so we created a process that served as an internal auditing tool for us, and an external advocacy tool for regional change agents. In our first year, we documented changes in regional sites that had previously proved elusive (3). Within this article, we describe our educational strategy, our reporting and follow up process, and lessons learned from the first year. In doing so, we aim to empower other simulation outreach services to connect their education more intentionally to quality improvement processes. BACKGROUND The STORK education delivery model STORK provides resuscitation training through a spiral curriculum named Optimising Paediatric Training In Emergencies Using Simulation (OPTIMUS). Our curriculum consists of four major constituent courses targeted as either skills development or skills maintenance vehicles for first or second tier responders. Additional simulation packages offer spaced repetition opportunities. Faculty resources are shared online for non-commercial use, with courses delivered by a combination of STORK staff and regional educators. The STORK team comprises four part-time paediatric emergency medicine physicians, two part-time simulation fellows, two nurse educators, five part-time simulation coordinators, and one administration officer. Courses are coordinated across facilities in Queensland year-round. Each simulation coordinator is responsible for 10-12 sites and for communication regarding courses, site summaries and follow-ups. This model has helped our team build and maintain relationships with colleagues working in rural and remote sites. METHODS Incorporating Quality Improvement into the Optimus PRIME course Our most frequently delivered course, “Preparing for Retrieval In Medical Emergencies (Optimus PRIME)”, focuses on resuscitation and retrieval of critically unwell children. We support participants to build on their current knowledge with an intentionally constructivist approach to learning. We focus on how to perform recurrent skills and why non-recurrent skills are performed in a particular way(4). Independent knowledge and skills application is supported during immersive simulation exercises. In our second iteration of the course, we introduced principles from translational simulation and quality improvement science(5). The 4-levels Human Factor Framework ‘Self, Team, Environment and System’ by Hicks & Petrosoniak(6) is described to participants early in the day. Faculty encourage participants to identify issues within each level during the course, encouraging reflection on ways to improve care at each level. Issues are documented on sticky notes and displayed (Figure 1). During course closure, participants and faculty review all sticky notes to identify solutions and nominate local stakeholders to champion them. Observations, suggestions and contact details are recorded electronically via a QR code to Microsoft Forms. Optimus PRIME Site Summary design The Optimus PRIME Site Summary is a 2-page report divided into four sections, (1) demographics (site name, date of visit, course delivered), (2) service strengths, (3) challenges/suggested actions/stakeholder contact for each category and (4) additional resources. A standardised appendix contains information on equipment ordering codes, their relevance in resuscitation, and suggestions for additional training or resources. Feedback is focused on actionable, contextualised solutions, with an emphasis on co-generation and ownership by local course participants. [ see online supplement: Optimus PRIME site summary] The initial design phase included meetings to identify key reporting categories. Five categories were identified as aligning well with the Human Factor framework: Strengths, Teamwork, Drug Safety, Equipment and Department Layout and Resources. Selection was informed by faculty knowledge about current challenges, as well as published work on paediatric readiness(7). A data collection form was created in Microsoft Forms using a mix of closed and forced choice answers to facilitate collection of demographic data and free text. Form submission auto-generates a Microsoft Word document within Microsoft Teams , importing information into a site summary template. Several design decisions were embedded within the report structure: it’s colourful and visually appealing, strengths are featured early to build rapport and reflect our Safety 2 mindset(8) , and recommendations are action-oriented. The summary synthesises challenges and solutions identified on the day, and isn’t an independent measure of service quality. Participant performance isn’t measured. The report system was created with an iterative design cycle approach. Over the first year, it was repeatedly adapted in response to feedback. For example, the term ‘PRIME Report’ was changed to ‘PRIME Site Summary’ in response to individual feedback regarding perceived evaluative stress. Optimus PRIME Site Summary delivery & follow-up After each course, a simulation coordinator revises the auto-generated summary document, refining wording and ensuring accuracy. After exporting the document to pdf format, each summary is emailed to stakeholders identified on the course. We target distribution of the summary within 14 days. A pre-scripted email introduction emphasises the summary is offered in good faith and not intended as an assessment. On a case-by-case basis, our simulation coordinators often send additional emails to specific local stakeholders to help kick-start conversations about improvement opportunities. A 3-month follow-up is organised with each site to review interval changes made within their system and explore ongoing challenges. Agenda items from the reports are discussed and categorised as either “no further action required”, “work in progress” or “item closed without action” and documented within a Microsoft Excel spreadsheet. Stakeholders are encouraged to provide feedback about the process itself. Data collection and analysis Data fromsite summaries and follow-ups were extracted by a researcher [A.SOSH] using a standardised abstraction form. Common themes were extracted, focusing on systemic issues or those representing a latent safety threat. Descriptive statistics for categorical variables were calculated using proportions. Non-parametric tests were used due to the distribution of the data. We have previously published the prominent latent safety threats identified(3). Within this article, we report findings more relevant for simulation services themselves. RESULTS Between March and December 2023, 40 Optimus PRIME courses were delivered in 37 healthcare facilities. Our faculty engaged with 346 participants, 72% were nurses and 28% doctors or medical trainees. Thirty-nine site summaries were distributed, as one site was visited a second time before the summary completion. A total of 242 problems were diagnosed during our study period. A median of 6 issues were discovered by participants per site, with problems relating to drug safety and equipment being more frequent (Table 1). Table 1. Findings of the initial Optimus PRIME Report Summaries. Initial Optimus PRIME Report Summary Issues per category Count (Mean per site; SD) Common themes Drug safety 81 (2.1; 1) •Infrequent infusion pump safety software library updates. Equipment 82 (2.1; 1.64) •Challenges with consistently maintaining paediatric airway equipment. •Lack of internet & computer access in ED. Resources 48 (1.23; 0.71) •Challenges with access and awareness of statewide open access resources. Teamwork 31 (0.79; 0.89) •Highly valued rich close knit relationship with colleagues. •Frequent hospital staff turn over. Total 242 (6.2; 2.9) At site follow-up, 45.5% of issues were resolved while 44.6% were still being addressed (Table 2). Most sites (64%, n = 25) were able to order resources recommended. Table 2. Findings of the follow-up process of the Optimus PRIME Reports Summaries. Follow-up of Optimus PRIME Summary Overall quality improvement metrics Count (Percentage) Improvement from initial summary •Issues resolved •Issues as work in progress •Issues with no action undertaken •Data not available 218/242 (90%) 110/242 (45.5%) 108/242 (44.6%) 18/242 (7.4%) 6/242 (2.5%) Sites able to order new resources 25/39 (64%) With respect to our performance, delays were noted with both the initial summary delivery (Median (IQR) 40 days [19.75 – 53.25]) and the timing of follow-ups (Median (IQR) 161 days [130.5 – 232.5]). The follow-up process took an average of 47 minutes per course to complete. DISCUSSION Our site summary process aimed to create a robust method of measuring the impact of our simulation program, that was achievable with our current staffing and synergistic with our organisational mission. Within our discussion, we reflect on each of these goals in turn. A more robust method of measuring the impact of a simulation program Simulation services seeking to measure their impact often reference Kirkpatrick’s framework(9). Kirkpatrick presents a hierarchy of four levels of measures, moving through participant reactions (e.g. participant surveys on engagement and relevance), learning (e.g. measured improvement on the day), behavior (e.g. integration of new skills into actual workplace practice) and results (e.g patient outcomes and organisational changes that occur subsequent to training). We had previously attempted to evaluate our training to the Kirkpatrick ‘Reactions’ level with participant surveys. Feedback and our net promoter score showed that we were a popular service, but comments were mostly appreciative rather than constructive. We intentionally avoid summative assessment of ‘Learning’ in our PRIME course due to concerns of generating high levels of social-evaluative stress when visiting regional areas. We argued that ‘Learning’ was partially demonstrated when individuals shared take-home learnings at the end of the course. The effectiveness of our course at the "Behaviors" level was reflected in participants' emails recounting successful resuscitations after the course (Figure 3). These accounts illustrate changes at Kirkpatrick's Level 3, showing that participants adopted behaviors capable of influencing and shaping an established culture. Our approach incorporated specific learning principles and practical design choices to drive these behavioral outcomes. Drawing from social cognitive learning principles, we structured simulations in situ to create an authentic, engaging representation of "work as done" rather than "work as imagined”. This approach mirrors real-world practices, providing concrete experiences essential to enhance learning, trigger reactions, and shape behaviors. Additionally, we used cognitive psychology’s problem-solving framework(10) to lead participants through stages of problem identification, reflection, solution discovery, and testing in their real world clinical environment. By using authentic equipment, and forming realistic teams with their inherent dynamics—while fostering problem-solving skills and generating practical, context-specific solutions—we empowered participants to adopt and sustain behaviors that would directly impact their clinical practice. While these narratives enriched our understanding of our impact, we lacked comprehensive data to confirm widespread impact and lacked a measure to assess the “Results” level. By shifting our focus from patient to system improvement, we documented site-specific work against identified issues, confirming discrete episodes of organisational change consistent with Kirkpatrick ‘Results’. Importantly we do not pretend these successes are ours alone, but argue they are the result of collective and coordinated action between simulation outreach and local clinicians. Our follow up process documented organisational, equipment and process changes at individual hospitals (Table 2). With 45% of issues fully resolved within three months and an additional 45% in progress, we documented changes on all 4-levels of Kirkpatrick’s Model. We also celebrated and propagated regional expertise and innovations across the state. These results highlight the strength of our reporting system, particularly its ability to harness individual clinician’s capacity to champion, coordinate and generate organisational changes. An achievable method of measuring the impact of a simulation program Our reporting process was facilitated without additional staffing or expenses. This was achievable through the use of familiar software, maximising automation, administrative support and longitudinal relationships with regional sites. We utilised Office 365 software throughout the process. Staff were familiar with our organisation’s licensed software, and the integration of QR codes to link to reporting forms enabled staff to quickly complete data collection on their smartphones during course closure or travel time from courses. Familiar technology avoided adding further barriers to data collection. A combination of software automation and consistent administrative support was essential. Forms and templates were codesigned by our administrative officer, ensuring every process was automated as much as possible. The use of Microsoft Forms to auto generate a Microsoft Word document decreased report writing times. Our administrative officer created automatic email reminders to prompt site summary completions at agreed upon times. A backup system evolved to ensure report completion, wherein outstanding forms are carbon copied to a consultant and nurse educator who could assist with completion. In addition to the technical processes described, we believe our new process was successful because it leveraged relationships built through a decade of delivering paediatric education to every Hospital and Health Service in Queensland. While this education provided individual clinicians with essential training, it also fostered our service’s relationship with other clinical teams. Outreach education has been described as a relational intervention (11,12), and as such, we regard our courses as vital steps in building psychologically safe relationships with regional educators and clinicians. Had we rolled out a statewide reporting process on paediatric resuscitation at the start of our ten year journey, we believe we would have decimated an early regional courtship. As such we strongly recommend other outreach services build psychological safety and establish relationships before launching into this depth of reporting. While we believe success was informed by these deliberate choices (familiar software, maximising automation, administrative support, and longitudinal relationships), the process did have challenges. Report writing was an unappealing new experience for many in our team and we faced difficulties meeting the pre-established contact points of 14 days and 3-months. Follow up was made more difficult due to frequent regional personnel turnover in educational and management roles. This caused interruption in communications and challenges coordinating time with educators amidst clinical duties. These logistic challenges are likely to persist. Some coordinators reported frustration with the additional taskwork, although this decreased with familiarity and time. Additional support processes were arranged for report completion. Two senior clinicians (BS and JA) with confidence writing diplomatic, unambiguous communication took on a ‘reviewer’ role, to support consistency and redistribute individual workloads. Administrative support, such as monitoring time intervals and creating clear, timely email templates for follow-up, helped to improve these metrics in the latter part of our study period. We perceived that staff frustration with the process decreased as follow up data began to show real impact. During team meetings, we focused on our primary goals of education delivery and collaboration with local champions, and emphasised flexibility and adaptability in our process. Finding the perfect balance between a personalised site-specific reporting timeline and a rigorous, standardised research reporting timeline remains a challenge. Therefore, we will continue our iterative process improvement cycle and aim to measure and report on the challenges and impact of our efforts prospectively. A synergistic approach to measuring the impact of a simulation program Our service sought a process that was synergistic with our mission and educational processes. Keeping our mission, “ensuring every child in Queensland has access to optimal resuscitative care” forefront ensured we designed a process that supported our work rather than distracting from it. At the practical level, we designed the summaries to serve multiple purposes. We use our site summaries to understand the impact of our service, but we use the same summaries in statewide meetings on recurrent themes throughout Queensland. In parallel, external staff members use them to advocate for changes in local steering groups, committees and in individual emails to key stakeholders. Several regional educators have stated the process provided them with the external authority needed to push for movement on a disregarded issue. The reporting process was a catalyst for change itself, providing us with the infrastructure needed to achieve what we had previously only hoped was happening. Our experience mirrors the conclusions of a systematic review of latent safety threats (LSTs) identified through In Situ simulation, which highlight that LSTs are frequently identified but infrequently closed. The authors emphasise that “clear and structured processes need to be implemented for the resolution of patient safety threats identified through simulation.”(13). The reporting system also helped us recognise issues that were shared across statewide hospital resuscitation systems, allowing us to advocate at higher levels for broader interventions. We have recognised that the ‘transformational conversations’ we were having in our debriefs are more likely to impact the healthcare system when combined with diligent follow up. We believe more simulation services could benefit from deliberately connecting findings from their learning conversations into quality improvement processes. Lessons learned and future avenues (limitations/ implications for future research) In order to optimise translation of our learning to other services, we have synthesised 5 key ‘lessons learned’ that we believe other services may find useful. 1. Education is insufficient, but vital. Until we changed our strategy, the systemic challenges we uncovered did not improve. However our prior decade of educational efforts were not wasted. Positive participant experiences and longitudinal relationships created the psychological safety needed to have harder conversations about the system. Simultaneously, our educators developed an incredible mental map of the issues faced by individual hospitals region-wide that heavily informed our understanding of statewide resuscitation. 2. Iterative approaches were important We received feedback from local educators requesting more direct communication with hospital stakeholders to support their efforts. We added additional contact lists to our data collection, carbon copied hospital executives and reached out directly to stakeholders to support participant’s initiative when needed. 3 . The mission is what matters While demonstrating impact on patient outcomes statewide was an alluring dream, it wasn’t feasible and could have distracted us from more targeted, useful questions. Keeping our mission at the centre of all design choices, “i.e. how does this help sick kids”, ensured a streamlined approach with a meaningful outcome. Diligence is key It was not an expensive new mannequin, a technological leap or a masterful debrief that magnified our impact, it was diligence. We postulate that simulation educators spend an extraordinary amount of time reflecting on the quality of their debriefs, but less time reflecting on how to harness those conversations for good, and how to connect that information to the right people. Our fullest potential was unlocked by diligent documentation, deliberate connection to key stakeholders, and gently belligerent follow up. 5. Local buy-in is essential While our PRIME Site Summary provided important data to hospitals, it was local change agents who actioned every response. Sites with passionate nurse educators, pharmacists and doctors were able to utilise the reports to build momentum for change. In one site in particular, we were humbled by the achievements of one nurse educator in closing every issue we found on every course. The reports and communication are catalysts for change, but the real heavy lifting is done by regional clinicians. This potential was hampered in sites with rapid staff turnover, but changes were remarkably dynamic in sites with long term, experienced educators. It is our honest reflection that there are few things more transformative for a regional hospital than a passionate local clinician armed with unwavering conviction, an external report, and their patient’s best interests at heart. As one regional nurse educator put it; “This is our community. These are our kids. They deserve the best” (L. Moore, personal communication, May 15th 2024). Limitations Within this paper we share findings from 12 months of our new site reporting process. We have documented action on identified latent safety threats within all hospitals involved, but acknowledge some latent safety threats would have been closed whether or not we had flagged them. For example, as some sites roll out electronic medical records, more computers in resuscitation spaces are routinely installed, conveniently addressing our concerns about internet access in resuscitation rooms. Regardless of this, we are confident that site summaries played a significant role in many improvement processes, as evidenced by the numerous verbal and written communication channels we’ve been involved in since. Identified latent safety threats are also specific to the curriculum of our course and shaped by the location of the simulation (In Situ versus education rooms), equipment used (training drug infusion pumps versus actual ward equipment) and the perspectives of individual participants and faculty. We argue however that our goal was not to provide an objective measure of performance, but to link valuable data generated in educational settings with organisational improvement systems. We have demonstrated such a system is both feasible and helpful for workplace improvement. While we are confident our intervention impacted paediatric resuscitation systems, we have not proved improvement of actual paediatric resuscitations. Some of our recommendations may have unpredicted negative impacts, however we argue this highlights the need for continuous cycles of performance improvement. Conclusion Simulation services seeking to measure impact need processes that are robust, achievable and synergistic with their mission. We have described our simulation outreach model and presented the results of our robust reporting and advocacy process. Our use of software automation, administrative support and longitudinal relationships made the process achievable without additional expense. We kept our processes synergistic with our mission by incorporating quality improvement methodologies into our existing educational courses, and in doing so have documented system problems and demonstrated tangible changes within healthcare systems at both the learner and institutional levels. We hope this article provides detailed methods and generalisable principles for other simulation programs seeking to demonstrate concrete outcomes, mitigate latent safety threats and prove their value. Abbreviations STORK: Simulation Training Optimising Resuscitation for Kids, the simulation outreach service for Children’s Health Queensland PRIME: Preparing for Retrieval in Medical Emergencies, a course frequently delivered by STORK ED: emergency department LSTs: latent safety threats Declarations Funding: No funding was sought or obtained for this study. Conflict of interest: Nil to declare Author Contribution A. SOSH wrote the first draft of the manuscript, collated and analysed the associated data, and created first drafts of the tables and figures. She took a primary role in the project.B.L, L.D, J.A, D.H all reviewed the document and contributed their perspectives, and were all heavily involved with the development and roll out of the initiatives described.B.S. contributed heavily to multiple sections of the manuscript, provided guidance on vision, and created the visual abstract. Acknowledgement The authors thank Tony Carter, Samuel Greer, Tina Haffenden, Jillian O'donnell, Emma Perry and Stefan Pietsch for their contribution to data acquisition, Nikki Harrison for her data management and online form creation, and for Lois Moore for her hard work in local organisational transformation and permission to use her quote. Availability of data and materials The dataset analysed during the current study are available from the corresponding author on reasonable request. References Varpio L, Sherbino J. Demonstrating causality, bestowing honours, and contributing to the arms race: Threats to the sustainability of HPE research. Med Educ. 2024 Jan;58(1):157–63. Soong C, Shojania KG. Education as a low-value improvement intervention: often necessary but rarely sufficient. BMJ Qual Saf. 2020 May;29(5):353–7. St‐Onge‐St‐Hilaire A, Acworth J, Lawton B, Williams M, Dodson L, Symon B. Paediatric resuscitation in regional Queensland: A simulation informed biopsy of current system challenges. Emerg Med Australas. 2025 Apr;37(2):e70028. Frerejean J, Van Merriënboer JJG, Condron C, Strauch U, Eppich W. Critical design choices in healthcare simulation education: a 4C/ID perspective on design that leads to transfer. Adv Simul. 2023 Feb 24;8(1):5. Brazil V. Translational simulation: not ‘where?’ but ‘why?’ A functional view of in situ simulation. Adv Simul. 2017 Dec;2(1):20. Hicks C, Petrosoniak A. The Human Factor. Emerg Med Clin North Am. 2018 Feb;36(1):1–17. Remick K, Gausche-Hill M, Joseph MM, Brown K, Snow SK, Wright JL, et al. Pediatric Readiness in the Emergency Department. Pediatrics. 2018 Nov 1;142(5):e20182459. Hollnagel E, Wears RL, Braithwaite J. From Safety-I to Safety-II: A White Paper [Internet]. University of Southern Denmark; 2015 [cited 2025 Mar 9]. Available from: http://rgdoi.net/10.13140/RG.2.1.4051.5282 Falletta S. Evaluating Training Programs: The Four Levels Donald L. Kirkpatrick, Berrett-Koehler Publishers, San Francisco, CA, 1996, 229 pp. Am J Eval. 1998;19(2):259–61. Hardin LE. Problem-Solving Concepts and Theories. J Vet Med Educ. 2003 Sep;30(3):226–9. Mattick K, Noble C. Education and educational interventions: moving beyond information provision. BMJ Qual Saf. 2024 Jan;33(1):10–2. Luetsch K, Wong G, Rowett D. A realist synthesis of educational outreach visiting and integrated academic detailing to influence prescribing in ambulatory care: why relationships and dialogue matter. BMJ Qual Saf. 2024 Jan;33(1):43–54. Grace MA, O’Malley R. Using In Situ Simulation to Identify Latent Safety Threats in Emergency Medicine: A Systematic Review. Simul Healthc J Soc Simul Healthc. 2024 Aug;19(4):243–53. Additional Declarations No competing interests reported. Supplementary Files DigitalSupplementOptimusPRIMESiteSummary.pdf floatimage1.jpeg VISUAL ABSTRACT Cite Share Download PDF Status: Published Journal Publication published 01 Sep, 2025 Read the published version in Advances in Simulation → Version 1 posted Editorial decision: Revision requested 04 Jun, 2025 Reviews received at journal 15 Apr, 2025 Reviews received at journal 13 Apr, 2025 Reviewers agreed at journal 12 Apr, 2025 Reviewers agreed at journal 31 Mar, 2025 Reviewers invited by journal 25 Mar, 2025 Editor assigned by journal 18 Mar, 2025 Submission checks completed at journal 18 Mar, 2025 First submitted to journal 15 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6230704","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"advancing-simulation-practice","associatedPublications":[],"authors":[{"id":436403447,"identity":"4230b415-77c4-4459-9ec2-ed46299f7515","order_by":0,"name":"A. St-Onge-St-Hilaire","email":"","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"A.","middleName":"","lastName":"St-Onge-St-Hilaire","suffix":""},{"id":436403451,"identity":"e417c1d1-575d-49a6-9525-94a3c1064fb1","order_by":1,"name":"B. Lawton","email":"","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"B.","middleName":"","lastName":"Lawton","suffix":""},{"id":436403454,"identity":"ec5c307a-253d-4ae0-bf8a-2d95fd21a360","order_by":2,"name":"L. Dodson","email":"","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"L.","middleName":"","lastName":"Dodson","suffix":""},{"id":436403455,"identity":"bb1f6ad4-f1cf-4510-98ab-5ec6cbcd7828","order_by":3,"name":"J. Acworth","email":"","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"J.","middleName":"","lastName":"Acworth","suffix":""},{"id":436403456,"identity":"7a70e008-baee-4725-b340-1ba85807efbb","order_by":4,"name":"D. Hufton","email":"","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":false,"prefix":"","firstName":"D.","middleName":"","lastName":"Hufton","suffix":""},{"id":436403458,"identity":"f2563311-a9fc-4df8-9542-4ac7001b09bc","order_by":5,"name":"Ben Symon","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA3ElEQVRIiWNgGAWjYDCCAzwwFvMBICEhQ4oWtgSQFh7cSjG18BiASYI6+G6fPfi44M/haH7pns+vbtRY8DCwHz66AZ8WyXN5ycYz2w7nzpxzdpt1zjGgw3jS0m7g02JwhsdMmrfhcO6GG7nbjHPYgFokeMwIaTH/zfMHpCXnmXHOP+K0mDHzsIG1MD/ObSNCi+QZvmRp3rb03Jkz0syYc/skeNgI+YXvDO/Bzzx/rHP7JZIff875VifHz374GF4tUNAMItgkwCQRykGgDkQwfyBS9SgYBaNgFIwwAAB20UmFpyGyWQAAAABJRU5ErkJggg==","orcid":"","institution":"Children’s Health Queensland Hospital and Health Service","correspondingAuthor":true,"prefix":"","firstName":"Ben","middleName":"","lastName":"Symon","suffix":""}],"badges":[],"createdAt":"2025-03-15 06:08:03","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6230704/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6230704/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s41077-025-00372-0","type":"published","date":"2025-09-01T15:57:20+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":79909974,"identity":"5234ecd4-7af4-4059-8ef6-a9f7d4c15671","added_by":"auto","created_at":"2025-04-04 11:27:31","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":156830,"visible":true,"origin":"","legend":"\u003cp\u003eVisual display of sticky notes categorised using the ‘Self, Team, Environment, System’ framework.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6230704/v1/3b6f325ab2e8b708f68b117b.jpeg"},{"id":79909978,"identity":"e8b20014-96e6-418b-a76c-6dd992e7f752","added_by":"auto","created_at":"2025-04-04 11:27:31","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":161193,"visible":true,"origin":"","legend":"\u003cp\u003eNarrative comments and feedback from course participants and educators in relation to Kirkpatrick’s Model.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6230704/v1/d01bdeb2074c96e9b4cd0cf3.jpeg"},{"id":90827937,"identity":"e99fbf1c-f659-4fbc-ac46-139df2d18eb4","added_by":"auto","created_at":"2025-09-08 16:03:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1043346,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6230704/v1/98468c09-d21c-4a62-8768-e1484f39afed.pdf"},{"id":79909979,"identity":"6f3ace5a-4dc9-485b-9021-d71551312693","added_by":"auto","created_at":"2025-04-04 11:27:31","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":495995,"visible":true,"origin":"","legend":"","description":"","filename":"DigitalSupplementOptimusPRIMESiteSummary.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6230704/v1/5681be17342629a0c19458c9.pdf"},{"id":79910779,"identity":"186038ef-e050-4c25-a7df-4912777328b3","added_by":"auto","created_at":"2025-04-04 11:35:31","extension":"jpeg","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":620441,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eVISUAL ABSTRACT\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-6230704/v1/23bd8f3161b3cb01c796d542.jpeg"}],"financialInterests":"No competing interests reported.","formattedTitle":"Outreach Simulation for System Improvement: a Novel Advocacy and Reporting Process","fulltext":[{"header":"INTRODUCTION","content":"\u003cp\u003eSimulation programs aiming to correlate their work with improved clinical outcomes face an extraordinarily challenging task. Understanding simulation\u0026rsquo;s impact is important, but it\u0026rsquo;s often one of many simultaneous improvement processes occurring in any healthcare system. Isolating simulation\u0026rsquo;s impact may necessitate large data sets, time and resources, and still has limited likelihood of proving causation(1). Making matters worse, spending resources to \u0026lsquo;prove sim works\u0026rsquo; may come at the expense of service provision. Conversely, avoiding measuring impact altogether sacrifices opportunities for accountability, reflexivity and service improvement. Measuring our impact matters, but needs to be feasible. To balance this tension between \u0026lsquo;what is desirable\u0026rsquo; and \u0026lsquo;what is feasible\u0026rsquo;, simulation services need pragmatic processes to measure their impact that are robust, achievable and synergistic with their mission.\u003c/p\u003e\n\u003cp\u003eAs a simulation education service, we wanted to understand the impact of our work. Founded in 2014, the Simulation Training Optimising Resuscitation for Kids (STORK) team from Children\u0026rsquo;s Health Queensland delivers courses to clinicians across Queensland, Australia. Our program\u0026rsquo;s mission is \u0026ldquo;ensuring every child in Queensland has access to optimal resuscitative care\u0026rdquo;. During our first decade of practice we pursued this mission through extensive provision of resuscitation education, delivering 100-140 courses per year, to over 60 hospitals. This involved every Hospital and Health Service in a state more than twice the size of Texas, USA. Measuring our impact on actual paediatric resuscitations was desirable but not feasible given our widely distributed and heterogeneous participant group. Any improvement in statewide paediatric outcomes would be tempting to claim as validation of our impact but statistically dubious. To measure our impact statewide, we needed a feasible alternative separate to measuring patient outcome trends.\u003c/p\u003e\n\u003cp\u003eSupported by recurrent funding from our organisation, our motivation to measure impact was intrinsically driven. Despite our educational reach and positive survey reactions, we noticed recurrent and persistent resuscitation challenges when returning to the same hospitals. We knew our educational programs were popular, but remained concerned about their limited impact on healthcare. While our simulations provided rich data about barriers to optimal care in regional settings, these insights weren\u0026rsquo;t consistently translated into action. Acceptance that \u0026ldquo;education was essential, but insufficient\u0026rdquo;(2) prompted us to embed quality improvement concepts within our second generation courses and pivot from hypothesising impact on resuscitated \u003cem\u003epatients\u003c/em\u003e, to measuring impact on resuscitation\u003cem\u003e\u0026nbsp;systems\u003c/em\u003e.\u003c/p\u003e\n\u003cp\u003eTo keep our methods \u003cem\u003erobust\u003c/em\u003e, we began to document, report and follow up on every latent safety threat identified on our courses. To keep the process \u003cem\u003eachievable\u003c/em\u003e, we utilised familiar technology: a combination of sticky notes, Microsoft Forms and Office. To keep our process \u003cem\u003esynergistic\u003c/em\u003e with our mission, we designed our reports for internal and external use, sharing with local hospitals. In doing so we created a process that served as an internal auditing tool for us, and an external advocacy tool for regional change agents. In our first year, we documented changes in regional sites that had previously proved elusive (3).\u003c/p\u003e\n\u003cp\u003eWithin this article, we describe our educational strategy, our reporting and follow up process, and lessons learned from the first year. In doing so, we aim to empower other simulation outreach services to connect their education more intentionally to quality improvement processes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eBACKGROUND\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cu\u003eThe STORK education delivery model\u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSTORK provides resuscitation training through a spiral curriculum named Optimising Paediatric Training In Emergencies Using Simulation (OPTIMUS). Our curriculum consists of four major constituent courses targeted as either skills development or skills maintenance vehicles for first or second tier responders. Additional simulation packages offer spaced repetition opportunities. Faculty resources are shared online for non-commercial use, with courses delivered by a combination of STORK staff and regional educators.\u003c/p\u003e\n\u003cp\u003eThe STORK team comprises four part-time paediatric emergency medicine physicians, two part-time simulation fellows, two nurse educators, five part-time simulation coordinators, and one administration officer. Courses are coordinated across facilities in Queensland year-round. Each simulation coordinator is responsible for 10-12 sites and for communication regarding courses, site summaries and follow-ups. This model has helped our team build and maintain relationships with colleagues working in rural and remote sites.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003e\u003cstrong\u003e\u003cu\u003eIncorporating Quality Improvement into the Optimus PRIME course\u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur most frequently delivered course, \u0026ldquo;Preparing for Retrieval In Medical Emergencies (Optimus PRIME)\u0026rdquo;, focuses on resuscitation and retrieval of critically unwell children. We support participants to build on their current knowledge with an intentionally constructivist approach to learning. We focus on \u003cem\u003ehow\u003c/em\u003e to perform recurrent skills and \u003cem\u003ewhy\u003c/em\u003e non-recurrent skills are performed in a particular way(4). Independent knowledge and skills application is supported during immersive simulation exercises.\u003c/p\u003e\n\u003cp\u003eIn our second iteration of the course, we introduced principles from translational simulation and quality improvement science(5). The 4-levels Human Factor Framework \u0026lsquo;Self, Team, Environment and System\u0026rsquo; by Hicks \u0026amp; Petrosoniak(6) is described to participants early in the day. Faculty encourage participants to identify issues within each level during the course, encouraging reflection on ways to improve care at each level. Issues are documented on sticky notes and displayed (Figure 1). During course closure, participants and faculty review all sticky notes to identify solutions and nominate local stakeholders to champion them. Observations, suggestions and contact details are recorded electronically via a QR code to Microsoft Forms.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cu\u003eOptimus PRIME Site Summary design\u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Optimus PRIME Site Summary is a 2-page report divided into four sections, (1) demographics (site name, date of visit, course delivered), (2) service strengths, (3) challenges/suggested actions/stakeholder contact for each category and (4) additional resources. A standardised appendix contains information on equipment ordering codes, their relevance in resuscitation, and suggestions for additional training or resources. Feedback is focused on actionable, contextualised solutions, with an emphasis on co-generation and ownership by local course participants. [\u003cem\u003esee online supplement: Optimus PRIME site summary]\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe initial design phase included meetings to identify key reporting categories. Five categories were identified as aligning well with the Human Factor framework: Strengths, Teamwork, Drug Safety, Equipment and Department Layout and Resources. Selection was informed by faculty knowledge about current challenges, as well as published work on paediatric readiness(7).\u003c/p\u003e\n\u003cp\u003eA data collection form was created in \u003cem\u003eMicrosoft Forms \u003c/em\u003eusing a mix of closed and forced choice answers to facilitate collection of demographic data and free text. Form submission auto-generates a \u003cem\u003eMicrosoft Word \u003c/em\u003edocument within \u003cem\u003eMicrosoft Teams\u003c/em\u003e, importing information into a site summary template.\u003c/p\u003e\n\u003cp\u003eSeveral design decisions were embedded within the report structure: it\u0026rsquo;s colourful and visually appealing, strengths are featured early to build rapport and reflect our Safety 2 mindset(8) , and recommendations are action-oriented. The summary synthesises challenges and solutions identified on the day, and isn\u0026rsquo;t an independent measure of service quality. Participant performance isn\u0026rsquo;t measured.\u003c/p\u003e\n\u003cp\u003eThe report system was created with an iterative design cycle approach. Over the first year, it was repeatedly adapted in response to feedback. For example, the term \u0026lsquo;PRIME Report\u0026rsquo; was changed to \u0026lsquo;PRIME Site Summary\u0026rsquo; in response to individual feedback regarding perceived evaluative stress.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003e\u003cu\u003eOptimus PRIME Site Summary delivery \u0026amp; follow-up \u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAfter each course, a simulation coordinator revises the auto-generated summary document, refining wording and ensuring accuracy. After exporting the document to pdf format, each summary is emailed to stakeholders identified on the course. We target distribution of the summary within 14 days. A pre-scripted email introduction emphasises the summary is offered in good faith and not intended as an assessment. On a case-by-case basis, our simulation coordinators often send additional emails to specific local stakeholders to help kick-start conversations about improvement opportunities.\u003c/p\u003e\n\u003cp\u003eA 3-month follow-up is organised with each site to review interval changes made within their system and explore ongoing challenges. Agenda items from the reports are discussed and categorised as either \u0026ldquo;no further action required\u0026rdquo;, \u0026ldquo;work in progress\u0026rdquo; or \u0026ldquo;item closed without action\u0026rdquo; and documented within a Microsoft Excel spreadsheet. Stakeholders are encouraged to provide feedback about the process itself. \u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cu\u003eData collection and analysis\u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData fromsite summaries and follow-ups were extracted by a researcher [A.SOSH] using a standardised abstraction form. Common themes were extracted, focusing on systemic issues or those representing a latent safety threat. Descriptive statistics for categorical variables were calculated using proportions. Non-parametric tests were used due to the distribution of the data. We have previously published the prominent latent safety threats identified(3). Within this article, we report findings more relevant for simulation services themselves.\u003c/p\u003e"},{"header":"RESULTS","content":"\u003cp\u003eBetween March and December 2023, 40 Optimus PRIME courses were delivered in 37 healthcare facilities. Our faculty engaged with 346 participants, 72% were nurses and 28% doctors or medical trainees. Thirty-nine site summaries were distributed, as one site was visited a second time before the summary completion.\u003c/p\u003e\n\u003cp\u003eA total of 242 problems were diagnosed during our study period. A median of 6 issues were discovered by participants per site, with problems relating to drug safety and equipment being more frequent (Table 1).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Table 1. Findings of the initial Optimus PRIME Report Summaries.\u003c/p\u003e\n\u003cdiv align=\"\"\u003e\n \u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"666\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 666px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eInitial Optimus PRIME Report Summary\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003eIssues per category\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003eCount\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e(Mean per site; SD)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003eCommon themes\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003eDrug safety\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003e81 (2.1; 1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003e\u0026bull;Infrequent infusion pump safety software library updates.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003eEquipment\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003e82 (2.1; 1.64)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003e\u0026bull;Challenges with consistently maintaining paediatric airway equipment.\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026bull;Lack of internet \u0026amp; computer access in ED.\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003eResources\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003e48 (1.23; 0.71)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003e\u0026bull;Challenges with access and awareness of statewide open access resources.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003eTeamwork\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003e31 (0.79; 0.89)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003e\u0026bull;Highly valued rich close knit relationship with colleagues.\u003c/p\u003e\n \u003cp\u003e\u0026bull;Frequent hospital staff turn over.\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 146px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTotal\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 145px;\"\u003e\n \u003cp\u003e242 (6.2; 2.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 375px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eAt site follow-up, 45.5% of issues were resolved while 44.6% were still being addressed (Table 2). Most sites (64%, n = 25) were able to order resources recommended.\u003c/p\u003e\n\u003cp\u003eTable 2. Findings of the follow-up process of the Optimus PRIME Reports Summaries.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"2\" style=\"width: 624px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFollow-up of Optimus PRIME Summary\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 303px;\"\u003e\n \u003cp\u003eOverall quality improvement metrics\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 321px;\"\u003e\n \u003cp\u003eCount (Percentage)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 303px;\"\u003e\n \u003cp\u003eImprovement from initial summary\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026bull;Issues resolved\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026bull;Issues as work in progress\u003c/p\u003e\n \u003cp\u003e\u0026bull;Issues with no action undertaken\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026bull;Data not available\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 321px;\"\u003e\n \u003cp\u003e218/242 (90%)\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e110/242 (45.5%)\u003c/p\u003e\n \u003cp\u003e108/242 (44.6%)\u003c/p\u003e\n \u003cp\u003e18/242 (7.4%)\u003c/p\u003e\n \u003cp\u003e6/242 (2.5%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 303px;\"\u003e\n \u003cp\u003eSites able to order new resources\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 321px;\"\u003e\n \u003cp\u003e25/39 (64%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;With respect to our performance, delays were noted with both the initial summary delivery (Median (IQR) 40 days [19.75 \u0026ndash; 53.25]) and the timing of follow-ups (Median (IQR) 161 days [130.5 \u0026ndash; 232.5]). The follow-up process took an average of 47 minutes per course to complete.\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eOur site summary process aimed to create a \u003cem\u003erobust\u003c/em\u003e method of measuring the impact of our simulation program, that was \u003cem\u003eachievable\u003c/em\u003e with our current staffing and \u003cem\u003esynergistic\u003c/em\u003e with our organisational mission. Within our discussion, we reflect on each of these goals in turn.\u003c/p\u003e\n\u003col start=\"1\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eA more robust method of measuring the impact of a simulation program\u003c/strong\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eSimulation services seeking to measure their impact often reference Kirkpatrick\u0026rsquo;s framework(9). Kirkpatrick presents a hierarchy of four levels of measures, moving through participant reactions (e.g. participant surveys on engagement and relevance), learning (e.g. measured improvement on the day), behavior (e.g. integration of new skills into actual workplace practice) and results (e.g patient outcomes and organisational changes that occur subsequent to training).\u003c/p\u003e\n\u003cp\u003eWe had previously attempted to evaluate our training to the Kirkpatrick \u0026lsquo;Reactions\u0026rsquo; level with participant surveys. Feedback and our net promoter score showed that we were a popular service, but comments were mostly appreciative rather than constructive.\u003c/p\u003e\n\u003cp\u003eWe intentionally avoid summative assessment of \u0026lsquo;Learning\u0026rsquo; in our PRIME course due to concerns of generating high levels of social-evaluative stress when visiting regional areas. We argued that \u0026lsquo;Learning\u0026rsquo; was partially demonstrated when individuals shared take-home learnings at the end of the course.\u003c/p\u003e\n\u003cp\u003eThe effectiveness of our course at the \u0026quot;Behaviors\u0026quot; level was reflected in participants\u0026apos; emails recounting successful resuscitations after the course (Figure 3). These accounts illustrate changes at Kirkpatrick\u0026apos;s Level 3, showing that participants adopted behaviors capable of influencing and shaping an established culture. Our approach incorporated specific learning principles and practical design choices to drive these behavioral outcomes. Drawing from social cognitive learning principles, we structured simulations \u003cem\u003ein situ\u003c/em\u003e to create an authentic, engaging representation of \u0026quot;work as done\u0026quot; rather than \u0026quot;work as imagined\u0026rdquo;. This approach mirrors real-world practices, providing concrete experiences essential to enhance learning, trigger reactions, and shape behaviors. Additionally, we used cognitive psychology\u0026rsquo;s problem-solving framework(10) to lead participants through stages of problem identification, reflection, solution discovery, and testing in their real world clinical environment. By using authentic equipment, and forming realistic teams with their inherent dynamics\u0026mdash;while fostering problem-solving skills and generating practical, context-specific solutions\u0026mdash;we empowered participants to adopt and sustain behaviors that would directly impact their clinical practice.\u003c/p\u003e\n\u003cp\u003eWhile these narratives enriched our understanding of our impact, we lacked comprehensive data to confirm widespread impact and lacked a measure to assess the \u0026ldquo;Results\u0026rdquo; level. By shifting our focus from patient to system improvement, we documented site-specific work against identified issues, confirming discrete episodes of organisational change consistent with Kirkpatrick \u0026lsquo;Results\u0026rsquo;. Importantly we do not pretend these successes are ours alone, but argue they are the result of collective and coordinated action between simulation outreach and local clinicians. Our follow up process documented organisational, equipment and process changes at individual hospitals (Table 2). With 45% of issues fully resolved within three months and an additional 45% in progress, we documented changes on all 4-levels of Kirkpatrick\u0026rsquo;s Model. We also celebrated and propagated regional expertise and innovations across the state. These results highlight the strength of our reporting system, particularly its ability to harness individual clinician\u0026rsquo;s capacity to champion, coordinate and generate organisational changes.\u003c/p\u003e\n\u003col start=\"2\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eAn achievable method of measuring the impact of a simulation program\u003c/strong\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eOur reporting process was facilitated without additional staffing or expenses. This was achievable through the use of familiar software, maximising automation, administrative support and longitudinal relationships with regional sites.\u003c/p\u003e\n\u003cp\u003eWe utilised\u003cem\u003e\u0026nbsp;Office 365\u003c/em\u003e software throughout the process. Staff were familiar with our organisation\u0026rsquo;s licensed software, and the integration of QR codes to link to reporting forms enabled staff to quickly complete data collection on their smartphones during course closure or travel time from courses. Familiar technology avoided adding further barriers to data collection.\u003c/p\u003e\n\u003cp\u003eA combination of software automation and consistent administrative support was essential. Forms and templates were codesigned by our administrative officer, ensuring every process was automated as much as possible. The use of \u003cem\u003eMicrosoft Forms\u003c/em\u003e to auto generate a \u003cem\u003eMicrosoft Word\u003c/em\u003e document decreased report writing times. Our administrative officer created automatic email reminders to prompt site summary completions at agreed upon times. A backup system evolved to ensure report completion, wherein outstanding forms are carbon copied to a consultant and nurse educator who could assist with completion.\u003c/p\u003e\n\u003cp\u003eIn addition to the technical processes described, we believe our new process was successful because it leveraged relationships built through a decade of delivering paediatric education to every Hospital and Health Service in Queensland. While this education provided individual clinicians with essential training, it also fostered our service\u0026rsquo;s relationship with other clinical teams. Outreach education has been described as a relational intervention (11,12), and as such, we regard our courses as vital steps in building psychologically safe relationships with regional educators and clinicians. Had we rolled out a statewide reporting process on paediatric resuscitation at the start of our ten year journey, we believe we would have decimated an early regional courtship. As such we strongly recommend other outreach services build psychological safety and establish relationships before launching into this depth of reporting.\u003c/p\u003e\n\u003cp\u003eWhile we believe success was informed by these deliberate choices (familiar software, maximising automation, administrative support, and longitudinal relationships), the process did have challenges. Report writing was an unappealing new experience for many in our team and we faced difficulties meeting the pre-established contact points of 14 days and 3-months. Follow up was made more difficult due to frequent regional personnel turnover in educational and management roles. This caused interruption in communications and challenges coordinating time with educators amidst clinical duties. These logistic challenges are likely to persist. Some coordinators reported frustration with the additional taskwork, although this decreased with familiarity and time. Additional support processes were arranged for report completion. Two senior clinicians (BS and JA) with confidence writing diplomatic, unambiguous communication took on a \u0026lsquo;reviewer\u0026rsquo; role, to support consistency and redistribute individual workloads. Administrative support, such as monitoring time intervals and creating clear, timely email templates for follow-up, helped to improve these metrics in the latter part of our study period.\u003c/p\u003e\n\u003cp\u003eWe perceived that staff frustration with the process decreased as follow up data began to show real impact. During team meetings, we focused on our primary goals of education delivery and collaboration with local champions, and emphasised flexibility and adaptability in our process. Finding the perfect balance between a personalised site-specific reporting timeline and a rigorous, standardised research reporting timeline remains a challenge. Therefore, we will continue our iterative process improvement cycle and aim to measure and report on the challenges and impact of our efforts prospectively.\u003c/p\u003e\n\u003col start=\"3\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eA synergistic approach to measuring the impact of a simulation program\u003c/strong\u003e\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eOur service sought a process that was synergistic with our mission and educational processes. Keeping our mission, \u0026ldquo;ensuring every child in Queensland has access to optimal resuscitative care\u0026rdquo; forefront ensured we designed a process that supported our work rather than distracting from it.\u003c/p\u003e\n\u003cp\u003eAt the practical level, we designed the summaries to serve multiple purposes. We use our site summaries to understand the impact of our service, but we use the same summaries in statewide meetings on recurrent themes throughout Queensland. In parallel, external staff members use them to advocate for changes in local steering groups, committees and in individual emails to key stakeholders. Several regional educators have stated the process provided them with the external authority needed to push for movement on a disregarded issue.\u003c/p\u003e\n\u003cp\u003eThe reporting process was a catalyst for change itself, providing us with the infrastructure needed to achieve what we had previously only hoped was happening. Our experience mirrors the conclusions of a systematic review of latent safety threats (LSTs) identified through In Situ simulation, which highlight that LSTs are frequently identified but infrequently closed. The authors emphasise that \u0026ldquo;clear and structured processes need to be implemented for the resolution of patient safety threats identified through simulation.\u0026rdquo;(13).\u003c/p\u003e\n\u003cp\u003eThe reporting system also helped us recognise issues that were shared across statewide hospital resuscitation systems, allowing us to advocate at higher levels for broader interventions.\u003c/p\u003e\n\u003cp\u003eWe have recognised that the \u0026lsquo;transformational conversations\u0026rsquo; we were having in our debriefs are more likely to impact the healthcare system when combined with diligent follow up. We believe more simulation services could benefit from deliberately connecting findings from their learning conversations into quality improvement processes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLessons learned and future avenues (limitations/ implications for future research)\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn order to optimise translation of our learning to other services, we have synthesised 5 key \u0026lsquo;lessons learned\u0026rsquo; that we believe other services may find useful.\u003c/p\u003e\n\u003cp\u003e1.\u003cstrong\u003e\u0026nbsp;Education is insufficient, but vital.\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eUntil we changed our strategy, the systemic challenges we uncovered did not improve. However our prior decade of educational efforts were not wasted. Positive participant experiences and longitudinal relationships created the psychological safety needed to have harder conversations about the system. Simultaneously, our educators developed an incredible mental map of the issues faced by individual hospitals region-wide that heavily informed our understanding of statewide resuscitation.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e2.\u003cstrong\u003e\u0026nbsp;Iterative approaches were important\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eWe received feedback from local educators requesting more direct communication with hospital stakeholders to support their efforts. We added additional contact lists to our data collection, carbon copied hospital executives and reached out directly to stakeholders to support participant\u0026rsquo;s initiative when needed.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e3\u003cstrong\u003e. The mission is what matters\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eWhile demonstrating impact on patient outcomes statewide was an alluring dream, it wasn\u0026rsquo;t feasible and could have distracted us from more targeted, useful questions. Keeping our mission at the centre of all design choices, \u0026ldquo;i.e. how does this help sick kids\u0026rdquo;, ensured a streamlined approach with a meaningful outcome.\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eDiligence is key\u003c/strong\u003e\u003c/li\u003e\n \u003cli\u003eIt was not an expensive new mannequin, a technological leap or a masterful debrief that magnified our impact, it was diligence. We postulate that simulation educators spend an extraordinary amount of time reflecting on the quality of their debriefs, but less time reflecting on how to harness those conversations for good, and how to connect that information to the right people. Our fullest potential was unlocked by diligent documentation, deliberate connection to key stakeholders, and gently belligerent follow up.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e5. \u003cstrong\u003eLocal buy-in is essential\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eWhile our PRIME Site Summary provided important data to hospitals, it was local change agents who actioned every response. Sites with passionate nurse educators, pharmacists and doctors were able to utilise the reports to build momentum for change. In one site in particular, we were humbled by the achievements of one nurse educator in closing every issue we found on every course. The reports and communication are catalysts for change, but the real heavy lifting is done by regional clinicians. This potential was hampered in sites with rapid staff turnover, but changes were remarkably dynamic in sites with long term, experienced educators.\u003c/li\u003e\n \u003cli\u003eIt is our honest reflection that there are few things more transformative for a regional hospital than a passionate local clinician armed with unwavering conviction, an external report, and their patient\u0026rsquo;s best interests at heart. As one regional nurse educator put it; \u0026ldquo;This is our community. These are our kids. They deserve the best\u0026rdquo; (L. Moore, personal communication, May 15th 2024).\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWithin this paper we share findings from 12 months of our new site reporting process. We have documented action on identified latent safety threats within all hospitals involved, but acknowledge some latent safety threats would have been closed whether or not we had flagged them. For example, as some sites roll out electronic medical records, more computers in resuscitation spaces are routinely installed, conveniently addressing our concerns about internet access in resuscitation rooms. Regardless of this, we are confident that site summaries played a significant role in many improvement processes, as evidenced by the numerous verbal and written communication channels we\u0026rsquo;ve been involved in since.\u003c/p\u003e\n\u003cp\u003eIdentified latent safety threats are also specific to the curriculum of our course and shaped by the location of the simulation (In Situ versus education rooms), equipment used (training drug infusion pumps versus actual ward equipment) and the perspectives of individual participants and faculty. We argue however that our goal was not to provide an objective measure of performance, but to link valuable data generated in educational settings with organisational improvement systems. We have demonstrated such a system is both feasible and helpful for workplace improvement.\u003c/p\u003e\n\u003cp\u003eWhile we are confident our intervention impacted paediatric resuscitation systems, we have not proved improvement of actual paediatric resuscitations. Some of our recommendations may have unpredicted negative impacts, however we argue this highlights the need for continuous cycles of performance improvement.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eSimulation services seeking to measure impact need processes that are robust, achievable and synergistic with their mission. We have described our simulation outreach model and presented the results of our robust reporting and advocacy process. Our use of software automation, administrative support and longitudinal relationships made the process achievable without additional expense. We kept our processes synergistic with our mission by incorporating quality improvement methodologies into our existing educational courses, and in doing so have documented system problems and demonstrated tangible changes within healthcare systems at both the learner and institutional levels.\u003c/p\u003e \u003cp\u003eWe hope this article provides detailed methods and generalisable principles for other simulation programs seeking to demonstrate concrete outcomes, mitigate latent safety threats and prove their value.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eSTORK: Simulation Training Optimising Resuscitation for Kids, the simulation outreach service for Children\u0026rsquo;s Health Queensland\u003c/p\u003e\n\u003cp\u003ePRIME: Preparing for Retrieval in Medical Emergencies, a course frequently delivered by STORK\u003c/p\u003e\n\u003cp\u003eED: emergency department\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLSTs: latent safety threats\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eFunding:\u003c/h2\u003e\n\u003cp\u003eNo funding was sought or obtained for this study.\u003c/p\u003e\n\u003ch2\u003eConflict of interest:\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eNil to declare\u003c/p\u003e\n\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\n\u003cp\u003eA. SOSH wrote the first draft of the manuscript, collated and analysed the associated data, and created first drafts of the tables and figures. She took a primary role in the project.B.L, L.D, J.A, D.H all reviewed the document and contributed their perspectives, and were all heavily involved with the development and roll out of the initiatives described.B.S. contributed heavily to multiple sections of the manuscript, provided guidance on vision, and created the visual abstract.\u003c/p\u003e\n\u003ch2\u003eAcknowledgement\u003c/h2\u003e\n\u003cp\u003eThe authors thank Tony Carter, Samuel Greer, Tina Haffenden, Jillian O\u0026apos;donnell, Emma Perry and Stefan Pietsch for their contribution to data acquisition, Nikki Harrison for her data management and online form creation, and for Lois Moore for her hard work in local organisational transformation and permission to use her quote.\u003c/p\u003e\n\u003ch2\u003eAvailability of data and materials\u003c/h2\u003e\n\u003cp\u003eThe dataset analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eVarpio L, Sherbino J. Demonstrating causality, bestowing honours, and contributing to the arms race: Threats to the sustainability of HPE research. Med Educ. 2024 Jan;58(1):157\u0026ndash;63. \u003c/li\u003e\n\u003cli\u003eSoong C, Shojania KG. Education as a low-value improvement intervention: often necessary but rarely sufficient. BMJ Qual Saf. 2020 May;29(5):353\u0026ndash;7. \u003c/li\u003e\n\u003cli\u003eSt‐Onge‐St‐Hilaire A, Acworth J, Lawton B, Williams M, Dodson L, Symon B. Paediatric resuscitation in regional Queensland: A simulation informed biopsy of current system challenges. Emerg Med Australas. 2025 Apr;37(2):e70028. \u003c/li\u003e\n\u003cli\u003eFrerejean J, Van Merri\u0026euml;nboer JJG, Condron C, Strauch U, Eppich W. Critical design choices in healthcare simulation education: a 4C/ID perspective on design that leads to transfer. Adv Simul. 2023 Feb 24;8(1):5. \u003c/li\u003e\n\u003cli\u003eBrazil V. Translational simulation: not \u0026lsquo;where?\u0026rsquo; but \u0026lsquo;why?\u0026rsquo; A functional view of in situ simulation. Adv Simul. 2017 Dec;2(1):20. \u003c/li\u003e\n\u003cli\u003eHicks C, Petrosoniak A. The Human Factor. Emerg Med Clin North Am. 2018 Feb;36(1):1\u0026ndash;17. \u003c/li\u003e\n\u003cli\u003eRemick K, Gausche-Hill M, Joseph MM, Brown K, Snow SK, Wright JL, et al. Pediatric Readiness in the Emergency Department. Pediatrics. 2018 Nov 1;142(5):e20182459. \u003c/li\u003e\n\u003cli\u003eHollnagel E, Wears RL, Braithwaite J. From Safety-I to Safety-II: A White Paper [Internet]. University of Southern Denmark; 2015 [cited 2025 Mar 9]. Available from: http://rgdoi.net/10.13140/RG.2.1.4051.5282\u003c/li\u003e\n\u003cli\u003eFalletta S. Evaluating Training Programs: The Four Levels Donald L. Kirkpatrick, Berrett-Koehler Publishers, San Francisco, CA, 1996, 229 pp. Am J Eval. 1998;19(2):259\u0026ndash;61. \u003c/li\u003e\n\u003cli\u003eHardin LE. Problem-Solving Concepts and Theories. J Vet Med Educ. 2003 Sep;30(3):226\u0026ndash;9. \u003c/li\u003e\n\u003cli\u003eMattick K, Noble C. Education and educational interventions: moving beyond information provision. BMJ Qual Saf. 2024 Jan;33(1):10\u0026ndash;2. \u003c/li\u003e\n\u003cli\u003eLuetsch K, Wong G, Rowett D. A realist synthesis of educational outreach visiting and integrated academic detailing to influence prescribing in ambulatory care: why relationships and dialogue matter. BMJ Qual Saf. 2024 Jan;33(1):43\u0026ndash;54. \u003c/li\u003e\n\u003cli\u003eGrace MA, O\u0026rsquo;Malley R. Using In Situ Simulation to Identify Latent Safety Threats in Emergency Medicine: A Systematic Review. Simul Healthc J Soc Simul Healthc. 2024 Aug;19(4):243\u0026ndash;53. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"advances-in-simulation","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"asim","sideBox":"Learn more about [Advances in Simulation](http://advancesinsimulation.biomedcentral.com/)","snPcode":"41077","submissionUrl":"https://submission.springernature.com/new-submission/41077/3","title":"Advances in Simulation","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Simulation, Outreach, Value, Quality \u0026 Safety, Improvement","lastPublishedDoi":"10.21203/rs.3.rs-6230704/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6230704/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eIntro\u003c/strong\u003e: Healthcare simulation programs measuring their value risk wasting resources in futile attempts to prove they impact patient outcomes. Simulation is one of many strategies used to enhance healthcare systems, and proving specific correlation with simulation will prove impossible in many circumstances. To maintain accountability but ensure feasibility, we argue simulation services need measurement processes that are robust, achievable and synergistic with their mission. In 2023, the STORK service in Queensland, Australia began measuring the impact of simulation on \u003cem\u003esystems\u003c/em\u003e rather than \u003cem\u003epatients\u003c/em\u003e, to define the extent to which their educational programs could impact system improvement.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e: Translational simulation methodologies and quality improvement measures were embedded in an established educational course. We used simulation activities to diagnose environmental and system-level problems in participants’ workplaces throughout Queensland. Courses included dedicated time to discuss site-specific actionable solutions with participants, and identified local champions to implement quality improvement changes. By designing a novel electronic reporting process (Optimus PRIME Course Summary), we documented issues and solutions identified in regional healthcare facilities and ensured they reached key stakeholders. We audited our ability to improve these systems through follow up data collection via phone and emails with local educators across the state.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e: From 40 courses delivered across 37 facilities, 242 issues were identified, primarily related to drug safety and equipment management. At follow-up, 45.5% of the issues were resolved, with 44.6% still being addressed. Recommended resources were successfully implemented in 64% of sites.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e: This process demonstrates that focusing on system-level changes can significantly enhance healthcare systems. The reporting framework provided a robust, achievable and synergistic method to measure simulation impact and influence change. Additionally, we share key lessons learned from the process to guide other simulation services in improving their own measurement strategies.\u003c/p\u003e","manuscriptTitle":"Outreach Simulation for System Improvement: a Novel Advocacy and Reporting Process","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-04 11:27:26","doi":"10.21203/rs.3.rs-6230704/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-06-04T08:29:23+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-16T01:47:40+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-04-13T21:19:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"82294455202033777048066854818409433649","date":"2025-04-12T10:52:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"228277997251176406404315581252010522967","date":"2025-03-31T14:28:30+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-26T02:27:35+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-03-18T06:41:27+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-03-18T06:35:56+00:00","index":"","fulltext":""},{"type":"submitted","content":"Advances in Simulation","date":"2025-03-15T05:52:03+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"advances-in-simulation","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"asim","sideBox":"Learn more about [Advances in Simulation](http://advancesinsimulation.biomedcentral.com/)","snPcode":"41077","submissionUrl":"https://submission.springernature.com/new-submission/41077/3","title":"Advances in Simulation","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"fb4bc1ed-ee36-420b-973a-ec979629216b","owner":[],"postedDate":"April 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-09-08T15:59:47+00:00","versionOfRecord":{"articleIdentity":"rs-6230704","link":"https://doi.org/10.1186/s41077-025-00372-0","journal":{"identity":"advances-in-simulation","isVorOnly":false,"title":"Advances in Simulation"},"publishedOn":"2025-09-01 15:57:20","publishedOnDateReadable":"September 1st, 2025"},"versionCreatedAt":"2025-04-04 11:27:26","video":"","vorDoi":"10.1186/s41077-025-00372-0","vorDoiUrl":"https://doi.org/10.1186/s41077-025-00372-0","workflowStages":[]},"version":"v1","identity":"rs-6230704","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6230704","identity":"rs-6230704","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}