Implementing a Rapid Improvement Event with anonymised individual performance reporting on benign hysterectomy care: a retrospective comparative analysis.

Hysterectomy is one of the most common gynaecological surgeries performed for a variety of benign and malignant conditions, including fibroids, adenomyosis, ovarian cysts, endometrial hyperplasia, abnormal uterine bleeding and cervical intraepithelial neoplasia. 1 Like any surgical intervention, a hysterectomy is associated with various potential complications, including haemorrhage, surgical site infections, visceral injuries and even mortality. These complications can lead to prolonged postoperative recovery periods and readmissions or necessitate additional surgeries. 2 However, even among various clinicians, clinical outcomes and quality of care are likely to vary, and clinicians may not accurately evaluate their own performance. 3 The lack of timely, relevant and adequate data 4 and poorly implemented feedback mechanisms 5 significantly limit a physician’s evaluation of his own performance. Thus, performance data help address limitations in self-assessment. While providing clinicians with performance data is beneficial, it is not sufficient on its own. Feedback is crucial for continuous professional development and quality improvement in healthcare. It provides clinicians with performance data relative to peers and standards, highlighting areas for improvement and encouraging adherence to best practices, leading to more consistent and higher quality care. 5 6 Furthermore, clinicians must actively engage with these data as part of a feedback loop for effective improvement. Well-designed feedback, especially when it highlights actionable areas for improvement, prompts clinicians to take steps to enhance the quality of care. 7 A systematic review of audit and feedback interventions has shown that these methods can lead to small but meaningful improvements in clinical practice. 5 In particular, individualised, non-punitive feedback has been shown to improve adherence to clinical guidelines within healthcare facilities. For instance, a study by Kiefe et al in 2001 introduced the ‘achievable benchmark method’ of performance feedback, where benchmarks are based on top performers within a peer group. Physicians receiving this type of feedback demonstrated significantly better quality of care than those who received traditional feedback methods. 8 9 Despite these positive findings, there remains a scarcity of studies examining the impact of individual clinician feedback on standardised surgery outcomes. This gap highlights the need for further research to explore how personalised feedback can drive improvements in surgical outcomes. Michael Porter and Elizabeth Teisberg first denoted the concept of Value-Based Healthcare (VBHC) in their 2006 book, Redefining Health Care: Creating Value-Based Competition on Results . VBHC is a comprehensive framework that aims to revolutionise healthcare delivery by focusing on patient outcomes and cost efficiency. Having clear outcomes data and a robust feedback loop would be vital in improving outcomes and effectively implementing VBHC. In 2017, the Ministry of Health Singapore launched the Value-Driven Care (VDC) programme, with principles similar to those of VBHC. The VDC initiative for hysterectomy at tertiary hospitals in Singapore began in 2021. This involves tracking multiple outcome indicators, which contributes to a composite score, the Clinical Quality Index (CQI). The CQI is calculated using an ‘all-or-none’ methodology to measure the attainment of ‘most optimal clinical care’. There are six clinical indicators involved in the CQI measurement: (1) postoperative length of stay (LOS) of 4 days or less; (2) no 30-day blood transfusions; (3) no complications within 30 days; (4) no return to the operating theatre within 30 days; (5) no readmissions through the emergency department (ED) within 30 days; and (6) no inpatient mortality. Secondary outcomes assessed these individual indicators respectively. Aggregated indicators and CQI performance are disseminated quarterly by the hospital’s Office of Value-Based Healthcare to the clinical champion and head of the department, facilitated by Tableau 10 dashboards (interactive, visual displays created using business intelligence and data visualisation software, allowing users to visualise and analyse large sets of data in a way that is easy to interpret, offering a range of charts, graphs and other visual elements). Despite this, improvements in CQI were suboptimal. To address this, an active feedback loop was designed, using confidential Individual Clinical Reports (ICRs) for individual clinicians and incorporating active discussion of aggregated performance dashboards during department meetings. The Agency for Healthcare Research and Quality guide for confidential physician feedback reports was referenced during the development of this feedback reporting system. 11 This study aims to examine the impact of an active feedback loop facilitated by confidential ICRs and active discussion of aggregated results including CQI scores, rates of blood transfusion and hospital LOS for patients undergoing hysterectomy for benign gynaecological conditions.

This is a retrospective analysis of a Rapid Improvement Event (RIE) programme conducted at a tertiary hospital in Singapore over an interval of 2.5 years between 2021 and 2023. The hospital has 1785 beds, including up to 60 beds in the Department of Obstetrics & Gynaecology (O&G). On average, approximately 245 hysterectomies for benign conditions and 185 hysterectomies for malignant conditions are performed annually. A total of 21 surgeons participated in this intervention. A single-centred, quasiexperimental study was performed for cases that underwent hysterectomy between January 2021 and June 2023 at the O&G department of a tertiary hospital in Singapore. The study involved a secondary analysis of data extracted from clinical records. Patients who underwent hysterectomy from January 2021 to June 2023 for non-malignant indications were included based on specific inclusion and exclusion criteria. Eligible participants had one or more procedures listed in the relevant Table of Surgical Procedures ( online supplemental appendix table 1 ) and a principal or secondary procedure from the Australian Classification of Health Interventions ( online supplemental appendix table 2 ). Patients were excluded if they had absconded or were discharged against medical advice, were under 18 years of age, were discharged from the ED or an equivalent unit, or were emergency inpatient admissions, except in cases where day surgery cases were converted to inpatient status. Clinical records were obtained from our institution’s clinical information system (Sunrise Clinical Manager (SCM), Allscripts, Illinois, USA) and stored in the enterprise data repository and analytics system (SingHealth-IHiS Electronic Health Intelligence System). 12 Information extracted from SCM included a patient’s medical history, American Society of Anesthesiologists (ASA) classification, operative details, number of blood transfusions, complication rates, LOS, return to the operating theatre and readmission to the hospital through the ED within 30 days. Mortality data in our clinical information system are synchronised with the National Electronic Health Records, ensuring near-complete capture of all-cause mortality. All the surgeons and clinicians who perform hysterectomy for benign gynaecological conditions are included in this initiative. The primary objective of this study is to examine the impact of the active feedback loop, which consists of ICRs and aggregated discussions, on the CQI score. The secondary objective is to examine the impact between the feedback loop and the individual indicators making up CQI. A workgroup was formed to discuss strategies for effectively implementing the RIE approach in July 2022. The team included the gynaecology clinician lead, VDC project managers and the data analysts who collaborated to design the automated workflow for sending clinicians’ personal performances. Based on the existing Tableau dashboard, the RIE workgroup decided to develop anonymised ICRs from the dashboard of aggregated data using robotic process automation (RPA). These ICRs provide an overview of each clinician’s CQI performance, along with a comparison to other clinicians, who are anonymised ( online supplemental appendix figure 1 ). The reports will be exported as PDFs and sent to each clinician via email quarterly. They will not be shared beyond the individual or used by the head of department (HOD) for performance management. This protection of confidentiality of individual performance data serves to provide a safe environment for clinicians to self-evaluate and benchmark against the rest of the department’s performance. The initial ICR for the first quarter of 2022 (2022 Q1) performance was released in August 2022 as quarterly reports are typically subject to a two-quarter delay before being published. Concurrently, the overall department outcomes are presented by the HOD at quarterly department meetings to reinforce VDC targets and highlight areas needing improvement as a department. During these sessions, performance gaps are identified, and proposals for addressing specific areas are explored. Regular sharing of the department’s overall performance fosters transparency and enables individual surgeons to better understand their performance relative to the aggregate results. A quasi-experimental design was employed in this analysis, with data divided into two separate pre-implementation periods (pretest 1: 2021 Q1–2021 Q3; pretest 2: 2021 Q4–2022 Q2), intervention and washout period (2022 Q3), and post implementation period (post-test: 2022 Q4–2023 Q2). Pretest 1 and pretest 2 were initially compared to ensure there were no pre-existing trends before evaluating the intervention’s effect, establishing a stable baseline for subsequent analysis. Data from pretest 2 and post-test were then compared to determine any significant improvements in the CQI and individual clinical indicators following the interventions. Mann-Whitney U tests were performed to evaluate differences in characteristics that are continuous variables, including age, body mass index (BMI) and operation length. Fisher’s exact tests were used to examine differences in categorical variables, such as smoking history, comorbidities, surgical group and ASA classification. Logistic regression analysis was performed to evaluate the effects of ICR implementation on overall CQI and individual indicators, comparing pretest 1 versus pretest 2, and pretest 2 versus post-test. To account for heterogeneity in casemix, the model included age, BMI, smoking history, length of operation, type of surgery (laparotomy vs minimally invasive surgery (MIS)) and ASA classification as covariates. Mann-Whitney U test and Fisher’s exact tests were performed using GraphPad Prism V.8 (GraphPad Software, San Diego, California, USA). The logistic regression analysis was conducted using SPSS V.25 (SPSS). Patients were not involved as this is a health service study involving a retrospective audit of aggregated data from hospital medical records. In the future, patient involvement can be considered.

Table 1 presents a detailed summary of patient and surgical characteristics across different time points in the study. A total of 159, 130 and 147 patients were included in the pretest 1, pretest 2 and post-test groups, respectively. The median age of patients remained consistent across all groups, ranging from 49 years to 50 years, with no statistically significant differences. However, BMI in the post-test group (p=0.006) was significantly lower in the post-test group, suggesting a potential shift in patient characteristics over time. Smoking prevalence remained low, with similar proportions across all groups (4.1–6.3%), showing no significant variation. ASA, American Society of Anesthesiologists; BMI, body mass index; MIS, minimally invasive surgery. The overall prevalence of comorbidities was stable (27.9–30.8%), though hypertension was less prevalent in the post-test group. The prevalence of non-uterine malignancy significantly varied across groups (p=0.029), indicating a shift in patient casemix, while other comorbidities, including diabetes and hyperlipidaemia, remained relatively unchanged. Regarding surgical approach, laparotomy was increasingly performed across groups (51.6%, 60.0% and 64.6%), while MIS decreased over time. However, these changes were not statistically significant. The distribution of ASA classification remained similar, with the majority of patients classified as ASA I or II, and no significant differences in the proportion of higher risk patients (ASA III and IV) across groups. Lastly, median operation length was consistent across all groups (115–120 min) with no statistically significant differences. Overall, the baseline characteristics of the study groups were comparable, with no major differences in demographics, surgical type or ASA classification. However, changes in BMI and prevalence of non-uterine malignancy were significant, potentially influencing postoperative outcomes. From table 2 , it can be seen that there was no significant difference in CQI between the pretest 1 and pretest 2 groups (p=0.36), suggesting that the CQI remained stable before the implementation of the intervention. The most noteworthy finding is the statistically significant improvement in the CQI observed between the pretest 2 group and the post-test group. Specifically, the CQI improved from 81.5% (106 of 130 cases) in the pretest 2 group to 90.5% (133 of 147 cases) in the post-test group, with a p value of 0.01 ( table 2 ). This indicates a meaningful improvement in the quality of care following the intervention. This stability between the pretest groups, coupled with the significant improvement observed postintervention, strongly suggests that the implementation of the CQI measures led to the observed improvements in clinical quality. Thus, the data indicate that the intervention was effective in enhancing the overall quality of patient care, as reflected by the CQI. p-value less than 0.05 (p < 0.05) indicates statistical significance. CQI, Clinical Quality Index; LOS, length of stay; OT, operating theatre. Figure 1 depicts a run chart of CQI over time, segmented into three phases: preintervention, intervention and washout, and postintervention. The CQI performance preintervention is relatively stable with some mild variability, hovering around the average of 84.9% to 81.5%. There is a clear improvement in CQI performance postintervention, with the average increasing to 90.5%, indicating that the intervention was successful in enhancing performance outcomes. A post hoc power analysis was conducted to indicate that the sample size was sufficient to identify a meaningful difference in CQI improvement after adjusting for multiple variables. From table 2 , it is evident that there was no statistically significant improvement in the postoperative LOS when comparing both pretest 2 and the post-test groups (median postoperative LOS: pretest 2=2.5 (95% CI 2 to 3), post-test=2 (95% CI 2 to 3), p=0.08), and the pretest 1 and pretest 2 groups (pretest 1=3 (95% CI 2 to 3), p=0.22). This indicates that the introduction of ICRs did not lead to a significant reduction in LOS during the study period. However, it is important to note that there is a trend towards significance over time, suggesting that the impact of ICRs on LOS may become more apparent with prolonged observation and further refinement of the feedback process. This trend highlights the potential for gradual improvements in LOS as clinicians continue to adapt to and use the feedback provided by ICRs, reinforcing the importance of ongoing monitoring and evaluation in quality improvement initiatives. Online supplemental appendix figure 2 depicts a run chart of LOS against time. The LOS performance remains relatively stable, fluctuating around an average of 93.7% to 90.8% preintervention. The LOS performance improved postintervention, reaching an average of 95.2%, suggesting that the intervention had a positive impact on reducing the LOS. Most patients did not require a blood transfusion during surgery, with the highest proportion in the post-test group (96.6%). A statistically significant increase in no blood transfusion rate was seen between pretest 2 and the post-test group (p=0.046) ( online supplemental appendix figure 3 ). The majority of patients across all groups did not experience 30-day postoperative complications. The rates were similar across groups, with no statistically significant differences. Over 99% of patients in all groups did not need to return to the operating theatre within 30 days of their initial surgery. No statistically significant differences were found between groups. Most patients did not require readmission within 30 days post-discharge, with readmission rates remaining low across all groups: pretest 1 (4 out of 159 cases, 2.5%), pretest 2 (5 out of 130 cases, 3.8%) and post-test (1 out of 147 cases, 0.7%). The reasons for readmission included postprocedural haemorrhage/haematoma, wound dehiscence, abnormal vaginal bleeding and intestinal obstruction. While no statistically significant differences were observed between the groups, there was a trend towards significance in the post-test group. The comparison between pretest 1 and pretest 2 resulted in a p value of 0.89, whereas the comparison between pretest 2 and post-test yielded a p value of 0.12. Furthermore, we observed a decrease in ED visits without admission within 30 days post-discharge following the intervention. The proportion of patients requiring ED visits was low across all groups, with pretest 1 (3 out of 159 cases, 1.9%), pretest 2 (7 out of 130 cases, 5.4%) and post-test (1 out of 147 cases, 0.7%). While the differences were not statistically significant, the decline in ED visits after the intervention suggests a potential improvement in postoperative recovery and care coordination (pretest 1 vs pretest 2: p=0.48; pretest 2 vs post-test: p=0.28). Detailed breakdown of the reasons for ED visits is provided in online supplemental appendix table 3 . No inpatient deaths were reported in any group, and there was a 100% survival rate across all study phases. No statistically significant differences were observed between groups ( table 2 ). All in all, there is improvement in all individual indicators making up CQI, with a statistically significant improvement in overall CQI and no blood transfusion rate.

This study demonstrates that an active feedback loop facilitated by ICRs and active discussion of results can significantly improve CQI and overall care quality for patients undergoing hysterectomy. By providing detailed performance data and fostering transparency, ICRs help clinicians align with best practices and improve patient outcomes. The findings of this study align with those of other studies, which show that providing clinicians with feedback through dashboards in a multidisciplinary setting, along with transparency about how an individual clinician’s performance compares with that of others, influences their intention to improve their practice based on performance data. A study in Dutch intensive care units used dashboards providing real-time feedback on performance across several indicators. Clinicians who received detailed feedback, including comparisons with peer performance, were more likely to set higher performance targets and show an intention to improve areas where their performance was lacking. 13 Another study showed that primary care practices that regularly used performance data to guide quality improvement efforts saw significant improvements in clinical outcomes, particularly in managing cardiovascular disease risk factors. 14 This approach has the potential to be applied to other standardised surgeries for similar improvements in care quality. The significant improvement in overall CQI and the reduction in blood transfusion rates suggest that the intervention had a positive impact on overall quality of care. However, while other individual indicators showed positive trends, they did not reach statistical significance. This is likely due to the high baseline performance of many assessed indicators before the intervention, particularly complication rates, return to the operating theatre, readmission rates and inpatient mortality, which exceeded 95% in the preintervention period. Future studies may consider exploring alternative indicators that are more sensitive to detecting changes in clinical performance over time. Although ICRs have been shown to improve CQI in this study, we note a slow and gradual improvement. This can be attributed to several potential factors. First, changing ingrained behaviours and the established culture within clinical practice is a gradual process. Clinicians may need time to adapt to new practices, especially when these changes require altering long-standing practices. Second, resistance to change is a common challenge in any field, and healthcare is no exception. Clinicians may resist new practices due to a lack of familiarity, comfort with existing methods or scepticism about the benefits of the changes. This resistance can slow the adoption of new quality improvement practices. Third, clinicians may need more reassurance that changes in practice are not punitive but are intended to enhance patient care and improve outcomes. Clear communication that emphasises the positive impact of these changes on patient outcomes can help alleviate fears and encourage participation. Fourth, more ongoing education is essential for clinicians to understand the rationale behind new practices and the evidence supporting them. Regular training sessions, workshops and educational materials can equip clinicians with the knowledge they need to feel confident in implementing new practices. The sustainability of this intervention is a key consideration. This initiative remains ongoing in our department, with parameters such as CQIs being monitored every 3 months. Reports are regularly generated and discussed during department meetings to address any new potential issues. This continuous tracking ensures that the intervention remains effective and relevant, allowing for timely adjustments based on real-time data. By embedding this process into routine clinical practice, we aim to sustain long-term improvements in care quality and clinician engagement. In this study, clinicians received individualised feedback on their performance, benchmarked against anonymised peer data. Although such comparative feedback can theoretically create a sense of competition, our approach deliberately focused on collaboration rather than competition by anonymising individual surgeon results. Initial concerns that clinicians might feel discomfort or pressure from direct peer comparisons were mitigated by anonymising individual surgeon data, thus preventing any undue scrutiny or punitive interpretations. This method allowed clinicians to meaningfully engage in self-assessment, promoting motivation and collaborative efforts towards continuous quality improvement without fostering unnecessary competition or stress. The second challenge involved the design of the ICRs. Ensuring that the data feeding into ICRs is accurate, timely and complete is crucial but can be difficult. Incomplete or outdated data, or data inaccurately linked to clinicians, can undermine trust and engagement. Clinicians were also concerned about the sensitivity of the performance data, fearing punitive consequences or public exposure, which further discouraged participation. To be effective, ICRs must show clinicians where they stand and offer actionable insights for improvement, such as access to educational resources, mentorship or targeted training programmes. Continuous improvement of ICRs requires gathering and acting on clinician feedback, making the design process iterative and essential for long-term success. Finally, logistical challenges arose that required the help of RPA. Clinician performance data are typically stored across multiple systems, making manual extraction of data time consuming and error prone. RPA bots were used to automatically extract and consolidate data from these disparate systems, significantly reducing manual input and ensuring timely, accurate data collection. Additionally, RPA handles large volumes of clinical data, which are updated frequently. Automating the data processing tasks minimised delays, ensured ICRs were always up to date and prevented human errors that could compromise the accuracy and reliability of the reports. There are a few limitations to this study. First, it was a single-centre study, which may increase the risk of selection bias. Second, the relatively small cohort population and short time frame employed in the analysis limit the statistical power of the study. Third, qualitative data on how ICRs improved the quality of care do not imply causality. Lastly, as this study is conducted at a single centre and limited to patients undergoing hysterectomy for benign gynaecological conditions, the findings of this study cannot be generalised to patients undergoing other standard surgical procedures in other institutions. This study can stand to benefit from benchmarking the findings with other institutions. However, this is limited by the lack of such data and different patient casemix.

An active feedback loop involving ICRs in a multidisciplinary team setting is a critical component of the broader spectrum of clinical quality improvement initiatives designed to enhance patient care and safety. In the case of hysterectomy for benign gynaecological conditions, an active feedback loop with a multidisciplinary approach—including the creation of ICRs for individual clinicians and the active discussion of aggregated performance dashboards during department meetings—has been shown to improve CQI performance. This approach could be effectively applied to other standardised surgeries, offering similar potential improvements in care quality.