The utility of website-based quality improvement tools for health professionals: a systematic review.

This review adheres to the Preferred Reporting Items for Systematic review and meta-Analysis (PRISMA) statement [ 10 ] and prospectively registered with PROSPERO no. CRD42023451346. The completed PRISMA checklist is provided in supplementary file 1 . Institutional ethics board approval was not required for this study since it was a systematic review and did not involve human subjects as research participants. Papers were included if they were peer reviewed, written in English, available in full text, based in a health setting and reported the development or evaluation of a website-based QI tool used by health professionals. In this study, website-based tools were defined as resources such as toolkits that users can access and use autonomously through a dedicated website. Studies that examined tools on other platforms, e.g. mobile applications (tools specifically designed as a mobile application, as these are different to websites that are also mobile-friendly) were excluded. This exclusion was due to the distinct nature of mobile applications which typically depend on the hardware and operating systems of mobile devices and require more platform-specific programming, potentially affecting the consistency and generalizability of results [ 11 ]. A computerized search was conducted in December 2022 and repeated in January 2024 using PubMed, MEDLINE, ScienceDirect, Wiley, Scopus, ProQuest, Education Resource Complete, and A+ Education. Published research in health information and technology emerged in 2008 but began increasing from 2012 [ 12 ]. Therefore, the databases were searched from January 2012 to January 2024. Search terms used were quality improvement tool OR quality tool AND web-based OR website-based. The complete search strategy is outlined in Supplementary File 2 . Reference lists were also screened. Identified articles were uploaded and duplicates removed in Covidence [ 13 ]. Titles and abstracts were reviewed by one author (G.T.). All potentially relevant full-text articles were independently assessed by two authors (among G.T., M.H., and J.N.). Any differences were discussed and resolved between reviewers. Data extracted by one author (G.T.) via Covidence included title, author, location, date, discipline area, study design, and study population. Two authors (G.T. and M.H.) independently extracted data on tool development process, QI mechanism, tool features/resources, and outcome measures and assessments. Differences were discussed and resolved between reviewers. Tool development data specifically examined four key steps—research/literature review, use of theoretical framework to guide development, co-developing with stakeholders, and field testing. These steps were based on the UK Medical Research Council framework for developing and evaluating complex interventions [ 14 ]. All data were entered into Covidence and narratively synthesized on tool development processes, elements of an effective QI tool, and outcome measures and assessments. Studies could contribute information on one or more of these aspects. Thematic analysis using NVivo [ 15 ] followed a six-phase process [ 16 ]. All papers were independently coded by two authors and cross-checked for any discrepancies. A meta-analysis was not possible as there was a high degree of heterogeneity. The Mixed Methods Appraisal Tool version 2018 [ 17 ] was used. The risk of bias was assessed independently by two authors (G.T. and M.H.). Any differences were discussed and resolved between the authors. The GRADE framework [ 18 ] was used to assess the overall body of evidence.

A total of 5308 articles were screened and 18 studies met the inclusion criteria ( Fig. 1 ). Study flow diagram of search results and the selection process. Table 1 summarizes the study characteristics. Studies that only described tool development included qualitative studies ( n  = 3) and a quantitative descriptive study ( n  = 1). Studies that reported tool evaluation included qualitative studies ( n  = 6), quantitative descriptive studies ( n  = 2), feasibility studies ( n  = 5), and a non-randomized experimental study ( n  = 1). No studies were randomized controlled trials. Characteristics of included studies. Four studies only described the tool development; therefore, outcome measures were not reported [ 19–22 ]. The outcomes of the remaining 14 studies are summarized in Table 2 . All 14 studies reported positive outcomes. Outcome measures varied across studies as they were dependent on the tool’s purpose. The various purposes of tools included improving clinical practice and compliance ( n  = 5), enhancing usability of traditional, paper-based or manual methods ( n  = 4), and supporting quality assurance and reflective practice ( n  = 5). Summary of tool outcome measures and assessment methods. Risk of bias results are outlined in Supplementary File 3 . Most studies ( n  = 16) had clear research objectives and appropriate methods. Four studies described tool development only and lacked formal data analyses. While most qualitative studies justified their approach, three studies lacked detail regarding the methodology. Assessing some quantitative studies were difficult due to lack of detailed methodology, sampling strategy, or measurements. There were three main themes that emerged: tool development processes, QI mechanisms, and barriers and facilitators to tool usage. Overall synthesis quality, as per the GRADE framework [ 18 ], was low due to the incorporation of qualitative studies. Eleven studies described the tool development process. Four studies undertook research or literature review prior to developing the tool, one study utilzsed a theoretical framework to guide tool development, nine studies co-developed the tool or consulted with stakeholders in the development process and four studies conducted field testing. The most adopted process was co-developing with stakeholders. The approaches varied across studies, reflecting different levels of stakeholder engagement and adoption of co-design methodologies. For instance, two studies formed specialized working groups, emphasizing collaboration among healthcare practitioners with expertise in the relevant fields [ 19 , 21 ]. Another study involved cyclical feedback from various stakeholders, including healthcare associations and quality improvement bodies, reflecting a broader engagement strategy [ 23 ]. Two studies demonstrated collaboration between professional societies and advisory groups to develop tools tailored to specific clinical domains [ 22 , 24 ]. The least commonly adopted process was utilizing a theoretical framework ( n  = 1). The development of the ‘Cancer Care Quality Improvement Toolkit’ [ 25 ] was guided by Roger’s Diffusion of Innovation Model [ 26 ], and analyses were guided by the Consolidated Framework for Implementation Research [ 27 ]. Table 3 summarizes the primary QI process/mechanisms that the tools utilized. These included digitalizing current QI processes (transforming traditional, paper-based, or manual methods into digital formats) ( n  = 7), identifying gaps in practice ( n  = 6), professional development ( n  = 3), and using clinical governance and organizational management as part of QI ( n  = 2). Some studies described the benefits of these processes, for example, digitalization was found to be feasible and preferred over paper-based methods [ 28 , 29 ]. These processes supported healthcare professionals by enhancing accuracy and efficiency in clinical tasks, improving adherence to guidelines, facilitating reflective practice, and providing tailored feedback for continuous quality improvement. For example, the EQUSUM tool [ 28 ] for endometriosis surgery significantly improved classification accuracy and usability compared to paper methods. Website-based tools grouped by primary QI process/mechanism. There were a variety of QI features. The most common were education/training resources ( n  = 7), assisting the user to assess current practices against standards or recommendations ( n  = 6), and recording activity or performance ( n  = 6). For example, the ‘Safety Profile Assessment’ [ 24 ] helped assess compliance with safety/quality indicators in radiation therapy using a 5-point Likert scale. Other features reported were an automated calculation/scoring system ( n  = 4), such as the ‘Paediatric Endoscopy Global Rating Scale’ [ 22 ] where each measure was assigned a level from D to A and a score was generated for each standard. In addition, action plans or recommendations for QI ( n  = 3), downloadable forms ( n  = 4), self-assessment of current practices ( n  = 2), and platforms for sharing data or information ( n  = 4) were reported. Some studies elaborated on how the tool’s features were advantageous in QI and resulted in better clinical outcomes, increased professional satisfaction, and streamlined workflow. The ‘Web-Based Tool to Report Adverse Drug Reactions’ [ 30 ] allowed for recording of adverse events with all the necessary information for submission to the Australian Therapeutic Goods Administration, in line with professional requirements. The ‘self-audit of methadone treatment’ [ 31 ] tool could analyse the inputted data and provide a comparison to the expected standards. Thirteen studies reported barriers and facilitators to tool usage ( Table 4 ). There were three main factors contributing to the facilitators in tool usage. Analysis of facilitators and barriers to tool usage. Usefulness to practice Alignment with professional standards/requirements Easy to use User-friendly Enhances current practices Allows for shared decision-making Increasing engagement with stakeholders Using the tool took up too much time or effort Resources were too long Information not specific to practice Technical concerns with implementation Content is difficult to understand Technical difficulties for the user Organization barriers to adopting new tool Lack of organization support required for implementation Relevance to practice: the tool was considered useful and aligned with professional standards ( n  = 5). For example, General Practice staffs who used the ‘self-audit of methadone treatment’ [ 31 ] found that the tool was helpful in reflection of their clinical practice and served as a reminder of the importance of standards. The staff that used the ‘Primary Care Practice Improvement Tool (PC-PIT)’ [ 23 ] thought that the tool was relevant to everyday practice work and planning, particularly as the tool addressed clinical governance. Accessibility: the tool was user-friendly and enhanced current practices through a web-based interactive platform. For example, endometriosis experts who utilized the EQUSUM tool [ 28 ] found that the tool was easy to use and there were visual advantages of using anatomical pictures for classification. Facilitating multidisciplinary action: the tool allowed for shared decision-making and increased engagement with stakeholders. For example, one of the identified facilitators of the ‘PC-PIT’ tool [ 23 ] was that it allowed all staff to be involved in the identification of areas for improvement. The ‘Paediatric Endoscopy Global Rating Scale’ [ 22 ] found that the tool increased engagement with surgical colleagues on delivery of endoscopy services. There were four main factors contributing to the barriers in tool usage. Time consuming: the tool took users too long to use. For example, some users found the ‘Residency Performance Index’ [ 20 ] tool time-consuming to use and the ‘self-audit of methadone treatment’ [ 31 ] tool lengthy and protracted. Irrelevant to practice: the tool did not provide information specific to practice, or there were concerns with implementation in the real-life setting. The ‘PC-PIT’ tool [ 23 ] covered areas that some users thought may be outside clinical management processes. The ‘PROP’ tool [ 32 ] study identified a technical barrier where the tool was unable to be integrated into electronic health records. Difficult to use: the information offered in the tool was difficult to understand, or technical difficulties were experienced with using the tool. The ‘PC-PIT’ tool [ 23 ] contained elements that were difficult to understand for some users. The ‘self-audit of methadone treatment’ [ 31 ] had poor clarity of questions for some, and ‘Quality Innovation Networks’ [ 33 ] was not considered easy to navigate or use by some. Lack of organization engagement: there were organization barriers that prevented implementation of the tool. The ‘Cancer Care Quality Improvement Toolkit’ [ 25 ] study identified that there was some reluctance to change and the tool required approvals for implementation that prevented wider use. The ‘Safety Profile Assessment’ tool [ 24 ] study found that resource constraints and clinical pressures prevented tool usage.

The findings emphasize the importance of co-development with healthcare professionals, practical usability, and addressing barriers to enhance engagement and effectiveness of QI digital tools. There is a lack of randomized controlled trials on the efficacy of these tools. Future work should address this knowledge gap.

The findings highlight that identifying gaps in practice, digitalizing existing processes and contributing to professional development were key mechanisms that tools adopted. These tools were associated with reported enhancement of efficiency in clinical tasks, improvement in adherence to guidelines, facilitation of reflective practice, and provision of tailored feedback for continuous quality improvement. The most common tool features were education/training resources and the ability to assist the user in assessing current practices, which supported healthcare professionals in achieving better clinical outcomes, increased professional satisfaction and streamlined workflow in various healthcare settings. Reported facilitators to tool usage included relevance to practice, accessibility, and facilitating multidisciplinary action, making these tools practical and time-efficient for healthcare settings. Barriers reported included being time-consuming, irrelevant to practice, difficult to use, and lack of organizational engagement, highlighting areas for improvement. The co-design approaches varied across studies, reflecting different levels of stakeholder engagement and adoption of co-design methodologies. This study adhered to the PRISMA statement and followed the registered study protocol in PROSPERO. This was the first systematic review to summarize how website-based tools can support health professionals with quality improvement. However, there are several limitations. Inclusion of qualitative studies prevented the ability to perform data analyses and overall quality for syntheses was low. Heterogeneous study designs, outcomes, and methodologies limited generalizability of findings. Some studies lacked the details regarding methodology, including justification for chosen designs. Narrative reviews may have limitations in terms of objectivity. Many tools in the current review helped the user to identify gaps in practice and this was suggested to be useful in QI, with ‘relevance to practice’ identified as a facilitator to tool usage. Identifying gaps in practice is a common feature across many QI projects in professional settings. For example, an auditing tool for surgical QI was designed to assess compliance with infection-related process measures and identify gaps in measure implementation. It was found that the tool was useful in identifying gaps and quarterly compliance improved for 80% of process measures [ 34 ]. Another tool for assessing resident’s competence was developed to provide data on gaps in knowledge, which could be used to guide curriculum development [ 35 ]. Identifying the gaps, especially between evidence and practice or policy-making, is one of the crucial first steps in knowledge translation [ 36 ]. This is likely why tools that can identify gaps in practice prove to be valuable in QI initiatives and can be used during the formative planning process. The findings of the current review show that many website-based tools have been developed to digitalize existing QI processes, and ‘accessibility’ was identified as a facilitator to tool usage. A study examining digital tools for patient monitoring in oncology care found that digitalization allowed for nurse practitioners to efficiently extend and improve symptom management [ 37 ]. Another study found that a digital tool for a home-based blood pressure monitoring programme was effective and timely in hypertension management [ 38 ]. In recent years, digital transformation has been observed across many professional sectors and this has been associated with numerous advantages. For instance, the implementation of the Electronic Health Record has led to improvements in quality of care, reduction in medical errors, and increased adherence to clinical guidelines [ 39 ]. Considering how important QI is in the workforce, it was not surprising that many tools analysed in the current review sought to digitalize existing processes. Although digitalization has many advantages, there are limitations that should be considered. In the current review, a barrier to tool usage was the tool being time consuming or difficult to use. Similarly, a systematic review examining web-based interventions for weight loss highlighted that one of the challenges was engagement and retention. The authors suggested that there is a need to investigate components of web-based tools that can maintain users’ motivation and interest [ 40 ]. A study exploring digital tools to deliver physical activity advice identified that the biggest influence was having the skills to utilize the tool [ 41 ]. These observations, including the findings of the current review, suggest that digital literacy is an important consideration when developing digital tools or interventions and addressing these barriers may assist in maximizing user engagement and successful tool implementation. Another key QI process that the tools in the current review adopted was contributions to professional development. Education/training resources were also identified as one the common tool features. Many QI tools are designed to assist with professional development. For example, a digital application designed for teachers to improve their students’ linguistic competence assisted the teacher to self-assess their classes, make decisions, and facilitate practice changes through professional development. The use of the tool led to improved competencies for both the teachers and students [ 42 ]. A study that reported the development of a self-assessment tool for dental faculty to map professional growth identified the lack of defined faculty competencies in medical and dental education. Subsequently, a tool was developed and could be used as part of professional development [ 43 ]. Literature across all professions have demonstrated great benefits in undertaking professional development and this highlights why it is an ongoing requirement of practice for some. For example, continuing professional development is a requirement for nurses and has been directly linked to nurses’ career satisfaction and continuous growth in their practice [ 44 ]. In the current review, almost all tools were co-developed with stakeholders. This finding is further supported by ‘facilitating multidisciplinary action’ being identified as a facilitator of tool usage. Similarly, a QI project to improve ambulatory care for patients with musculoskeletal disorders focused on collaboration between general practitioners and specialists [ 45 ]. The study found that collaborative care was associated with a lower risk of osteoarthritis-related hospitalization, higher participation in exercise interventions, and more frequently prescribed physical therapy. The benefits of collaboration can be attributed to many factors, such as the opportunity to learn from others, access to new resources, increased productivity, and shared goals [ 46 ]. The current review identified lack of organizational engagement as a barrier, which included reluctance to change. A study exploring resistance towards changes among healthcare staff identified several reasons for reluctance, including personal reluctance, misunderstanding of project/initiative aims, and a dislike of the methods by which projects have been promoted [ 47 ]. A QI initiative for educational programmes found that there was reluctance among some teachers due to their uncertainty about potential benefits that could arise from the initiative [ 48 ]. In the current review, the qualitative data indicated factors, such as resource constraints, low staffing, and managerial support as barriers. These findings indicate that the reasons for lack of organizational engagement are multifactorial, and further research exploring this barrier would be beneficial. The findings offer valuable insights for future development of QI website-based tools. It is expected that more digital tools will be developed to drive practice improvements and the findings of this review can be useful in guiding the development process. Recommendations include ensuring tools are co-developed with healthcare professionals, focusing on practical usability, and addressing common barriers to enhance engagement and effectiveness in improving healthcare quality. The insights into the barriers and facilitators of tool usage can be broadly applied to any project that seeks to improve practice. Randomized controlled trials are warranted to provide objective evidence of tool efficacy.

Quality improvement (QI) involves stakeholders devising plans to enhance current practices to improve outcomes [ 1 ]. It is an ongoing process to continually improve practices and is often required as part of accreditation [ 2 ]. A range of frameworks are available to guide the approach to QI. For example, the ‘Plan-Do-Study-Act’ framework allows health departments to systematically compare current practices against established standards and to modify practices for continuous QI [ 2 ]. In addition, there are QI tools available. QI tools are specifically referring to instruments, such as checklists and process maps, used to implement and measure improvement initiatives, whereas QI frameworks provide systematic approaches for guiding the overall process of QI [ 3 ]. Technological advances have allowed tools to be integrated with online platforms. Research exploring how technology can support QI in a hospital setting concluded that such technology allowed for the provision of efficient and adequate feedback of performance [ 4 ]. For example, electronic health records integrated with real-time data analytics enable clinicians to receive immediate feedback on patient outcomes, facilitating prompt adjustments to care protocols and improving overall patient safety [ 5 ]. In addition, websites provide several advantages over other types of online platforms, such as more flexibility, accessibility and cross-platform compatibility [ 6 ]. As technology continues to advance and digitalization becomes more common, it is important to understand how digital tools can support QI in healthcare. There is abundant literature on QI frameworks [ 7–9 ]; however, to our knowledge, there is no literature summarizing QI tool design processes or components that can be used to guide tool development. With the advancement of technology, many digital tools will continue to be developed. QI projects can be complex and websites are a good technological solution to assist in these processes. This systematic review aims to understand how website-based tools can support health professionals with QI. The primary research question is: What are the optimal website-based QI tool design processes and elements of an effective QI tool?