Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol

Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Protocol Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol Anmol Shahid, Brianna K Rosgen, Karla K Krewulak, Diane L Lorenzetti, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-185385/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Background: Citizen engagement in research is an emerging practice that involves members of the general public in research processes such as priority-setting, planning, decision-making, research conduct, implementation, evaluation, and dissemination. Engaging citizens in research, particularly health research, increases the relevance of study findings, minimizes waste by facilitating stewardship over resources, and builds public trust in the research. While several existing frameworks guide the application of citizen engagement principles to health research, it is unclear how citizen engagement can be utilized to maximize benefits and minimize risks and challenges in health research. To address the gaps in knowledge around citizen engagement in health research, we propose a scoping review to synthesize the state of knowledge on methods to incorporate and evaluate citizen engagement in research. A protocol is presented in this manuscript. Methods: The methodology for our scoping review is guided by Arksey and O’ Malley’s framework for scoping reviews, and additional recommendations by Levac and colleagues. We will include peer-reviewed and grey literature that report on citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health) and report method(s) to conduct, measure, or evaluate citizen engagement. We will systematically search electronic databases (MEDLINE, EMBASE, CINAHL, JSTOR, PsycINFO, Scopus, and Science Direct) and relevant organizations’ websites for additional studies, frameworks, and reports on citizen engagement. Title and abstract and full-text citations will be screened independently and in duplicate. Data will be extracted independently and in duplicate, including: document characteristics, citizen engagement definitions and goals, and outcomes of citizen engagement (e.g., impact, barriers, facilitators). Discussion: This review will synthesize the definitions, goals, methods, outcomes, and significance of citizen engagement in health research, as well as any potential barriers, facilitators, and challenges outlined in existing literature. The findings will provide an evidence-based foundation for developing new or improved guidance for citizen engagement in health research. Overall, we anticipate that our scoping review will be a preliminary step to meaningful engagement of citizens in research and strengthen the relationship between the scientific community and the public through transparency and collaboration. SYSTEMATIC REVIEW REGISTRATION: Not applicable. Health Economics & Outcomes Research Citizen engagement public participation health research co-designed research models frameworks health services research scoping review protocol Background Citizen engagement in research is defined as the effective and systematic involvement of members of the general public or persons from affected community groups such as patients, caregivers, advocates, and representatives in research processes ( 1 ). Organizations worldwide have suggested that citizen engagement in research may confer benefits to the conduct and uptake of scientific research through improving the relevance of study findings, encouraging the representation of diverse groups in research studies, minimizing waste by facilitating stewardship over resources, promoting mutual learning and understanding, and allowing broad dissemination of research findings beyond traditional academic audiences ( 2 – 6 ). Most importantly, citizen engagement has demonstrated utility in building public trust in science and research ( 7 ). This has led to citizen engagement becoming a priority for research funding organizations (e.g., the Canadian Institutes of Health Research [CIHR], National Institutes of Health [NIH] in the United States, and the National Institute for Health Research [NIHR] in the United Kingdom), policymakers, academic journals, and researchers themselves ( 1 , 8 – 12 ). As a result, citizen engagement is now changing scientific research, particularly health research, to incorporate real world lived-experiences and perspectives of members of the public. Engaging citizens in research necessitates researchers to modify their research approach and critically evaluate their work to improve its generalizability and impact. However, while citizen engagement in research has been conceptually substantiated, it is unclear when, for whom, and how it can be most impactful ( 13 ). Despite the availability of frameworks and guidelines for citizen engagement in health research ( 1 , 2 ), there remains a lack of evidence-based methods to incorporate effective and systematic, or meaningful, engagement and valid measures to assess its impact. To effectively change health research so that it meaningfully includes the citizen perspective, there is a pressing need for a systematic synthesis of the broad field of citizen engagement in health research. The findings may be used to develop a new framework of citizen engagement or enhance and refine existing frameworks with a strong evidentiary basis, ultimately leading to effective and unified citizen engagement strategies. To address the gaps in knowledge around citizen engagement in health research, we will conduct a scoping review to synthesize the state of knowledge on methods to incorporate and evaluate citizen engagement in this field. The outcomes of this scoping review will provide a descriptive summary of the literature on citizen engagement including frameworks, models, and interventions aimed at improving engagement in health research. This will yield foundational results including strengths and weaknesses in existing frameworks, models and interventions, and help pinpoint areas that can be targeted to improve the relevance and effectiveness of future frameworks and recommendations. Methods This scoping review will be conducted according to recommendations specified by Arksey and O’Malley ( 14 ), and enhanced by Levac and colleagues ( 15 ). The conduct and reporting of this scoping review with be further guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guide review reporting ( 16 ). Stage 1: Identification of the research question We designed the research question to be broad in order to capture all potentially relevant information sources, as recommended by Arksey and O’Malley ( 14 ). This scoping review aims to answer the question: “ what is the state of knowledge on methods to incorporate and evaluate citizen engagement in health research? ”. The proposed research question provides direction to the subsequent steps, and defines the scope of inquiry with respect to population, concept, and outcomes of interest ( 14 ). For the purpose of this scoping review, our target population “citizens” will be defined as consumers of health services, informal caregivers, advocates and representatives from community organizations, and members of the general public. Our target concept is engagement or participation in health research. For the purposes of this scoping review, we will adopt the CIHR definition of health research, which includes research in the biomedical, clinical, health systems and services, and social, cultural, environmental and population health fields ( 17 ). We recognize that engagement may take many forms, including but not limited to: priority-setting, planning, decision-making, research conduct, implementation, evaluation, or dissemination. Additionally, citizen engagement may occur when citizens are joint grant holders or co-applicants on research grants, help to identify research priorities, hold membership in a project advisory or steering group, comment and develop research materials, interact with research participants, and/or help carry out research activities. Stage 2: Identification of relevant studies As recommended by Arksey & O’Malley and Levac and colleagues ( 14 , 15 ), we will identify relevant literature using a pre-determined plan for data sources and search strategy, including search terms, languages, and dates of search. The search strategies will be designed to maximize the comprehensiveness and breadth of the search, while considering time and personnel workload as limiting factors ( 14 , 15 ). Search strategy A search strategy will be developed by the study team, including a medical librarian (DLL), who have led multiple initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research ( 18 – 22 ). The following databases will be searched from inception: MEDLINE (1879-present), EMBASE (1947-present), Cochrane Library (1996-present), CINAHL (1961-present), PsycINFO (1927-present), Scopus (1970-present), and Web of Science (1964-present). Our search will be broad to encompass relevant terminology in this area, including subject headings and keywords and relevant synonyms related to three concepts: ( 1 ) citizens (e.g., community member, lay person, public, stakeholder), ( 2 ) engagement (e.g., collaboration, engagement, participation), ( 3 ) health research (e.g., biomedical research, health research, public health research). No exclusion criteria will be placed on language, though we will exclude studies published prior to the year 2000. Prior to implementation, the search strategy will be independently reviewed using the Peer Review of Electronic Search Strategies (PRESS) checklist ( 23 ). A draft search strategy (MEDLINE) is shown in Additional File 1. A targeted search of the grey literature will also be conducted, searching relevant local, provincial, national, and international organizations’ websites and related scientific or national funding organizations (i.e., OpenGrey, Trip, Involve, CIHR, Patient-Centered Outcomes Research Institute [PCORI]) for additional studies, frameworks, and reports on citizen engagement. Finally, the reference lists of included studies and related systematic reviews will also be screened to identify potentially relevant literature in health research. Stage 3: Study selection Following developing a strategy to identify studies, we will screen and select relevant studies for inclusion in the scoping review. As recommended by Levac and colleagues, we will develop inclusion and exclusion criteria a priori , with additional meetings with the study team to refine the study selection process at the beginning, midpoint, and endpoint of the citation screening process in case there are any unforeseen factors to be considered ( 15 ). Eligibility criteria Studies will be eligible for inclusion if they: ( 1 ) are primary (e.g., observational or interventional studies) or secondary (e.g., systematic or scoping reviews) research, frameworks, reviews, or reports, ( 2 ) report citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health, as defined by CIHR) and ( 3 ) report method(s) to conduct, measure, or evaluate citizen engagement. No restrictions will be placed on language or study design. Studies published prior to the year 2000 will be excluded. Selection process Retrieved articles will be imported to Covidence (Veritas Health Innovation, Melbourne, Australia) for title & abstract screening, completed independently and in duplicate by two reviewers. Reviewers will pilot screen the titles and abstracts of 50 articles to ensure consistency of inclusion and exclusion criteria and once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved, proceed to screen the remaining articles. If one reviewer indicates an article as potentially relevant at the title & abstract screening phase, the article will proceed to full-text review to ensure inclusivity. Following title & abstract screening, the full- text of selected studies will be screened independently and in duplicate by two reviewers. Reviewers will pilot screen the full text of 20 articles, and proceed to screen the remaining articles once a Kappa coefficient of inter-rater agreement ≥ 0.8 is achieved. Both reviewers must agree on inclusion status and reason for exclusion, if relevant. Any disagreements between the reviewers will be resolved by discussion or the involvement of a third reviewer, if required. Stage 4: Charting the data We will chart the data according to Arksey & O’Malley and Levac’s recommendations ( 14 , 15 ), beginning with defining the information to be extracted from the studies by developing an initial data charting form and then refining this form iteratively and through regular discussion with the study team. The study team will develop a standardized data extraction form in Microsoft Excel (version 16.29.1). The data extraction form will be piloted by two reviewers with ten included studies, and revised as needed. Once the final data abstraction form is developed, all relevant articles will then be abstracted independently and in duplicate by two reviewers. Extracted variables in the initial data extraction form will include study characteristics, participants, type and goals of citizen engagement, engagement frameworks and other strategies, engagement uptake, and findings and outcomes of citizen engagement in the context of health research. A draft list of variables to be extracted is shown in Table 1 . Table 1 Draft list of variables to be extracted Document identification List of authors (surnames) Year of publication Publisher (e.g., journal, institution) Research area (e.g., health sciences, social sciences, and natural sciences and engineering) Document type Framework OR Model OR Report Primary study OR Secondary research Characteristics of intended participants Population of focus (patient, family, healthcare provider, researcher, knowledge-user, decision-maker) Age Sex Gender Geographical location Citizen engagement Operationalization Definition of citizen engagement Type and goals of engagement (e.g., priority setting, decision-making, research conduct, implementation, evaluation, dissemination) Engagement frameworks used Engagement uptake Methods (if applicable) Study design Sample size Outcome measurement Results and Discussion (if applicable) Findings and outcomes of citizen engagement Barriers to citizen engagement Facilitators of citizen engagement Benefits of citizen engagement Challenges and risks of citizen engagement Strategies to mitigate challenges and risks of citizen engagement Author-stated conclusions (if applicable) Stage 5: Collating summarizing and reporting the results We will collate, summarize, and report the results according to the Arksey & O’Malley and Levac guidelines, which recommend undertaking synthesis of results by i) analyzing the data, ii) reporting results, ii) and evaluating their meaning ( 14 , 15 ). More specifically, these guidelines suggest that researchers execute a detailed thematic analysis of their data guided by qualitative analytical techniques before reporting results in a suitable form such as a narrative synthesis, a table of barriers/facilitators or knowledge gaps, or a framework depending on the goal of the review ( 14 , 15 ). Arksey & O’Malley and Levac further recommend that researchers aim to contextualize their findings within existing literature and research, practice, and policy ( 14 , 15 ). We will use a PRISMA-ScR checklist to guide reporting and the flow diagram to report the number of unique articles identified, excluded articles, reasons for exclusion, and included articles in the final scoping review ( 16 ). We will present a descriptive summary of characteristics of the included documents, and the characteristics of the intended participants or audiences for these documents. If applicable, we will conduct a narrative synthesis of extracted data variables, including, but not limited to: definition of citizen engagement, goal of citizen engagement, impact of citizen engagement, outcome measurement, author stated conclusions, barriers, facilitators and benefits, risks, and challenges of citizen engagement. Stage 6: Consultation As recommended by Arksey and O’Malley ( 14 ), we will involve citizens (NF, BS) in study conception and design as well as interpretation and contextualization of the data. This stakeholder consultation (i.e., citizen engagement) will allow for the incorporation of perspectives and insights beyond the literature and is necessary to improve the rigour and uptake of findings ( 14 , 15 ). MAIN DELIVERABLE Results from this scoping review will be synthesized to inform the future development or refinement of a framework for incorporating meaningful citizen engagement in health research. The barriers, facilitators, risks and challenges to citizen engagement identified through this scoping review will be used by our team in future framework development or refinement. Specific focus will be placed on evaluation of citizen engagement in order to improve upon existing methods of incorporating citizen engagement in health research. Once developed, a draft of this framework will be circulated through surveys to stakeholders including health researchers, decision and policy-makers, and citizens who indicate interest to solicit feedback. Focus groups will be conducted to discuss specific items in the framework which will lead the reiterative refinement of this work. While the limitations to citizen engagement have been outlined before ( 24 – 26 ), this applied framework will chart barriers and facilitators to citizen engagement systematically, allowing our team and stakeholders to effectively utilize this knowledge to initiate, perform and carry out an evidence-based process of framework building. Discussion This scoping review aims to synthesize published research, guidelines, and frameworks relevant to citizen engagement in health research. The findings of this scoping review will be applicable to many research disciplines beyond health research such as social sciences research and natural sciences research. The outcome of this scoping review will be a summary of the definitions, goals, methods, outcomes, significance, and evaluation of citizen engagement alongside potential barriers, facilitators, benefits, challenges, or risks. The potential findings of this scoping review will have several applications. First, we will provide an evidence-based foundation to create novel guidance for citizen engagement in research or enhance existing guidelines, such as those developed by CIHR ( 1 ) and NIHR ( 1 , 3 ). Second, this scoping review may highlight potential gaps in the literature, directing future areas of study in the science of citizen engagement in health research. Third, our scoping review will act as a comprehensive information source of citizen engagement in research that research teams may learn from and apply to their own work. Fourth, this scoping review will identify where sufficient literature exists to warrant systematic reviews that could inform citizens of their potential to be impactful co-developers of research across many disciplines. Overall, we hope that our study will be a preliminary step to meaningful and effective engagement of citizens in research to improve the impact and uptake of research findings, and strengthen the relationship between the scientific community and the public through transparency and collaboration. Team preparedness and dissemination of research findings We are well-positioned to complete this project due to our knowledge and experience in engaging and evaluating the impact of citizen engagement in health research. We have led initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research and routine clinical care through our research programs ( 18 – 22 ). We have experience with engaging and training patient and family partners in critical care medicine research (e.g., generating ideas, writing grants, priority setting, conducting research, writing manuscripts), and involving patient advisors in communication of research findings to members of the public at open engagement sessions ( 18 – 22 ). Additionally, we are well-equipped to undertake the methodologies outlined in this proposed research. We have a track-record of high-quality and pertinent systematic and scoping review publications that inform our research ( 27 – 30 ). The findings of this scoping review will be disseminated widely by the study team members through our research team and institutional websites and presentations at local, national, and international conferences. Our citizen research partners will aid in disseminating the study findings (both to lay and scientific audiences), a step we consider critical for this work to have an impact on citizen engagement in research. We intend to submit the findings for peer-reviewed publication. Abbreviations CIHR: Canadian Institutes of Health Research NIH: National Institutes of Health NIHR: National Institute for Health Research PRESS: Peer Review of Electronic Search Strategies PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews Declarations Ethics approval and consent to participate Not applicable. Consent to publish Not applicable. Availability of data and materials Not applicable. Competing interests The authors declare that they have no competing interests. Funding This study is currently unfunded but is seeking local and national sources of funding. Authors' contributions All authors (AS, BKR, KDK, DLL, JPL, NF, BS, HTS, and KF) have contributed to the conception and design of this protocol. AS and BKR drafted the manuscript. All authors critically revised the manuscript and approved of the final submitted version. Acknowledgements Not applicable. References Canadian Institutes of Health Research. CIHR’s Framework for Citizen Engagement. Ottawa, ON; 2012. INVOLVE. Exploring Impact: Public involvement in NHS, public health and social care research. 2009. National Institute for Health Research. Going the extra mile: Improving the nation’s health and wellbeing through public involvement in research [Available from: https://www.nihr.ac.uk/documents/about-us/our-contribution-to-research/how-we-involve-patients-carers-and-the-public/Going-the-Extra-Mile.pdf . Core Values for the Practice of Public Participation Sooke, BC: IAP2 Canada; 2018 [Available from: https://www.iap2canada.ca/foundations . Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387-95. Shimmin C, Wittmeier KDM, Lavoie JG, Wicklund ED, Sibley KM. Moving towards a more inclusive patient and public involvement in health research paradigm: the incorporation of a trauma-informed intersectional analysis. BMC Health Serv Res. 2017;17(1):539. Haywood BK. Beyond Data Points and Research Contributions: The Personal Meaning and Value Associated with Public Participation in Scientific Research. International Journal of Science Education, Part B. 2016;6(3):239-62. Walshe K, McKee M, Groenewegen P, Hansen J, Figueras J, Boccia S, et al. Reshaping the agenda of the European Commission for the health systems and policy research in Europe within Horizon 2020. Epidemiology Biostatistics and Public Health. 2013;10. Horobin A. Going the extra mile - creating a co-operative model for supporting patient and public involvement in research. Res Involv Engagem. 2016;2:9-. Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8(3):e020452. Social Sciences and Humanities Research Council. Community Engagement 2019 [Available from: https://www.sshrc-crsh.gc.ca/society-societe/community-communite/index-eng.aspx . Natural Sciences and Engineering Research Council of Canada. NSERC 2020 A Strategic Plan 2020 [June 23 2020]. Available from: https://www.nserc-crsng.gc.ca/nserc-crsng/nserc2020-crsng2020/index_eng.asp . Marris C, Rose N. Open engagement: exploring public participation in the biosciences. PLoS Biol. 2010;8(11):e1000549. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19-32. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Science. 2010;5(1):69. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467-73. Research CIoH. CIHR background: Government of Canada; 2019 [Available from: https://letstalk-cihr.ca/about . Fiest KM, Krewulak KD, Ely EW, Davidson JE, Ismail Z, Sept BG, et al. Partnering With Family Members to Detect Delirium in Critically Ill Patients. Crit Care Med. 2020;48(7):954-61. Fiest KM, Krewulak KD, Sept BG, Spence KL, Davidson JE, Ely EW, et al. A study protocol for a randomized controlled trial of family-partnered delirium prevention, detection, and management in critically ill adults: the ACTIVATE study. BMC Health Serv Res. 2020;20(1):453. Krewulak KD, Sept BG, Stelfox HT, Ely EW, Davidson JE, Ismail Z, et al. Feasibility and acceptability of family administration of delirium detection tools in the intensive care unit: a patient-oriented pilot study. CMAJ Open. 2019;7(2):E294-e9. Boyd JM, Burton R, Butler BL, Dyer D, Evans DC, Felteau M, et al. Development and Validation of Quality Criteria for Providing Patient- and Family-centered Injury Care. Ann Surg. 2017;266(2):287-96. Potestio ML, Boyd JM, Bagshaw SM, Heyland D, Oxland P, Doig CJ, et al. Engaging the Public to Identify Opportunities to Improve Critical Care: A Qualitative Analysis of an Open Community Forum. PLoS One. 2015;10(11):e0143088. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40-6. Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89. Grand A, Davies G, Holliman R, Adams A. Mapping public engagement with research in a UK University. PLoS One. 2015;10(4):e0121874. Powell MC, Colin M. Participatory Paradoxes:Facilitating Citizen Engagement in Science and Technology From the Top-Down? Bulletin of Science, Technology & Society. 2009;29(4):325-42. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452-7. Cherak SJ, Fiest KM, VanderSluis L, Basualdo-Hammond C, Lorenzetti DL, Buhler S, et al. Nutrition interventions in populations with mental health conditions: a scoping review. Appl Physiol Nutr Metab. 2020;45(7):687-97. Rosgen B, Krewulak K, Demiantschuk D, Ely EW, Davidson JE, Stelfox HT, et al. Validation of Caregiver-Centered Delirium Detection Tools: A Systematic Review. Journal of the American Geriatrics Society. 2018;66(6):1218-25. Cherak SJ, Rosgen BK, Amarbayan M, Plotnikoff K, Wollny K, Stelfox HT, et al. Impact of social media interventions and tools among informal caregivers of critically ill patients after patient admission to the intensive care unit: A scoping review. PLoS One. 2020;15(9):e0238803. Supplementary Files AdditionalFile1.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Minor revision 19 Jul, 2021 Review # 1 received at journal 18 Jul, 2021 Editor assigned by journal 01 Jul, 2021 Reviewers invited by journal 01 Jul, 2021 Reviewer # 1 agreed at journal 01 Jul, 2021 Reviews received at journal 01 Jul, 2021 Editor invited by journal 01 Jul, 2021 Submission checks completed at journal 06 Feb, 2021 First submitted to journal 29 Jan, 2021 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-185385","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Protocol","associatedPublications":[],"authors":[{"id":10655141,"identity":"88c4ccef-9ab6-450d-ad3b-fcb0a77294ae","order_by":0,"name":"Anmol Shahid","email":"","orcid":"https://orcid.org/0000-0001-5638-4988","institution":"University of Calgary Cumming School of Medicine","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Anmol","middleName":"","lastName":"Shahid","suffix":""},{"id":10655142,"identity":"d80eb340-00a5-471f-8759-5f8536b3e103","order_by":1,"name":"Brianna K Rosgen","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Brianna","middleName":"K","lastName":"Rosgen","suffix":""},{"id":10655143,"identity":"db99605f-6f21-49e6-9a43-c7d2bbec3970","order_by":2,"name":"Karla K Krewulak","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Karla","middleName":"K","lastName":"Krewulak","suffix":""},{"id":10655144,"identity":"6b0124a2-a5da-4126-9bc9-108d90638635","order_by":3,"name":"Diane L Lorenzetti","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Diane","middleName":"L","lastName":"Lorenzetti","suffix":""},{"id":10655145,"identity":"87781462-8ff2-402a-af9d-ba76903fa809","order_by":4,"name":"Nadine Foster","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Nadine","middleName":"","lastName":"Foster","suffix":""},{"id":10655146,"identity":"2d97547f-9d3f-469f-9040-340aaa700cb7","order_by":5,"name":"Bonnie G Sept","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Bonnie","middleName":"G","lastName":"Sept","suffix":""},{"id":10655147,"identity":"11d58180-0a43-4e4b-b5e1-0a14e53f7e42","order_by":6,"name":"Jeanna Parsons Leigh","email":"","orcid":"","institution":"Dalhousie University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jeanna","middleName":"Parsons","lastName":"Leigh","suffix":""},{"id":10655148,"identity":"961aeb44-9444-4923-8ec7-209a098be5a5","order_by":7,"name":"Henry T Stelfox","email":"","orcid":"","institution":"University of Calgary","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Henry","middleName":"T","lastName":"Stelfox","suffix":""},{"id":10655149,"identity":"4fb730f4-e7bf-4675-a427-9e594e6c9054","order_by":8,"name":"Kirsten M Fiest","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYBACCQbGBweAtBwQMx4gUguzAVClgTGIQ7wWIGWQ2EC0FslphxkPfPjzJ33D8cMPDn6puZfYwH74AV4t0tLJDAdnthnkbjiTZnBY5lhxYgNPmgFeLXLS+QcO8zYAtdxgMDgswZaQ2CDBQEhLMsPhP38M0g1usH84LPEPpIX9A0GHHWZgM0gwuMFjcPBjG0gLD35bJGcD/dLbZmw480xOwWHGvgTjNp6cArxaJG4nM3/48UdOnu/48Y0Pf3xLkO1nP74BrxYUwMzDwODYRrx6IGD8wcBgT5KOUTAKRsEoGBEAAD/HTX050VcCAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-7299-6594","institution":"Department of Critical Care Medicine Ground Floor, McCaig Tower 3134 Hospital Drive NW, Calgary, Alberta, Canada, T2N 5A1","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Kirsten","middleName":"M","lastName":"Fiest","suffix":""}],"badges":[],"createdAt":"2021-01-29 17:31:34","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-185385/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-185385/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":13658834,"identity":"604bf5cc-0fe7-403c-988c-9d9b576e90d9","added_by":"auto","created_at":"2021-09-17 10:16:19","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":413828,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-185385/v1/f71ec894-9752-4554-bf91-fbb2cd99ee95.pdf"},{"id":5838483,"identity":"69e9cc4d-a8ce-4eda-8754-888d53b90d44","added_by":"auto","created_at":"2021-02-10 19:38:55","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":296440,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalFile1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-185385/v1/eec7e6a893a32e057a49568d.pdf"}],"financialInterests":"","formattedTitle":"\u003cp\u003eIncorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol\u003c/p\u003e","fulltext":[{"header":"Background","content":" \u003cp\u003eCitizen engagement in research is defined as the effective and systematic involvement of members of the general public or persons from affected community groups such as patients, caregivers, advocates, and representatives in research processes (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). Organizations worldwide have suggested that citizen engagement in research may confer benefits to the conduct and uptake of scientific research through improving the relevance of study findings, encouraging the representation of diverse groups in research studies, minimizing waste by facilitating stewardship over resources, promoting mutual learning and understanding, and allowing broad dissemination of research findings beyond traditional academic audiences (\u003cspan additionalcitationids=\"CR3 CR4 CR5\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). Most importantly, citizen engagement has demonstrated utility in building public trust in science and research (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). This has led to citizen engagement becoming a priority for research funding organizations (e.g., the Canadian Institutes of Health Research [CIHR], National Institutes of Health [NIH] in the United States, and the National Institute for Health Research [NIHR] in the United Kingdom), policymakers, academic journals, and researchers themselves (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan additionalcitationids=\"CR9 CR10 CR11\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). As a result, citizen engagement is now changing scientific research, particularly health research, to incorporate real world lived-experiences and perspectives of members of the public.\u003c/p\u003e \u003cp\u003eEngaging citizens in research necessitates researchers to modify their research approach and critically evaluate their work to improve its generalizability and impact. However, while citizen engagement in research has been conceptually substantiated, it is unclear when, for whom, and how it can be most impactful (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). Despite the availability of frameworks and guidelines for citizen engagement in health research (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e), there remains a lack of evidence-based methods to incorporate effective and systematic, or meaningful, engagement and valid measures to assess its impact. To effectively change health research so that it meaningfully includes the citizen perspective, there is a pressing need for a systematic synthesis of the broad field of citizen engagement in health research. The findings may be used to develop a new framework of citizen engagement or enhance and refine existing frameworks with a strong evidentiary basis, ultimately leading to effective and unified citizen engagement strategies. To address the gaps in knowledge around citizen engagement in health research, we will conduct a scoping review to synthesize the state of knowledge on methods to incorporate and evaluate citizen engagement in this field. The outcomes of this scoping review will provide a descriptive summary of the literature on citizen engagement including frameworks, models, and interventions aimed at improving engagement in health research. This will yield foundational results including strengths and weaknesses in existing frameworks, models and interventions, and help pinpoint areas that can be targeted to improve the relevance and effectiveness of future frameworks and recommendations.\u003c/p\u003e "},{"header":"Methods","content":"\u003cp\u003eThis scoping review will be conducted according to recommendations specified by Arksey and O\u0026rsquo;Malley (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e), and enhanced by Levac and colleagues (\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e). The conduct and reporting of this scoping review with be further guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guide review reporting (\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e).\u003c/p\u003e\n\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 1: Identification of the research question\u003c/h2\u003e\n\u003cp\u003eWe designed the research question to be broad in order to capture all potentially relevant information sources, as recommended by Arksey and O\u0026rsquo;Malley (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e). This scoping review aims to answer the question: \u0026ldquo;\u003cem\u003ewhat is the state of knowledge on methods to incorporate and evaluate citizen engagement in health research?\u003c/em\u003e\u0026rdquo;. The proposed research question provides direction to the subsequent steps, and defines the scope of inquiry with respect to population, concept, and outcomes of interest (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eFor the purpose of this scoping review, our target population \u0026ldquo;citizens\u0026rdquo; will be defined as consumers of health services, informal caregivers, advocates and representatives from community organizations, and members of the general public. Our target concept is engagement or participation in health research. For the purposes of this scoping review, we will adopt the CIHR definition of health research, which includes research in the biomedical, clinical, health systems and services, and social, cultural, environmental and population health fields (\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e). We recognize that engagement may take many forms, including but not limited to: priority-setting, planning, decision-making, research conduct, implementation, evaluation, or dissemination. Additionally, citizen engagement may occur when citizens are joint grant holders or co-applicants on research grants, help to identify research priorities, hold membership in a project advisory or steering group, comment and develop research materials, interact with research participants, and/or help carry out research activities.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 2: Identification of relevant studies\u003c/h2\u003e\n\u003cp\u003eAs recommended by Arksey \u0026amp; O\u0026rsquo;Malley and Levac and colleagues (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e), we will identify relevant literature using a pre-determined plan for data sources and search strategy, including search terms, languages, and dates of search. The search strategies will be designed to maximize the comprehensiveness and breadth of the search, while considering time and personnel workload as limiting factors (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e\n\u003cdiv id=\"Sec5\" class=\"Section3\"\u003e\n\u003ch2\u003eSearch strategy\u003c/h2\u003e\n\u003cp\u003eA search strategy will be developed by the study team, including a medical librarian (DLL), who have led multiple initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research (\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e). The following databases will be searched from inception: MEDLINE (1879-present), EMBASE (1947-present), Cochrane Library (1996-present), CINAHL (1961-present), PsycINFO (1927-present), Scopus (1970-present), and Web of Science (1964-present). Our search will be broad to encompass relevant terminology in this area, including subject headings and keywords and relevant synonyms related to three concepts: (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e) citizens (e.g., community member, lay person, public, stakeholder), (\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e) engagement (e.g., collaboration, engagement, participation), (\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e) health research (e.g., biomedical research, health research, public health research). No exclusion criteria will be placed on language, though we will exclude studies published prior to the year 2000. Prior to implementation, the search strategy will be independently reviewed using the Peer Review of Electronic Search Strategies (PRESS) checklist (\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e). A draft search strategy (MEDLINE) is shown in Additional File 1.\u003c/p\u003e\n\u003cp\u003eA targeted search of the grey literature will also be conducted, searching relevant local, provincial, national, and international organizations\u0026rsquo; websites and related scientific or national funding organizations (i.e., OpenGrey, Trip, Involve, CIHR, Patient-Centered Outcomes Research Institute [PCORI]) for additional studies, frameworks, and reports on citizen engagement. Finally, the reference lists of included studies and related systematic reviews will also be screened to identify potentially relevant literature in health research.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 3: Study selection\u003c/h2\u003e\n\u003cp\u003eFollowing developing a strategy to identify studies, we will screen and select relevant studies for inclusion in the scoping review. As recommended by Levac and colleagues, we will develop inclusion and exclusion criteria \u003cem\u003ea priori\u003c/em\u003e, with additional meetings with the study team to refine the study selection process at the beginning, midpoint, and endpoint of the citation screening process in case there are any unforeseen factors to be considered (\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e\n\u003cdiv id=\"Sec7\" class=\"Section3\"\u003e\n\u003ch2\u003eEligibility criteria\u003c/h2\u003e\n\u003cp\u003eStudies will be eligible for inclusion if they: (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e) are primary (e.g., observational or interventional studies) or secondary (e.g., systematic or scoping reviews) research, frameworks, reviews, or reports, (\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e) report citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health, as defined by CIHR) and (\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e) report method(s) to conduct, measure, or evaluate citizen engagement. No restrictions will be placed on language or study design. Studies published prior to the year 2000 will be excluded.\u003c/p\u003e\n\u003cdiv id=\"Sec8\" class=\"Section4\"\u003e\n\u003ch2\u003eSelection process\u003c/h2\u003e\n\u003cp\u003eRetrieved articles will be imported to Covidence (Veritas Health Innovation, Melbourne, Australia) for title \u0026amp; abstract screening, completed independently and in duplicate by two reviewers. Reviewers will pilot screen the titles and abstracts of 50 articles to ensure consistency of inclusion and exclusion criteria and once a Kappa coefficient of inter-rater agreement\u0026thinsp;\u0026ge;\u0026thinsp;0.8 is achieved, proceed to screen the remaining articles. If one reviewer indicates an article as potentially relevant at the title \u0026amp; abstract screening phase, the article will proceed to full-text review to ensure inclusivity. Following title \u0026amp; abstract screening, the full- text of selected studies will be screened independently and in duplicate by two reviewers. Reviewers will pilot screen the full text of 20 articles, and proceed to screen the remaining articles once a Kappa coefficient of inter-rater agreement\u0026thinsp;\u0026ge;\u0026thinsp;0.8 is achieved. Both reviewers must agree on inclusion status and reason for exclusion, if relevant. Any disagreements between the reviewers will be resolved by discussion or the involvement of a third reviewer, if required.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 4: Charting the data\u003c/h2\u003e\n\u003cp\u003eWe will chart the data according to Arksey \u0026amp; O\u0026rsquo;Malley and Levac\u0026rsquo;s recommendations (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e), beginning with defining the information to be extracted from the studies by developing an initial data charting form and then refining this form iteratively and through regular discussion with the study team.\u003c/p\u003e\n\u003cp\u003eThe study team will develop a standardized data extraction form in Microsoft Excel (version 16.29.1). The data extraction form will be piloted by two reviewers with ten included studies, and revised as needed. Once the final data abstraction form is developed, all relevant articles will then be abstracted independently and in duplicate by two reviewers.\u003c/p\u003e\n\u003cp\u003eExtracted variables in the initial data extraction form will include study characteristics, participants, type and goals of citizen engagement, engagement frameworks and other strategies, engagement uptake, and findings and outcomes of citizen engagement in the context of health research. A draft list of variables to be extracted is shown in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eDraft list of variables to be extracted\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eDocument identification\u003c/p\u003e\n\u003c/th\u003e\n\u003cth colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003eList of authors (surnames)\u003c/p\u003e\n\u003cp\u003eYear of publication\u003c/p\u003e\n\u003cp\u003ePublisher (e.g., journal, institution)\u003c/p\u003e\n\u003cp\u003eResearch area (e.g., health sciences, social sciences, and natural sciences and engineering)\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDocument type\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003eFramework OR Model OR Report\u003c/p\u003e\n\u003cp\u003ePrimary study OR Secondary research\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCharacteristics of intended participants\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003ePopulation of focus (patient, family, healthcare provider, researcher, knowledge-user, decision-maker)\u003c/p\u003e\n\u003cp\u003eAge\u003c/p\u003e\n\u003cp\u003eSex\u003c/p\u003e\n\u003cp\u003eGender\u003c/p\u003e\n\u003cp\u003eGeographical location\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd rowspan=\"3\" align=\"left\"\u003e\n\u003cp\u003eCitizen engagement\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eOperationalization\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDefinition of citizen engagement\u003c/p\u003e\n\u003cp\u003eType and goals of engagement (e.g., priority setting, decision-making, research conduct, implementation, evaluation, dissemination)\u003c/p\u003e\n\u003cp\u003eEngagement frameworks used\u003c/p\u003e\n\u003cp\u003eEngagement uptake\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMethods\u003c/p\u003e\n\u003cp\u003e(if applicable)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eStudy design\u003c/p\u003e\n\u003cp\u003eSample size\u003c/p\u003e\n\u003cp\u003eOutcome measurement\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eResults and Discussion\u003c/p\u003e\n\u003cp\u003e(if applicable)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eFindings and outcomes of citizen engagement\u003c/p\u003e\n\u003cp\u003eBarriers to citizen engagement\u003c/p\u003e\n\u003cp\u003eFacilitators of citizen engagement\u003c/p\u003e\n\u003cp\u003eBenefits of citizen engagement\u003c/p\u003e\n\u003cp\u003eChallenges and risks of citizen engagement\u003c/p\u003e\n\u003cp\u003eStrategies to mitigate challenges and risks of citizen engagement\u003c/p\u003e\n\u003cp\u003eAuthor-stated conclusions (if applicable)\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 5: Collating summarizing and reporting the results\u003c/h2\u003e\n\u003cp\u003eWe will collate, summarize, and report the results according to the Arksey \u0026amp; O\u0026rsquo;Malley and Levac guidelines, which recommend undertaking synthesis of results by i) analyzing the data, ii) reporting results, ii) and evaluating their meaning (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e). More specifically, these guidelines suggest that researchers execute a detailed thematic analysis of their data guided by qualitative analytical techniques before reporting results in a suitable form such as a narrative synthesis, a table of barriers/facilitators or knowledge gaps, or a framework depending on the goal of the review (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e). Arksey \u0026amp; O\u0026rsquo;Malley and Levac further recommend that researchers aim to contextualize their findings within existing literature and research, practice, and policy (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eWe will use a PRISMA-ScR checklist to guide reporting and the flow diagram to report the number of unique articles identified, excluded articles, reasons for exclusion, and included articles in the final scoping review (\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e). We will present a descriptive summary of characteristics of the included documents, and the characteristics of the intended participants or audiences for these documents. If applicable, we will conduct a narrative synthesis of extracted data variables, including, but not limited to: definition of citizen engagement, goal of citizen engagement, impact of citizen engagement, outcome measurement, author stated conclusions, barriers, facilitators and benefits, risks, and challenges of citizen engagement.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\n\u003ch2\u003eStage 6: Consultation\u003c/h2\u003e\n\u003cp\u003eAs recommended by Arksey and O\u0026rsquo;Malley (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e), we will involve citizens (NF, BS) in study conception and design as well as interpretation and contextualization of the data. This stakeholder consultation (i.e., citizen engagement) will allow for the incorporation of perspectives and insights beyond the literature and is necessary to improve the rigour and uptake of findings (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e\n\u003ch2\u003eMAIN DELIVERABLE\u003c/h2\u003e\n\u003c/div\u003e\n\u003cp\u003eResults from this scoping review will be synthesized to inform the future development or refinement of a \u003cem\u003eframework for incorporating meaningful citizen engagement in health research.\u003c/em\u003e The barriers, facilitators, risks and challenges to citizen engagement identified through this scoping review will be used by our team in future framework development or refinement. Specific focus will be placed on evaluation of citizen engagement in order to improve upon existing methods of incorporating citizen engagement in health research. Once developed, a draft of this framework will be circulated through surveys to stakeholders including health researchers, decision and policy-makers, and citizens who indicate interest to solicit feedback. Focus groups will be conducted to discuss specific items in the framework which will lead the reiterative refinement of this work. While the limitations to citizen engagement have been outlined before (\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e), this applied framework will chart barriers and facilitators to citizen engagement systematically, allowing our team and stakeholders to effectively utilize this knowledge to initiate, perform and carry out an evidence-based process of framework building.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis scoping review aims to synthesize published research, guidelines, and frameworks relevant to citizen engagement in health research. The findings of this scoping review will be applicable to many research disciplines beyond health research such as social sciences research and natural sciences research. The outcome of this scoping review will be a summary of the definitions, goals, methods, outcomes, significance, and evaluation of citizen engagement alongside potential barriers, facilitators, benefits, challenges, or risks.\u003c/p\u003e\n\u003cp\u003eThe potential findings of this scoping review will have several applications. First, we will provide an evidence-based foundation to create novel guidance for citizen engagement in research or enhance existing guidelines, such as those developed by CIHR (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e) and NIHR (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e). Second, this scoping review may highlight potential gaps in the literature, directing future areas of study in the science of citizen engagement in health research. Third, our scoping review will act as a comprehensive information source of citizen engagement in research that research teams may learn from and apply to their own work. Fourth, this scoping review will identify where sufficient literature exists to warrant systematic reviews that could inform citizens of their potential to be impactful co-developers of research across many disciplines. Overall, we hope that our study will be a preliminary step to meaningful and effective engagement of citizens in research to improve the impact and uptake of research findings, and strengthen the relationship between the scientific community and the public through transparency and collaboration.\u003c/p\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n\u003ch2\u003eTeam preparedness and dissemination of research findings\u003c/h2\u003e\n\u003cp\u003eWe are well-positioned to complete this project due to our knowledge and experience in engaging and evaluating the impact of citizen engagement in health research. We have led initiatives to develop, adapt, and evaluate approaches to incorporate citizen engagement into research and routine clinical care through our research programs (\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e). We have experience with engaging and training patient and family partners in critical care medicine research (e.g., generating ideas, writing grants, priority setting, conducting research, writing manuscripts), and involving patient advisors in communication of research findings to members of the public at open engagement sessions (\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e). Additionally, we are well-equipped to undertake the methodologies outlined in this proposed research. We have a track-record of high-quality and pertinent systematic and scoping review publications that inform our research (\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e30\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eThe findings of this scoping review will be disseminated widely by the study team members through our research team and institutional websites and presentations at local, national, and international conferences. Our citizen research partners will aid in disseminating the study findings (both to lay and scientific audiences), a step we consider critical for this work to have an impact on citizen engagement in research. We intend to submit the findings for peer-reviewed publication.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Abbreviations","content":"\u003cp\u003e\u003cstrong\u003eCIHR:\u003c/strong\u003e Canadian Institutes of Health Research\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNIH:\u003c/strong\u003e National Institutes of Health\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNIHR:\u003c/strong\u003e National Institute for Health Research\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePRESS:\u003c/strong\u003e Peer Review of Electronic Search Strategies\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePRISMA-ScR:\u003c/strong\u003e Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to publish\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study is currently unfunded but is seeking local and national sources of funding.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors' contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors (AS, BKR, KDK, DLL, JPL, NF, BS, HTS, and KF) have contributed to the conception and design of this protocol. AS and BKR drafted the manuscript. All authors critically revised the manuscript and approved of the final submitted version.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements \u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eCanadian Institutes of Health Research. CIHR\u0026rsquo;s Framework for Citizen Engagement. Ottawa, ON; 2012.\u003c/li\u003e\n\u003cli\u003eINVOLVE. Exploring Impact: Public involvement in NHS, public health and social care research. 2009.\u003c/li\u003e\n\u003cli\u003eNational Institute for Health Research. Going the extra mile: Improving the nation\u0026rsquo;s health and wellbeing through public involvement in research [Available from: \u003ca href=\"https://www.nihr.ac.uk/documents/about-us/our-contribution-to-research/how-we-involve-patients-carers-and-the-public/Going-the-Extra-Mile.pdf\"\u003ehttps://www.nihr.ac.uk/documents/about-us/our-contribution-to-research/how-we-involve-patients-carers-and-the-public/Going-the-Extra-Mile.pdf\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eCore Values for the Practice of Public Participation Sooke, BC: IAP2 Canada; 2018 [Available from: \u003ca href=\"https://www.iap2canada.ca/foundations\"\u003ehttps://www.iap2canada.ca/foundations\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eBrett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387-95.\u003c/li\u003e\n\u003cli\u003eShimmin C, Wittmeier KDM, Lavoie JG, Wicklund ED, Sibley KM. Moving towards a more inclusive patient and public involvement in health research paradigm: the incorporation of a trauma-informed intersectional analysis. BMC Health Serv Res. 2017;17(1):539.\u003c/li\u003e\n\u003cli\u003eHaywood BK. Beyond Data Points and Research Contributions: The Personal Meaning and Value Associated with Public Participation in Scientific Research. International Journal of Science Education, Part B. 2016;6(3):239-62.\u003c/li\u003e\n\u003cli\u003eWalshe K, McKee M, Groenewegen P, Hansen J, Figueras J, Boccia S, et al. Reshaping the agenda of the European Commission for the health systems and policy research in Europe within Horizon 2020. Epidemiology Biostatistics and Public Health. 2013;10.\u003c/li\u003e\n\u003cli\u003eHorobin A. Going the extra mile - creating a co-operative model for supporting patient and public involvement in research. Res Involv Engagem. 2016;2:9-.\u003c/li\u003e\n\u003cli\u003ePrice A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, et al. Frequency of reporting on patient and public involvement (PPI) in research studies published in a general medical journal: a descriptive study. BMJ Open. 2018;8(3):e020452.\u003c/li\u003e\n\u003cli\u003eSocial Sciences and Humanities Research Council. Community Engagement 2019 [Available from: \u003ca href=\"https://www.sshrc-crsh.gc.ca/society-societe/community-communite/index-eng.aspx\"\u003ehttps://www.sshrc-crsh.gc.ca/society-societe/community-communite/index-eng.aspx\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eNatural Sciences and Engineering Research Council of Canada. NSERC 2020 A Strategic Plan 2020 [June 23 2020]. Available from: \u003ca href=\"https://www.nserc-crsng.gc.ca/nserc-crsng/nserc2020-crsng2020/index_eng.asp\"\u003ehttps://www.nserc-crsng.gc.ca/nserc-crsng/nserc2020-crsng2020/index_eng.asp\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eMarris C, Rose N. Open engagement: exploring public participation in the biosciences. PLoS Biol. 2010;8(11):e1000549.\u003c/li\u003e\n\u003cli\u003eArksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19-32.\u003c/li\u003e\n\u003cli\u003eLevac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Science. 2010;5(1):69.\u003c/li\u003e\n\u003cli\u003eTricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467-73.\u003c/li\u003e\n\u003cli\u003eResearch CIoH. CIHR background: Government of Canada; 2019 [Available from: \u003ca href=\"https://letstalk-cihr.ca/about\"\u003ehttps://letstalk-cihr.ca/about\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eFiest KM, Krewulak KD, Ely EW, Davidson JE, Ismail Z, Sept BG, et al. Partnering With Family Members to Detect Delirium in Critically Ill Patients. Crit Care Med. 2020;48(7):954-61.\u003c/li\u003e\n\u003cli\u003eFiest KM, Krewulak KD, Sept BG, Spence KL, Davidson JE, Ely EW, et al. A study protocol for a randomized controlled trial of family-partnered delirium prevention, detection, and management in critically ill adults: the ACTIVATE study. BMC Health Serv Res. 2020;20(1):453.\u003c/li\u003e\n\u003cli\u003eKrewulak KD, Sept BG, Stelfox HT, Ely EW, Davidson JE, Ismail Z, et al. Feasibility and acceptability of family administration of delirium detection tools in the intensive care unit: a patient-oriented pilot study. CMAJ Open. 2019;7(2):E294-e9.\u003c/li\u003e\n\u003cli\u003eBoyd JM, Burton R, Butler BL, Dyer D, Evans DC, Felteau M, et al. Development and Validation of Quality Criteria for Providing Patient- and Family-centered Injury Care. Ann Surg. 2017;266(2):287-96.\u003c/li\u003e\n\u003cli\u003ePotestio ML, Boyd JM, Bagshaw SM, Heyland D, Oxland P, Doig CJ, et al. Engaging the Public to Identify Opportunities to Improve Critical Care: A Qualitative Analysis of an Open Community Forum. PLoS One. 2015;10(11):e0143088.\u003c/li\u003e\n\u003cli\u003eMcGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40-6.\u003c/li\u003e\n\u003cli\u003eDomecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.\u003c/li\u003e\n\u003cli\u003eGrand A, Davies G, Holliman R, Adams A. Mapping public engagement with research in a UK University. PLoS One. 2015;10(4):e0121874.\u003c/li\u003e\n\u003cli\u003ePowell MC, Colin M. Participatory Paradoxes:Facilitating Citizen Engagement in Science and Technology From the Top-Down? Bulletin of Science, Technology \u0026amp; Society. 2009;29(4):325-42.\u003c/li\u003e\n\u003cli\u003eBowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452-7.\u003c/li\u003e\n\u003cli\u003eCherak SJ, Fiest KM, VanderSluis L, Basualdo-Hammond C, Lorenzetti DL, Buhler S, et al. Nutrition interventions in populations with mental health conditions: a scoping review. Appl Physiol Nutr Metab. 2020;45(7):687-97.\u003c/li\u003e\n\u003cli\u003eRosgen B, Krewulak K, Demiantschuk D, Ely EW, Davidson JE, Stelfox HT, et al. Validation of Caregiver-Centered Delirium Detection Tools: A Systematic Review. Journal of the American Geriatrics Society. 2018;66(6):1218-25.\u003c/li\u003e\n\u003cli\u003eCherak SJ, Rosgen BK, Amarbayan M, Plotnikoff K, Wollny K, Stelfox HT, et al. Impact of social media interventions and tools among informal caregivers of critically ill patients after patient admission to the intensive care unit: A scoping review. PLoS One. 2020;15(9):e0238803.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"systematic-reviews","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sysr","sideBox":"Learn more about [Systematic Reviews](http://systematicreviewsjournal.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/sysr/default.aspx","title":"Systematic Reviews","twitterHandle":"@MedicalEvidence","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Citizen engagement, public participation, health research, co-designed research, models, frameworks, health services research, scoping review, protocol","lastPublishedDoi":"10.21203/rs.3.rs-185385/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-185385/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e\u0026nbsp;Citizen engagement in research is an emerging practice that involves members of the general public in research processes such as priority-setting, planning, decision-making, research conduct, implementation, evaluation, and dissemination. Engaging citizens in research, particularly health research, increases the relevance of study findings, minimizes waste by facilitating stewardship over resources, and builds public trust in the research. While several existing frameworks guide the application of citizen engagement principles to health research, it is unclear how citizen engagement can be utilized to maximize benefits and minimize risks and challenges in health research. To address the gaps in knowledge around citizen engagement in health research, we propose a scoping review to synthesize the state of knowledge on methods to incorporate\u003cem\u003e \u003c/em\u003eand evaluate citizen engagement in research. A protocol is presented in this manuscript. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eMethods:\u0026nbsp;\u003c/strong\u003eThe methodology for our scoping review is guided by Arksey and O’ Malley’s framework for scoping reviews, and additional recommendations by Levac and colleagues. We will include peer-reviewed and grey literature that report on citizen engagement in health research (including biomedical, clinical, health systems and services, and social, cultural, environmental and population health) and report method(s) to conduct, measure, or evaluate citizen engagement. We will systematically search electronic databases (MEDLINE, EMBASE, CINAHL, JSTOR, PsycINFO, Scopus, and Science Direct) and relevant organizations’ websites for additional studies, frameworks, and reports on citizen engagement. Title and abstract and full-text citations will be screened independently and in duplicate. Data will be extracted independently and in duplicate, including: document characteristics, citizen engagement definitions and goals, and outcomes of citizen engagement (e.g., impact, barriers, facilitators). \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDiscussion:\u003c/strong\u003e\u0026nbsp;This review will synthesize the definitions, goals, methods, outcomes, and significance of citizen engagement in health research, as well as any potential barriers, facilitators, and challenges outlined in existing literature. The findings will provide an evidence-based foundation for developing new or improved guidance for citizen engagement in health research. Overall, we anticipate that our scoping review will be a preliminary step to meaningful engagement of citizens in research and strengthen the relationship between the scientific community and the public through transparency and collaboration.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eSYSTEMATIC REVIEW REGISTRATION:\u003c/strong\u003e\u0026nbsp;Not applicable.\u003c/p\u003e","manuscriptTitle":"Incorporating and Evaluating Citizen Engagement in Health Research: a Scoping Review Protocol","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-02-10 19:38:54","doi":"10.21203/rs.3.rs-185385/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Minor revision","date":"2021-07-19T12:47:59+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2021-07-19T00:00:00+00:00","index":1,"fulltext":"Recommendation: Minor Revision\nForm responses:\n---\n\nComments to Author:\n---\nThank you for inviting me to review this interesting paper. Overall, this is a well-written review protocol on incorporating and evaluating citizen engagement in health research. The key elements of the planned scoping review are clearly justified, and contributors should be congratulated. Please, see some points of clarification/corrections considering planned methods, but also journal reporting standards.\n\nAbstract\nPage 2. Lines 40 and 41. Please, state planned date limits. For example: \"from inception onwards\".\nPage 3. Line 53. Systematic review registration. The authors' state: \"Not applicable\". Please, revise. Systematic Reviews encourages prospective registration of (scoping) review protocols in the Open Science Framework, or other relevant platforms. Please, pre-register a revised version of this protocol and provide the name of the registry (such as Open Science Framework) and registration number. For example: \"Systematic Review Registration: Open Science Framework (osf/io/xxxx).\" Thank you.\n\nBackground\nPage 5. Please, describe the rationale for the review in the context of what is already known (e.g. previous reviews and studies on citizen engagement in health research). Explain why the review questions/objectives lend themselves to a (scoping) review approach. The background of a scoping review protocol should be comprehensive and cover the main topic elements, important definitions, and existing knowledge in the field. See examples in PRISMA-ScR E\u0026E document, and/or examples of scoping review protocols published in the journal. Thank you.\n\nMethods.\nPage 5. Lines 93-97. Please, include a first paragraph including information related to protocol reporting/registration. Systematic Reviews encourages prospective registration of scoping and other types of review in other relevant registration platforms (e.g. Open Science Framework). Please, register a revised version of this study protocol, and provide registration information, including the registration number, and reporting information. See comments on Abstract.\nFor example: \" This scoping review will be conducted according to recommendations specified by Arksey and O'Malley (14), and enhanced by Levac and colleagues (15). The review protocol has been registered within the Open Science Framework database (registration number: [please include]) and is being reported in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) statement [include reference] (see checklists in Additional file 1). The proposed scoping review will be reported in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) extension for Scoping Reviews (PRISMA-ScR) [include reference].\"\nIn lines 94 and 95, the authors' state: \"The conduct and reporting of this scoping review will be guided by PRISMA-ScR\". To clarify, PRISMA is a reporting guideline, not a guideline for review conduct. Please, delete/revise. Thank you.\nPage 5. Line 100. Please, delete: \"as recommended by Arksey and O'Malley (14).\".\nPage 6. Line 118. Please, delete: \"As recommended by Arksey \u0026 O'Malley and Levac and colleagues (14, 15),\". Thank you. This is reiterative.\nPage 7. Line 137. Please, revise as: \"Additional file 2\". Thank you.\nPage 8. Line 157. Please, provide a rationale/justification for: \"(…)to the year 2000 will be excluded\". Thank you.\nPage 9. Line 172 and 173. Please, delete: \"according to Arksey \u0026 O'Malley and Levac's recommendations (14, 15)\".\nPage 9. Lines 180-183. Please, be more explicit. Authors should list and define all variables for which data will be sought, including any pre-planned data assumptions and simplifications. Thank you.\nPage 9. Line 186. Please, delete: \"according to the Arksey \u0026 O'Malley and Levac guidelines, which recommend\".\nPage 9. Lines 186-188, and p.10 lines 189-201. Please, be more explicit. Authors should describe the methods of handling and summarizing the data that will be charted (e.g. data will be synthesised with descriptive statistics? Any specific graphical/visual mapping is anticipated?). Authors should clearly describe how the evidence will be presented: in a narrative format, table, or visual representation, including a map or diagram. Please, revise (see reporting examples in PRISMA-ScR E\u0026E document).\nPage 10. Lines 208-222. Please, delete.\n\nDiscussion\nPage 11. Please, include brief discussion of any practical or operational issues involved in performing the study you anticipate and any issues not covered in other sections (e.g. data-sharing?).\nPage 12. Please, discuss potential limitations at source of evidence level, and at scoping review level you anticipate\nPage 12. Please, include information on potential amendments to this protocol. For example: \"Any amendments made to this protocol when conducting the study will be outlined in the Open Science Framework and reported in the final manuscript\".\n\nReferences\nPlease, revise/update considering the comments above.\n\nAdditional files\nPlease, revise/update considering the comments above.\nPlease, include PRISMA-P populated checklist as new Additional file 1. Thank you.\nPlease, include draft search for MEDLINE as new Additional file 2. Thank you.\n* Publons Reviewer Recognition. Springer Nature can send verification of this review directly to Publons (a subsidiary of Clarivate Analytics). If you would like to take advantage of this service, please click on the “Yes” option below. Your name, email address, title of the reviewed manuscript, name of the journal, and date of your review submission (the “Review Data”) will then be transmitted to Publons after the final decision on the manuscript has been made. If you have already registered at Publons, they will notify you of the receipt of this review and update your profile as per your settings and their policy. If you are not registered with Publons, you will receive an email from them asking you to register in order for them to be able to recognize your review on your new profile page. Publons may use the Review Data to generate derivative metadata for the benefit of Publons and you as a reviewer, carefully considering the sensitivity of such information. For example, Publons may verify your record as a reviewer by updating your profile published on its webservice if you have registered for such service or help editors to identify candidate reviewers. Please find the details of processing in Publons’ privacy policy https://publons.com/about/terms: **Yes**\n* Level of interest: **An article whose findings are important to those with closely related research interests**\n* Quality of written English: **Needs some language corrections before being published**\n* Declaration of competing interests: **I declare that I have no competing interests.**\n* I agree to the open peer review policy of the journal. I understand that my name will be included on my report to the authors and, if the manuscript is accepted for publication, my named report including any attachments I upload will be posted on the website along with the authors' responses. I agree for my report to be made available under an Open Access Creative Commons CC-BY license (http://creativecommons.org/licenses/by/4.0/). I understand that any comments which I do not wish to be included in my named report can be included as confidential comments to the editors, which will not be published.: **\nI agree to the open peer review policy of the journal**\n* Were you mentored through this peer review?: **No**\n"},{"type":"editorAssigned","content":"","date":"2021-07-02T00:00:00+00:00","index":"","fulltext":""},{"type":"reviewersInvited","content":"","date":"2021-07-02T00:00:00+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2021-07-02T00:00:00+00:00","index":1,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2021-07-02T00:00:00+00:00","index":0,"fulltext":""},{"type":"editorInvited","content":"","date":"2021-07-01T23:00:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2021-02-06T12:54:51+00:00","index":"","fulltext":""},{"type":"submitted","content":"Systematic Reviews","date":"2021-01-29T12:31:24+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"systematic-reviews","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sysr","sideBox":"Learn more about [Systematic Reviews](http://systematicreviewsjournal.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/sysr/default.aspx","title":"Systematic Reviews","twitterHandle":"@MedicalEvidence","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6d953046-c4d7-4145-9dfb-079ed880f34a","owner":[],"postedDate":"February 10th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":2352021,"name":"Health Economics \u0026 Outcomes Research"}],"tags":[],"updatedAt":"2021-09-12T13:46:47+00:00","versionOfRecord":[],"versionCreatedAt":"2021-02-10 19:38:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-185385","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-185385","identity":"rs-185385","version":["v1"]},"buildId":"FbvkV6FR0MCFSLy54lSbu","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}