Can artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey

Can artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Can artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey Carole Lunny, Sera Whitelaw, Emma K Reid, Yuan Chi, Jia He Zhang, and 10 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2416773/v2 This work is licensed under a CC BY 4.0 License Status: Posted Version 2 posted You are reading this latest preprint version Show more versions Abstract Background Systematic reviews (SRs) are being published at an accelerated rate. Decision makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision makers (e.g., practitioners, policymakers, researchers) use SRs to inform decision making, and to explore the role of a proposed AI tool to assist in critical appraisal and choosing amongst SRs. Methods We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks. Results Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students, and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR amongst multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%), or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs. Conclusions Respondents often sought out SRs as a source of evidence in their decision making, and often encountered more than one SR on a given topic of interest. Many decision makers struggled to choose the most trustworthy SR amongst multiple, related to a lack of time and difficulty comparing SRs varying in methodological quality. An AI tool to facilitate comparison of the relevance of SRs, the search, and methodological quality, would help users efficiently choose amongst SRs and make healthcare decisions. Epidemiology systematic reviews evidence synthesis implementation science decision makers knowledge translation research methodology Figures Figure 1 Figure 2 Strengths and limitations of this study Our study was conducted in accordance with our protocol published a priori. We attempted to maximize survey reach and responses by using emails and social media distribution. However, our email response rate was low (<10%) and due to the nature of social media advertising, we were unable to calculate a true survey response rate. Our targeted emails and social media advertising may have missed important decision makers that use systematic reviews. Individuals involved in guideline development and policymaking may have been more likely to respond to the survey. Response bias is a factor as survey respondents working in higher income countries were over-represented. Background Evidence-informed healthcare requires decision makers to identify evidence, appraise its methodological quality and relevance, and apply it to a particular practice scenario [1]. Systematic reviews (SRs) are used to collate evidence from primary studies (e.g., randomized controlled trials, cohort studies) which are appraised and synthesised using systematic methods [2]. Methodologically sound SRs that are well-reported and with a low risk of bias are widely regarded as a gold standard for healthcare decision making. Each year there is an exponential rise in the volume of research produced in healthcare [3], including SRs [4-6]. Between 2000 and 2019, the number of SRs produced annually increased twenty-fold, with 80 SRs published per day by 2019 [7]. Along with the increasing prevalence of SRs overall, there has been an increase in the number of duplicated SRs with the same or similar research questions and eligibility criteria [8, 9]. Between 2000 and 2020, approximately 1200 and 1600 systematic review clusters (i.e., duplicated SRs) addressing the same clinical, public health, or policy questions were identified by two bibliometric studies [10, 11]. Duplicated publications increased over time, with the highest increase occurring in the most recent five-year period (2016 to 2020). Despite similar research questions and focus, duplicated SRs may report discordant results and conclusions. Research groups have studied overlapping SRs in attempt to examine and identify sources of discordance [12-14] and approaches for managing discordant SR findings [15,16]. In 1998, Jadad et al published an algorithm tool [12] to help users select the “best evidence” review amongst multiple discordant SRs [13-15]. To assess reproducibility of the algorithm, our research group performed a replication study that compared the findings of 21 publications that used the Jadad tool to choose one or more SRs as best evidence with our own independent Jadad assessment [16]. In 62% of cases, replication was unsuccessful, and a different, higher methodological quality SR was chosen by our group. Sources of discrepancies included different PICO (Population, Intervention, Comparison, Outcome) eligibility criteria, databases searched, primary studies, and/or analysis methods [16]. These studies highlight the expertise required by experienced researchers to manually assess and compare similar SRs that differ across their results and conclusions. At this time, it is unknown whether decision makers, such as practitioners and policymakers, struggle with comparing and/or choosing SRs when there are multiple on the same topic, and what barriers, if any, are faced. There is also value in learning which variables or features of SRs are considered most important when comparing multiple on the same topic. Keeping abreast of the latest research on a topic is already a monumental task [5], so we proposed that a tool that incorporates artificial intelligence (AI) to help navigate a growing body of literature would be beneficial and could increase efficiencies. The purpose of this study was to explore the role of a proposed AI-informed, evidence-based tool to help healthcare decision makers appraise and choose amongst SRs for real-world practice. To understand current need, we surveyed decision makers to determine how they use SRs to inform their decisions, and how they choose the best SR evidence when there are multiple SRs addressing the same clinical question. Methods Protocol The study protocol can be found on the Open Science Framework at https://osf.io/nbcta/ . Approval from the University British Columbia Ethics Board was obtained in the form of an exemption (ID H20-02013). The reporting of this survey is in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES; Appendix A ) [17]. Important definitions are found in Box 1 . Box 1. Important definitions Decision maker We define “decision makers” as individuals who are likely to be able to use research results to make informed decisions about health policies, programs and/or clinical practices [18]. A decision maker can be, but is not limited to, a health practitioner, a policymaker, an educator, a health care administrator, a community leader or an individual in a health charity, patient group, private sector organization or media outlet [18]. The following groups were considered decision makers: • Health practitioner (individuals that provide care, e.g., nurses, physicians, pharmacists, mental health counsellors, community-based workers) • Patient, caregiver, family member, patient and consumer advocacy organization representative, community leader • Policymaker (government representative, public funding agency representative, healthcare/hospital administrator, Clinical Practice Guideline developer, Health Technology Assessment [HTA] developer) • Educator • Industry representative (e.g., drug/device manufacturers) • Researcher and/or academic • Information scientist/medical librarian • Journal editor, publishers, news media • Student, trainee, postdoctoral fellow, graduate student/post graduate trainee/undergoing practicum in a clinical program or focused on health policy or research Evidence-informed decision-making Evidence-informed decision-making stresses that the best available evidence from research should inform decisions, as well as other factors such as context, public opinion, equity, feasibility of implementation, affordability, sustainability, and acceptability to stakeholders. It is a systematic and transparent approach that applies structured and replicable methods to identify, appraise, and make use of evidence across decision-making processes, including for implementation. Systematic review A systematic review attempts to collate all study-specific evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimising bias, thus providing more reliable findings from which conclusions can be drawn and decisions made [2]. Meta-analysis Traditional meta-analysis is a statistical method to combine the results from two or more primary studies (e.g. randomised controlled trials, cohort studies), to produce a point estimate of an effect and measures of the precision of that estimate [2]. Network meta-analysis (NMA) “Any set of studies that links three or more interventions via direct comparisons forms a network of interventions. In a network of interventions there can be multiple ways to make indirect comparisons between the interventions. These are comparisons that have not been made directly within studies, and they can be estimated using mathematical combinations of the direct intervention effect estimates available [2].” A network is composed by at least three nodes (interventions or comparators) and these are connected (graphically depicted as lines/edges) when at least one study compares the underlying two interventions - the direct comparisons. Reviews that intend to compare multiple treatments with an NMA but then find that the expectations or assumptions are violated (e.g. the network is ‘disconnected’, studies are too heterogeneous to combine, underlying assumptions of the method are not met), and hence an NMA is not possible or optimal, are also considered in our definition [19] . Discordance Discordance is when SRs with similar public health, or policy eligibility criteria (as expressed in PICO) report different results or conclusions for the same outcome. We define discordant results as differences based on the methodological decisions SR authors make, or different interpretations or judgments about these results [16]. Risk of bias assessment in the systematic review level A risk of bias assessment evaluates limitations in the way in which the results were planned, analysed, and presented. If these methods are inappropriate, the validity of the findings can be compromised. Bias may also be introduced when interpreting the results to draw conclusions. Conclusions may include ‘spin’ (e.g. biased mis-representation of the evidence, perhaps to facilitate publication) or (erroneous) mis-interpretation of the evidence [20]. Ideally, potential biases identified in the results of the SR might be acknowledged and addressed appropriately when drawing conclusions [21]. Similarly, a well conducted SR draws conclusions that are appropriate to the included evidence and therefore is free of bias even when the primary studies included in the review have high risk of bias. On the basis of the risk of bias assessment, supported by balanced reporting of SR/meta-analysis (MA) interpretation of findings, relevance of included studies to the SR/MA’ question, a final consideration is performed on whether the SR/MA as a whole is at ‘low’, ‘high’, or ‘unclear’ risk of bias. Reporting comprehensiveness The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) standard encourages reporting completeness or comprehensiveness when authors write up the results of their SRs prior to publication [22]. Often a PRISMA checklist is required when submitting a systematic review to a peer reviewed journal for consideration. A review can be well conducted, but poorly reported; or poorly conducted, but well reported (even if methods were poor). The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network [23]is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines. Quality of conduct (systematic reviews) Methodological quality is about how well the research is conducted according to established guidance (e.g. Cochrane Handbook [2], JBI Manual ). The Assessing the Methodological Quality of SRs (AMSTAR) measurement tool was designed to appraise the quality of conduct of SRs [24]. AMSTAR has been validated and proven popular as a simple means of assessing the quality of reviews [25, 26]. A recently updated version (AMSTAR-2) was published in 2017 [27]. Certainty of the evidence assessment An assessment of the certainty of evidence is defined as any of evaluation of the strength of the evidence such as the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach [28], criteria for credibility assessment, and other approaches used to grade the overall body of the evidence. GRADE is a well-established approach to assess the certainty of evidence based on the following criteria: risk of bias of the primary studies, imprecision, indirectness, inconsistency, and publication bias. GRADE is designed for assessing the certainty of the evidence deriving from primary studies. Survey Design Our investigative team used a cross-sectional survey design, informed by established approaches for conducting needs assessments and the Dillman approach for conducting online surveys [29]. The survey was conducted using Qualtrics (Qualtrics Labs, Provo, UT, USA) [30, 31]. No incentive or compensation was offered to respondents. Survey responses were anonymous. Personal identifying information was only collected on a voluntary basis from respondents who wished to be contacted about the survey’s results. Informed consent was implied when participants ticked a consent box on the first survey page. We created an English-language survey with 21 questions, primarily closed-ended in nature (full survey questions in Appendix B ). The survey questions were sub-divided into three parts: (a) demographics, (b) experiences and barriers to choosing SRs when more than one exists on the same topic, and (c) data elements to consider when choosing the SRs from multiple on the same topic. Respondents were allowed to skip questions they did not wish to answer and were able to review and change their answers prior to submitting their responses. We were particularly interested in gaining insight on the specific features a SR decision makers would consider when comparing and selecting amongst multiple SRs, as these features would be crucial to inform priority areas for an AI tool. We recognized the potential for “leading” or influencing survey respondents by providing a curated list of SR features our steering group deemed relevant in advance of their independent responses. To protect against this influence, we created two similar questions (Q11 and 12a-c), which were randomly allocated to half of the respondents: the first (Q11) was a multiple-choice drop-down list of elements to choose from, and the second (Q12a-c), included a short case study summarizing the characteristics, methods, and results from three similar SRs on the same topic, ultimately asking the respondent to choose which SR(s) they would use to inform a discussion with a patient, and which features of the SR led to their decision. Eight academics piloted the survey and modified it iteratively to improve clarity, face validity, and content validity. Sample size The sample size was calculated to detect mean difference of 50% between two factor levels with 90% power [32]. To detect this difference, a sample of 440 decision makers was required, assuming a standard deviation of 1.6 points (based on similar surveys [33, 34]) and a 5% significance level. We assumed that contacting quadruple (i.e. 1760) the number of decision makers would be sufficient to recruit the required number (assuming a 25% response rate), allowing for failed email addresses and non-response. Distribution of the Survey We aimed to survey individuals from organizations or institutions which produce SRs, as well as decision makers of all types who use SRs. We developed an email list of SR-producing groups including Cochrane Multiple Treatments Methods Group, Guidelines International Network, JBI (formerly the Joanna Briggs Institute), Campbell Collaboration, U.S. Agency for Healthcare Research & Quality’s Evidence-Based Practice Centre program, Centre for Reviews and Dissemination, Canadian Agency for Drugs and Technologies in Health [CADTH], Evidence for Policy and Practice Information and Co-ordinating Centre [EPPI-Centre], Clinical Epidemiology program at the Ottawa Hospital Research Institute, the GRADE group). These potential survey participants were sent an email describing the purpose of the study, requesting their participation, and providing a link to the survey. We leveraged the professional contacts from our steering committee and distributed the survey to an additional 28 organisations anonymously ( Appendix C) . We also included participants from a UBC Methods Speaker Series on evidence synthesis methods ( https://www.ti.ubc.ca/2023/01/10/methods-speaker-series-2023/ ). In addition, we advertised through the e-newsletters of Knowledge Translation Canada, SPOR (Strategy for Patient-Oriented Research) Evidence Alliance and Therapeutics Initiative. We also contacted professional healthcare organisations (e.g. Canadian Association for Physiotherapists) to promote the survey in their newsletters. A distribution plan was followed to disseminate and advertise the survey. The Dillman approach [29] suggests repeated contact to boost responses, which we followed by sending out three reminders to email recipients, and repeated advertisement through social media outlets. Anonymous links were included in LinkedIn and Twitter posts which were circulated through targeted Twitter accounts, such as the Knowledge Translation Program, SPOR Evidence Alliance, and the Therapeutics Initiative. Tweets were retweeted amongst followers. We used twitter cards (i.e. advertisements with pictures) and targeted hashtags to increase awareness of the survey. We also advertised through two LinkedIn accounts. Timing The survey ran from July 19 to August 19, 2022. Qualtrics email reminders were scheduled at two-week intervals throughout this period to unfinished or non-respondents. We estimated that the survey would take approximately 10 minutes of a respondent’s time. Data analysis Prior to data analysis, the responses were transferred from Qualtrics to MS Excel. Questionnaires that were terminated before completion, were included in analyses, but those that were entirely blank were excluded. We measured the time respondents took to fill in a questionnaire regardless of whether it was complete. Descriptive statistics were calculated for each closed response question, including count, frequency, with denominators taken as the number who provided a response to the question. One researcher coded responses to the open-ended questions or comments independently by identifying themes. The free-form responses were then presented descriptively as counts and frequencies in an identical fashion to closed responses. We stratified the analysis by type of decision maker as presented in Box 1. When a respondent indicated that they were more than one type of decision maker, we calculated the response for all their respondent types. For example, if they identified as both a practitioner and researcher, and responded yes to question 1, we counted a yes for both practitioner and researcher types. We compared the results of the two randomly presented questions. Results Recruitment results A total of 3158 email invitations were sent to advertise the survey. Of these, 197 emails failed to reach the recipients due to incorrect addresses, no longer at the related job post, etc., resulting in a total of 2,961 email invitations successfully delivered ( Figure 1 ). After consolidating duplicates (n = 25) and blank responses (n = 83), a total of 684 survey responses were included in the analysis. Most respondents completed the survey by clicking and completing an anonymous link distributed over social media and e-newsletters (n = 450), compared to those who responded through the Qualtrics email link (n = 234). As per our sample size calculation, we expected a 25% response rate to email invitations but only achieved 7.9%. Of the 684 respondents, 462 (67.5%) answered all the survey questions, and 97 (14.2%) completed less than 50% of the questions. For those who completed the survey, the median response time was 11 minutes. Demographics and characteristics of respondents The majority of surveyed decision makers identified as researchers (58.2%), practitioners (37.1%), students/trainees (19.2%), and policymakers (13.5%), and many respondents identified as more than one role. For example, of the 62 (13.5%) policymakers, 37 (59.7%) also identified as a researcher, 14 (22.6%) as a practitioner, 9 (14.5%) as a journal editor, and 4 (6.4%) as a patient. The majority of respondents lived in North America (52.8%) and Europe (34.2%). When comparing survey responses by role we focus here on those categorized as researchers, practitioners, and policymakers, as these were well-represented decision maker subgroups who serve unique knowledge user roles. Full characteristics of respondents are summarized in Appendix D, Table 1 . The vast majority of respondents reported they were familiar with SRs (621/684 [90.8%]). Two survey questions were included to further gauge the respondents’ understanding of SRs ( Appendix D, Table 2 Q2-3 ). The first question asked if PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) was used to assess the methodological quality of a SR. This question was ambiguous as PRISMA is a reporting checklist to determine the comprehensiveness of reporting of a published SR manuscript (and not the methodological quality). However, the majority (335/542 [61.8%]) correctly agreed that PRISMA was not used to assess methodological quality [of conduct] of a SR. The second question asked respondents to identify limitations that occurred at the level of the SR. Slightly more than half (308/547 [56.3%)] correctly answered that all options except risk of bias of the primary studies applied. More than a third (199/547 [36.4%]) incorrectly answered that all but selective reporting of results and analyses applied. Experiences and barriers to choosing SRs when more than one exists on the same topic Respondents (n=558) often (64.5%) or sometimes (32.6%) sought out SRs as a source of evidence in their decision making (never did=2.9%; Appendix D, Table 3 ). Respondents (n=538) reported facing a situation where they found more than one SR on a given topic sometimes (54.8%) or often (43.1%) ( Table 1 Q6 ). Just over half (50.8%) responded that they struggled to choose the most valid and trustworthy SR amongst multiple SRs on the same topic ( Table 1 Q10 ). Overall, the most common barrier to making this decision was reported to be a lack of time to fully read and evaluate each SR (55.2%) ( Table 1 Q10 ). Other frequent barriers were related to variation in the quality of conduct of SRs (54.2%), differences in results and conclusions across SRs (49.7%), variation in the primary studies included in the SRs (44.6%), and slightly different clinical focuses of the SRs (43.1%) (Table 1 Q10) . Of interest, when asked why they struggled to pick one SR, 35.6% (180/505) of respondents said it was because there was insufficient data from titles and abstracts to assess relevance to their question. Additionally, 27% of respondents recognized their inexperience assessing the methodological quality of SRs as being a barrier. The reported approach to choosing the most appropriate SR tended to vary, depending on the type of decision maker ( Table 1 Q8 ): Practitioners most often (75/170 [44.1%]) chose the most recently published SR(s) that were relevant to their topic. About one quarter (45/170 [26.4%]) found as many as they could that were relevant to their topic and then reviewed them all, and one tenth (18/170 [10.6%]) chose the SR from the highest impact factor journal. Policymakers (34/62 [54.8%]) and researchers (111/266 [41.7%]) most often reported finding as many SRs as possible relevant to their topic of interest, and reviewed them all. The majority of decision makers (335/385 [87.0%]) responded that they would use a free, automated, AI-informed, evidence-based online tool to assist in choosing the best SR(s) among multiple on the same question, if it were available ( Figure 2 ). Of respondent subgroups, practitioners reported the highest interest (90.5%), followed by researchers (86.9%), then policymakers (79.5%). Data elements to consider when choosing the SRs best suited to my needs from multiple on the same topic The two survey questions asking respondents to identify important SR features either via a pre-populated list (Q11) or determined through free-form response to a case study (Q12a-c) were randomised to 274 and 278 respondents, and answered by 274 and 186 respondents, respectively. See Appendix D Table 4 for complete answers. Q11 – pre-defined features provided via drop-down menu The relevance of the SR’s research question to the respondent’s clinical question or learning needs was selected most frequently as an important feature (83.6%), followed by the methodological quality and reproducibility of the SR (79.2%). The SR search strategy was also identified a key consideration, with recency of the SR search date (74.8%), and comprehensive search strategy (69.0%) frequently selected. Other features more than half of respondents recognized as important to consider were: (i) the relevance of clinical outcomes (65.7%), (ii) having a risk of bias assessment conducted for primary studies (60.9%), (iii) a published protocol or pre-registration for the SR (59.1%), and (iv) consideration for the types of studies included (i.e. randomized controlled trials versus non-randomized) (55.1%). Q12 – free-form responses identifying features considered when choosing between SRs relevant to a case The case study was based on the clinical question “Is acupuncture effective/efficacious and safe for women with primary dysmenorrhoea.” The PICO, characteristics, and features of three SRs (Lui 2017 [35]; Yu 2017 [36]; Woo 2018 [37]) were presented in a table. Respondents chose the SR by Woo 2018 (86.6%) most frequently based on its strengths and weaknesses ( Appendix D, Figure 4 Q12a ). Four out of ten policymakers indicated that they would use all three (44.8%) in their decision making. When asked which criteria helped the respondents decide, the most common response involved using a hierarchy of features, not just one feature, to choose between SRs (or elements of SRs) (73.1%). The next most common consideration was the number of studies or patients included (45.2%). This response was most common amongst practitioners, about two-thirds, compared with one third of researchers, and one tenth of policymakers. The recency of the SR search date (29.6%), the risk of bias of the primary studies being assessed (28.5%), and the methodological quality of the SR (26.3%) were the next most common responses. Additional features identified by free-form responses and not included in the pre-populated elements (Q11) were: heterogeneity, and the process for selecting, extracting and assessing studies. Table 1: Considerations when there are multiple SRs on the same topic Item Responses ALL Policymaker Practitioner Researcher Q6. How often have you faced a situation where you find more than one SR on a given topic of interest to you? Never (n=538) (n=62) (n=167) (n=266) 12 (2.2%) 0 (0%) 4 (2.3%) 5 (1.9%) Sometimes 295 (54.8%) 30 (48.3%) 106 (63.4%) 123 (46.2%) Often 232 (43.1%) 31 (50.0%) 57 (34.1%) 138 (51.9%) Q8. When you encounter multiple SRs on the same topic how do you choose the one(s) most likely to address your clinical/public health/policy question or your learning needs? I typically choose the first one I find that is relevant to my topic (n=552) (n=62) (n=170) (n=266) 13 (2.4%) 1 (1.6%) 2 (1.1%) 4 (1.5%) I find as many as I can that are relevant to my topic and then review them all 207 (37.5%) 34 (54.8%) 45 (26.4%) 111 (41.7%) I typically choose the most recently published one(s) that are relevant to my topic 171 (31.0%) 7 (11.2%) 75 (44.1%) 60 (22.6%) I typically choose the one from the highest impact factor journal 34 (6.2%) 1 (1.6%) 18 (10.6%) 10 (3.8%) Q9. When you have encountered multiple SRs on the same topic, which of the following statements resonates most with you? I can usually identify the SR(s) best suited to my needs (n=548) (n=62) (n=170) (n=264) 268 (48.9%) 35 (56.4%) 56 (32.9%) 152 (57.6%) I sometimes struggle to identify the SR(s) that are best suited to my needs 238 (43.4%) 24 (38.7%) 94 (55.2%) 101 (38.2%) I often struggle to identify the SR(s) best suited to my needs 41 (7.4%) 3 (4.8%) 19 (11.1%) 10 (3.8%) Q10. If/when you struggle to choose the SR(s) best suited to your needs, the barriers to you being able to make this decision are Insufficient data from titles and abstracts to assess relevance to my question (n=505) (n=60) (n=165) (n=251) 180 (35.6%) 21 (35.0%) 55 (33.3%) 77 (30.7%) Inexperience with assessing the methodological quality of (or biases in) SRs 140 (27.7%) 9 (15%) 72 (43.6%) 30 (12.0%) Not enough time to read each SR in full to evaluate all the options 279 (55.2%) 23 (38.3%) 110 (66.7%) 119 (47.4%) You don't trust the conclusions 56 (11.0%) 11 (18.3%) 21 (12.7%) 34 (13.5%) Different results and conclusions across the SRs 251 (49.7%) 28 (46.7%) 94 (57.0%) 130 (51.8%) Variation in the quality of how the SRs were conducted 274 (54.2%) 34 (56.7%) 79 (47.9%) 146 (58.1%) Variation in searches across the SRs 172 (34.0%) 23 (38.3%) 46 (27.9%) 95 (37.8%) Variation in included primary studies across the SRs 225 (44.6%) 30 (50.0%) 63 (38.1%) 120 (47.8%) Variation in how across the SRs results were synthesized 194 (38.4%) 28 (46.7%) 51 (30.9%) 106 (42.2%) Slightly different clinical focus between SRs 218 (43.1%) 27 (45.0%) 74 (44.8%) 107 (42.6%) *Numbers do not add up to 100% because respondents may have chosen more than one response option, and the majority of respondents identified as more than one type of decision maker (e.g. researcher and patient) Discussion We surveyed policymakers, practitioners, researchers, learners and other respondent types to understand how they use SRs to inform decision making, and how they compare and select one or more SRs when there are multiple addressing the same clinical, public health or policy question. These individuals, who demonstrated good baseline knowledge of SRs, often sought out SRs as a source of evidence in their work and decision making. They were also frequently faced with a situation where they find more than one SR on a given topic of interest. Nearly half of all respondents have struggled to choose the most valid and trustworthy SR, most often due to lack of time, and owing to varying methodological quality of identified SRs, variability in the primary studies included, and differences in their results and conclusions. The proposed use of an AI tool to assist in comparing multiple SRs on the same topic was well-accepted by survey respondents. When comparing SRs on the same topic, themes of important characteristics were related to the relevance of the SR to their PICO question of interest, the robustness of the literature search and included studies, the recency, and the methodological quality. However, the answers varied by type of decision maker, where healthcare practitioners more often chose the most recently published review(s) relevant to their topic, and policymakers and researchers most often reviewed all the relevant SRs on their topic of interest based on a hierarchy of criteria. This may indicate that practitioners are happy with a decision support tool to compare features of SRs which presents the “bottom line” synthesis or ranking of the SR evidence, but policymakers and researchers want the distilled information of all the presented reviews from which to make their own methodological judgments. Another identified theme was that there is usually not one single best SR to ultimately choose. While a review may have a good AMSTAR-2 quality rating [27], it still may contain important flaws like failing to report patient important outcomes (e.g. adverse events). It may also miss data and information relevant to the decision maker. The three SRs in our case study contained different primary studies and publication dates, making their comparison especially difficult. To get the most out of the data for their decision making, respondents (especially policymakers) commented that they often would review and read all the SRs on a topic and assess the strengths and weaknesses before making any decisions. Strengths and limitations A strength of our research was that we conducted it in accordance with an a priori published protocol. We combined newsletter, email distribution lists, and social media to reach a wide range of decision makers from across the globe. We attempted to maximise the response rate by sending email reminders and repeating messages through social media. Social media circulation as a distribution method proved fruitful although we had no control over exposure. This is the first study to our knowledge that explores this topic. A limitation was that we were expecting a 25% response rate to email invitations but achieved <10%. Another limitation is that we were unable to calculate a true survey response rate since two thirds (n=450) responded through social media links which were anonymous. Survey fatigue is a significant issue with this form of research and a multimodal approach to maximize reach, even at the expense of being able to calculate a response rate, was deemed necessary. Response bias in our sample was also a major limitation as decision makers working in higher income countries were over-represented. Additionally, the sample represented in this survey, namely individuals involved in guideline development and policymaking, may have been more likely to have responded. Another limitation is that our targeted emails and social media advertisement may have missed important decision makers that use systematic reviews. Implication for practice, policy and knowledge translation With the number of SRs on the same topic growing exponentially each year, we can predict that the challenge of decision makers struggling to compare and choose between SR evidence will continue to increase. Our survey suggested that currently, over one third of decision makers reviewed only the data in the title and abstract when making their choice of SR (as opposed to the full text) when faced with clinical or policy decisions. Titles and abstracts may not contain enough information to make informed choices between SRs, and the full text publications should be sought to enable methodological quality/risk of bias assessment and a full comparison of the strengths and weaknesses across SRs on the same topic. If a health care decision is informed by SR evidence that is low methodological quality and where inappropriate methods were used, this risks negative patient care outcomes [5, 38-40]. An example of misleading results from SRs with meta-analysis is when ivermectin, an antiparasitic medication, was widely promoted across the world for preventing and treating COVID-19 [39, 41-43]. A search (Aug 5, 2022) for Ivermectin for COVID-19 yielded 149 SRs in the PubMed database. Many of the SRs are discordant [39] and vary in methodological quality. A few of these meta-analyses were found to contain impossible numbers, unexplainable cohort mismatches, inconsistent timepoints, and substantial methodological flaws. One of these meta-analyses, submitted as a preprint (https://www.researchsquare.com/article/rs-100956/v2), has since been withdrawn, whereas a published meta-analysis ( https://academic.oup.com/ofid/article/8/11/ofab358/6316214 ) has been retracted after it was found to include fraudulent data (note that we intentionally omitted a formal reference to these studies). Despite these serious concerns, millions of doses of ivermectin have already been given to treat or prevent COVID-19 globally, potentially risking unnecessary toxicity and depleting supply where it is otherwise indicated. The biased results from these and other poorly designed and reported SRs can mislead decision-making at all levels [44-46]. A user-friendly tool which would efficiently highlight SR methodology weaknesses would help facilitate efficient and appropriately informed health care decisions. Future research We have identified a role for an AI tool to help decision makers more efficiently compare and choose SR evidence, and preferred SR features be the focus of the tool. Preferences for AI assistance appears to vary depending on the type of decision-maker, and we strive to develop a tool that is suitable to the needs of all users. The WISEST AI tool, in development, will present the scope, strengths and limitations of all the SRs found on the users’ topic, as well as a methodological quality ranking [47]. Users can then compare the different SRs and make their own clinical and methodological judgment about which SRs are most suited to their needs. The WISEST AI tool will aim to provide a “supporting” role for decision makers when comparing and choosing between reviews and not a substitution role [47]. Our survey identified that there is a considerable proportion of SR users who feel they have inadequate assessment skills to adequately assess SR methodological quality. The availability of the WISEST AI tool will help distill relevant SR information but is not intended to replace critical assessment or judgment. Appropriate training coupled with the user-friendly tool is necessary to overcome this knowledge gap. Conclusions Policymakers, practitioners and researchers often sought out SRs as a source of evidence in their decision making, and often encountered more than one SR on a given topic of interest. Just over half of those surveyed struggled to choose the most valid and trustworthy SR amongst multiple. These difficulties related to lack of time, and difficulty comparing different SRs when they vary in methodological quality and characteristics. When comparing SRs on the same topic, relevance to the question of interest, robustness and recency of the search, and methodological quality of the SR were most important to respondents. The development and implementation of an AI tool to rapidly highlight the features, strengths and weaknesses of SRs will help address these challenges and facilitate healthcare decision making. Declarations Funding source We received no funding to conduct this survey. Conflict of interest We have no conflicts of interest to declare with regards to this survey and project. Patient involvement Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research. Declarations Ethics was obtained from the University of British Columbia Behavioural Research Ethics Board (ID H20-02013). Data sharing and availability of data and materials All data is contained in the manuscript and appendices. SUPPLEMENTAL FILES Appendices A-D References 1. Sackett, D.L., Evidence-based medicine. Semin Perinatol, 1997. 21 (1): p. 3-5. 2. Higgins, J.P., et al., Cochrane handbook for systematic reviews of interventions . 2019: John Wiley & Sons. 3. Bornmann, L. and R. Mutz, Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology, 2015. 66 (11): p. 2215-2222. 4. Bastian, H., P. Glasziou, and I. Chalmers, Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS medicine, 2010. 7 (9): p. e1000326. 5. Ioannidis, J.P., The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q, 2016. 94 (3): p. 485-514. 6. Taito, S., et al., Assessment of the Publication Trends of COVID-19 Systematic Reviews and Randomized Controlled Trials. Annals of Clinical Epidemiology, 2021. 3 (2): p. 56-58. 7. Hoffmann, F., et al., Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin Epidemiol, 2021. 138 : p. 1-11. 8. Page, M.J. and D. Moher, Mass production of systematic reviews and meta‐analyses: an exercise in mega‐silliness? The Milbank Quarterly, 2016. 94 (3): p. 515. 9. Moher, D., The problem of duplicate systematic reviews . 2013, British Medical Journal Publishing Group. 10. Lunny, C., et al., Bibliometric study of 'overviews of systematic reviews' of health interventions: Evaluation of prevalence, citation and journal impact factor. Res Synth Methods, 2022. 13 (1): p. 109-120. 11. Bougioukas, K.I., et al., Global mapping of overviews of systematic reviews in healthcare published between 2000 and 2020: a bibliometric analysis. Journal of Clinical Epidemiology, 2021. 137 : p. 58-72. 12. Jadad, A.R., D.J. Cook, and G.P. Browman, A guide to interpreting discordant systematic reviews. Cmaj, 1997. 156 (10): p. 1411-6. 13. Li, Q., et al., Minimally invasive versus open surgery for acute Achilles tendon rupture: a systematic review of overlapping meta-analyses. J Orthop Surg Res, 2016. 11 (1): p. 65. 14. Mascarenhas, R., et al., Is double-row rotator cuff repair clinically superior to single-row rotator cuff repair: a systematic review of overlapping meta-analyses. Arthroscopy, 2014. 30 (9): p. 1156-65. 15. Zhao, J.G., J. Wang, and L. Long, Surgical Versus Conservative Treatments for Displaced Midshaft Clavicular Fractures: A Systematic Review of Overlapping Meta-Analyses. Medicine (Baltimore), 2015. 94 (26): p. e1057. 16. Lunny, C., Thirugnanasampanthar, S.S., Kanji, S., Pieper, D., Whitelaw, S., Tasnim, S., Reid, E., Zhang, J.H.J., Kalkat, B., Chi, Y., Abdoulrezzak, R., Zheng, D.W.W., Pangka, L., Wang, D.X.R., Safavi, P., Sooch, A., Kang, K.T., Ferri, N., Nelson, H., Tricco, A.C., How can clinicians choose between conflicting and discordant systematic reviews? A replication study of the Jadad algorithm. BMC Med Res Methodol 2022. 22 (27): p. https://rdcu.be/c3yM9. 17. Eysenbach, G., Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res, 2004. 6 (3): p. e34. 18. CIHR, Knowledge User Engagement . 2016, Canadian Institute for Health Research. 19. Lunny, C., et al., Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool). BMJ Evid Based Med, 2022. 20. Boutron, I. and P. Ravaud, Misrepresentation and distortion of research in biomedical literature. Proceedings of the National Academy of Sciences, 2018. 115 (11): p. 2613. 21. Whiting, P., et al., ROBIS: a new tool to assess risk of bias in systematic reviews was developed. Journal of clinical epidemiology, 2016. 69 : p. 225-234. 22. Page, M.J., et al., The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews, 2021. 10 (1): p. 1-11. 23. EQUATOR Network. EQUATOR (Enhancing the QUAlity and Transparency of health Research) Network. https://www.equator-network.org/about-us/ . 2022. 24. Shea, B.J., et al., Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC medical research methodology, 2007. 7 (1): p. 10. 25. Lorenz, R.C., et al., A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool. Journal of clinical epidemiology, 2019. 114 : p. 133-140. 26. Pollock, M., R.M. Fernandes, and L. Hartling, Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions. BMC Medical Research Methodology, 2017. 17 (1): p. 1-13. 27. Shea, B.J., et al., AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. bmj, 2017. 358 . 28. Balshem, H., et al., GRADE guidelines: 3. Rating the quality of evidence. Journal of clinical epidemiology, 2011. 64 (4): p. 401-406. 29. Dillman, D.A., Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide . 2011: John Wiley & Sons. 30. Gupta, K., A practical guide to needs assessment . 2011: John Wiley & Sons. 31. Dillman, D.A., Mail and internet surveys . 2nd edition ed. 2007, Hoboken, New Jersey: John Wiley & Sons Inc. 18. 32. Lwanga, S.K., S. Lemeshow, and W.H. Organization, Sample size determination in health studies: a practical manual . 1991: World Health Organization. 33. Keating, J.L., et al., Providing services for acute low-back pain: a survey of Australian physiotherapists. Manual Therapy, 2016. 22 : p. 145-152. 34. Walker, B.F., et al., Management of people with acute low-back pain: a survey of Australian chiropractors. Chiropr Man Therap, 2011. 19 (1): p. 29. 35. Liu, T., et al., Acupuncture for Primary Dysmenorrhea: A Meta-analysis of Randomized Controlled Trials. Alternative Therapies in Health & Medicine, 2017. 23 (7). 36. Yu, S.-y., et al., Electroacupuncture is beneficial for primary dysmenorrhea: the evidence from meta-analysis of randomized controlled trials. Evidence-Based Complementary and Alternative Medicine, 2017. 2017 . 37. Woo, H.L., et al., The efficacy and safety of acupuncture in women with primary dysmenorrhea: a systematic review and meta-analysis. Medicine, 2018. 97 (23). 38. Harris, R.G., E.P. Neale, and I. Ferreira, When poorly conducted systematic reviews and meta-analyses can mislead: a critical appraisal and update of systematic reviews and meta-analyses examining the effects of probiotics in the treatment of functional constipation in children. Am J Clin Nutr, 2019. 110 (1): p. 177-195. 39. Llanaj, E. and T. Muka, Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis. J Clin Med, 2022. 11 (14). 40. Lucenteforte, E., et al., Discordances originated by multiple meta-analyses on interventions for myocardial infarction: a systematic review. J Clin Epidemiol, 2015. 68 (3): p. 246-56. 41. Hill, A., et al., Ivermectin for the prevention of COVID-19: addressing potential bias and medical fraud. J Antimicrob Chemother, 2022. 77 (5): p. 1413-1416. 42. Lawrence, J.M., et al., The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable. Nat Med, 2021. 27 (11): p. 1853-1854. 43. O'Mathúna, D.P., Ivermectin and the Integrity of Healthcare Evidence During COVID-19. Front Public Health, 2022. 10 : p. 788972. 44. Mhaskar, R., et al., Critical appraisal skills are essential to informed decision-making. Indian J Sex Transm Dis AIDS, 2009. 30 (2): p. 112-9. 45. Petticrew, M., Why certain systematic reviews reach uncertain conclusions. Bmj, 2003. 326 (7392): p. 756-8. 46. Page, M.J., et al., Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database of Systematic Reviews, 2014(10). 47. Lunny, C., Thirugnanasampanthar, S.S., Kanji, S., Ferri, N., Thabet, P., Pieper, D., Tasnim, S., Nelson, H., Reid, E., Zhang, J.H.J., Kalkat, B., Chi, Y., Thompson, J., Abdoulrezzak, R., Zheng, D.W.W., Pangka, L., Wang, D.X.R., Safavi, P., Sooch, A., Kang, K., Whitelaw, S., Tricco, A.C., Protocol and plan for the development of the automated algorithm for choosing the best systematic review. Available at: https://osf.io/nbcta/ . 2021. Additional Declarations The authors declare no competing interests. Supplementary Files WISESTappendices20240109v12.docx Appendices A-D Cite Share Download PDF Status: Posted Version 2 posted You are reading this latest preprint version Show more versions Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2416773","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":166602557,"identity":"4b8af9bd-d9fc-4df3-a16e-3e1438a8b930","order_by":0,"name":"Carole Lunny","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1UlEQVRIiWNgGAWjYLCCBwwMMmDGB4YDRGpJYGDgAdGMM0jWwsxDjBb+9rMPHyRUMPAY3O49Jm3bdkeegf3wA7xaJM6kGxsknAFquXMuTTq37ZlhA0+aAV4tBgxpbBKJbUAtN3LMgFoOJzBIMBDQwv8MqOUfVIslWAv7B/xaJEC2NEC1MIK18OC3ReLGM2aDhGMSPJI3cowte849M2zjySnAq4W/P43xwYcaGzm+GzmGN36U3ZHnZz++Aa8WmGUgggVMshGjHgaY8Xt7FIyCUTAKRiwAAIBSPtHTHtUUAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-7825-6765","institution":"Knowledge Translation Program, St Michaels Hospital, Unity Health Toronto","correspondingAuthor":true,"prefix":"","firstName":"Carole","middleName":"","lastName":"Lunny","suffix":""},{"id":166602558,"identity":"8c11eccc-95b4-481a-a298-89e6df4d767b","order_by":1,"name":"Sera Whitelaw","email":"","orcid":"","institution":"Faculty of Medicine and Health Sciences, McGill University","correspondingAuthor":false,"prefix":"","firstName":"Sera","middleName":"","lastName":"Whitelaw","suffix":""},{"id":166602569,"identity":"c9dea66d-f64d-4127-b14b-f6e4cdf98734","order_by":2,"name":"Emma K Reid","email":"","orcid":"","institution":"Queen Elizabeth II Health Sciences Centre","correspondingAuthor":false,"prefix":"","firstName":"Emma","middleName":"K","lastName":"Reid","suffix":""},{"id":166602559,"identity":"14ce7eab-3026-4444-b426-818011ca47bb","order_by":3,"name":"Yuan Chi","email":"","orcid":"","institution":"Beijing Yealth Technology; Cochrane Campbell Global Ageing Partnership","correspondingAuthor":false,"prefix":"","firstName":"Yuan","middleName":"","lastName":"Chi","suffix":""},{"id":166602560,"identity":"8f2b438b-a143-408c-8d74-4f46fd55fda7","order_by":4,"name":"Jia He Zhang","email":"","orcid":"","institution":"Department of Anesthesiology, Pharmacology, and Therapeutics, Faculty of Medicine, University of British Columbia","correspondingAuthor":false,"prefix":"","firstName":"Jia","middleName":"He","lastName":"Zhang","suffix":""},{"id":166602561,"identity":"ff7c9843-613e-4f94-a428-e4e838ae63ac","order_by":5,"name":"Nicola Ferri","email":"","orcid":"","institution":"Department of Biomedical and Neuromotor Sciences (DIBINEM), Alma Mater Studiorum, University of Bologna","correspondingAuthor":false,"prefix":"","firstName":"Nicola","middleName":"","lastName":"Ferri","suffix":""},{"id":166602562,"identity":"2ba8ff5e-a90c-4601-940e-626401a964fa","order_by":6,"name":"Salmaan Kanji","email":"","orcid":"","institution":"The Ottawa Hospital; Ottawa Health Research Institute, University of Ottawa","correspondingAuthor":false,"prefix":"","firstName":"Salmaan","middleName":"","lastName":"Kanji","suffix":""},{"id":166602563,"identity":"289b2b9a-65b0-434d-b062-4e9ec5a94865","order_by":7,"name":"Dawid Pieper","email":"","orcid":"","institution":"Institute for Health Services and Health System Research, Faculty of Health Sciences Brandenburg, Brandenburg Medical School","correspondingAuthor":false,"prefix":"","firstName":"Dawid","middleName":"","lastName":"Pieper","suffix":""},{"id":166602564,"identity":"df7a20a4-a37b-4595-a768-818fa7e099dc","order_by":8,"name":"Beverley Shea","email":"","orcid":"","institution":"The Ottawa Health Research Institute, University of Ottawa","correspondingAuthor":false,"prefix":"","firstName":"Beverley","middleName":"","lastName":"Shea","suffix":""},{"id":166602565,"identity":"8b52838b-89d8-4bef-aab6-5eaec6ad7c96","order_by":9,"name":"Jasmeen Dourka","email":"","orcid":"","institution":"Knowledge Translation Program, St Michael's Hospital, Unity Health Toronto","correspondingAuthor":false,"prefix":"","firstName":"Jasmeen","middleName":"","lastName":"Dourka","suffix":""},{"id":166602566,"identity":"d68018c6-39f7-494c-b476-4c09765d8c4d","order_by":10,"name":"Areti Angeliki Veroniki","email":"","orcid":"https://orcid.org/0000-0001-6388-4825","institution":"Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto; Institute of Health Policy Management and Evaluation, University of Toronto","correspondingAuthor":false,"prefix":"","firstName":"Areti","middleName":"Angeliki","lastName":"Veroniki","suffix":""},{"id":166602567,"identity":"fcd0a0eb-a08d-40fb-a8a6-c02df0c39344","order_by":11,"name":"Clare L Arden","email":"","orcid":"","institution":"Department of Family Practice, University of British Columbia; Sport \u0026 Exercise Medicine Research Centre, La Trobe University","correspondingAuthor":false,"prefix":"","firstName":"Clare","middleName":"L","lastName":"Arden","suffix":""},{"id":166602568,"identity":"e5b9217f-ec0b-4b55-a3f8-cd7abfd1820b","order_by":12,"name":"Ba' Pham","email":"","orcid":"","institution":"Knowledge Translation Program, St Michael's Hospital, Unity Health Toronto","correspondingAuthor":false,"prefix":"","firstName":"Ba'","middleName":"","lastName":"Pham","suffix":""},{"id":166602570,"identity":"f9c93336-2d6b-45b5-ba65-7524d794ba8f","order_by":13,"name":"Ebrahim Bagheri","email":"","orcid":"","institution":"Department of Electrical and Computer Engineering, Ryerson University","correspondingAuthor":false,"prefix":"","firstName":"Ebrahim","middleName":"","lastName":"Bagheri","suffix":""},{"id":166602571,"identity":"3503b42c-aed5-4297-aefa-d13f19fe7d5f","order_by":14,"name":"Andrea C Tricco","email":"","orcid":"","institution":"Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto; Epidemiology Division and Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto; Queen’s Collaboration for Health Care Quality: A JBI Centre of Excellence, Queen’s University","correspondingAuthor":false,"prefix":"","firstName":"Andrea","middleName":"C","lastName":"Tricco","suffix":""}],"badges":[],"createdAt":"2022-12-26 20:01:41","currentVersionCode":2,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-2416773/v2","doiUrl":"https://doi.org/10.21203/rs.3.rs-2416773/v2","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":49428507,"identity":"abe7dc2c-fcf5-48b2-9d61-e92b87db61ce","added_by":"auto","created_at":"2024-01-10 16:24:40","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":67155,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRecruitment of Decision Makers\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-2416773/v2/bea6adfe7a03368d7b0b0057.png"},{"id":49428506,"identity":"4086eb26-b4f9-4877-b1e9-4eb55438b739","added_by":"auto","created_at":"2024-01-10 16:24:39","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":48468,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eIf a free, automated, tool was available to assist you in choosing the best systematic review(s) among multiple on the same question, would you use it? (Q7)\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e* Denominators were calculated to only include respondents that answered “yes”, “no” or “unsure”.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-2416773/v2/930f6f5d4bf7852ffbe56a08.png"},{"id":49428728,"identity":"c62ea0b2-7314-478c-a257-bb4326cf256e","added_by":"auto","created_at":"2024-01-10 16:32:40","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":777043,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2416773/v2/846b3436-b25e-44a2-afd9-0d461a0e6401.pdf"},{"id":49428508,"identity":"f1c856bd-7f00-4df1-b3f7-e4590d8056af","added_by":"auto","created_at":"2024-01-10 16:24:40","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1114970,"visible":true,"origin":"","legend":"\u003cp\u003eAppendices A-D\u003c/p\u003e","description":"","filename":"WISESTappendices20240109v12.docx","url":"https://assets-eu.researchsquare.com/files/rs-2416773/v2/f9755485276282d7c6ee1004.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eCan artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"Strengths and limitations of this study","content":"\u003cul\u003e\n \u003cli\u003eOur study was conducted in accordance with our protocol published a priori.\u003c/li\u003e\n \u003cli\u003eWe attempted to maximize survey reach and responses by using emails and social media distribution. However, our email response rate was low (\u0026lt;10%) and due to the nature of social media advertising, we were unable to calculate a true survey response rate.\u003c/li\u003e\n \u003cli\u003eOur targeted emails and social media advertising may have missed important decision makers that use systematic reviews. Individuals involved in guideline development and policymaking may have been more likely to respond to the survey.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eResponse bias is a factor as survey respondents working in higher income countries were over-represented.\u0026nbsp;\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"Background","content":"\u003cp\u003eEvidence-informed healthcare requires decision makers to identify evidence, appraise its\u0026nbsp;methodological quality\u0026nbsp;and relevance, and apply it to a particular practice scenario\u0026nbsp;[1]. Systematic reviews (SRs) are used to collate evidence from primary studies (e.g., randomized controlled trials, cohort studies) which are appraised and synthesised using systematic methods\u0026nbsp;[2]. Methodologically sound SRs that are well-reported and with a low risk of bias are widely regarded as a gold standard for healthcare decision making.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eEach year there is an exponential rise in the volume of research produced in healthcare\u0026nbsp;[3], including SRs\u0026nbsp;[4-6]. Between 2000 and 2019, the number of SRs produced annually increased twenty-fold, with 80 SRs published per day by 2019\u0026nbsp;[7]. Along with the increasing prevalence of SRs overall, there has been an increase in the number of duplicated SRs with the same or similar research questions and eligibility criteria\u0026nbsp;[8, 9]. Between 2000 and 2020, approximately 1200 and 1600 systematic review clusters (i.e., duplicated SRs) addressing the same clinical, public health, or policy questions were identified by two bibliometric studies\u0026nbsp;[10, 11]. Duplicated publications increased over time, with the highest increase occurring in the most recent five-year period (2016 to 2020).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDespite similar research questions and focus, duplicated SRs may report discordant results and conclusions. Research groups have studied overlapping SRs in attempt to examine and identify sources of discordance [12-14] and approaches for managing discordant SR findings [15,16]. In 1998, Jadad et al published an algorithm tool\u0026nbsp;[12]\u0026nbsp;to help users select the \u0026ldquo;best evidence\u0026rdquo; review amongst multiple discordant SRs\u0026nbsp;[13-15]. To assess reproducibility of the algorithm, our research group performed a replication study that compared the findings of 21 publications that used the Jadad tool to choose one or more SRs as best evidence with our own independent Jadad assessment\u0026nbsp;[16]. In 62% of cases, replication was unsuccessful, and a different, higher\u0026nbsp;methodological quality\u0026nbsp;SR was chosen by our group. Sources of discrepancies included different PICO (Population, Intervention, Comparison, Outcome) eligibility criteria, databases searched, primary studies, and/or analysis methods\u0026nbsp;[16]. These studies highlight the expertise required by experienced researchers to manually assess and compare similar SRs that differ across their results and conclusions.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAt this time, it is unknown whether decision makers, such as practitioners and policymakers, struggle with comparing and/or choosing SRs when there are multiple on the same topic, and what barriers, if any, are faced. There is also value in learning which variables or features of SRs are considered most important when comparing multiple on the same topic. Keeping abreast of the latest research on a topic is already a monumental task [5], so we proposed that a tool that incorporates artificial intelligence (AI) to help navigate a growing body of literature would be beneficial and could increase efficiencies.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe purpose of this study was to explore the role of a proposed AI-informed, evidence-based tool to help healthcare decision makers appraise and choose amongst SRs for real-world practice. To understand current need, we surveyed decision makers to determine how they use SRs to inform their decisions, and how they choose the best SR evidence when there are multiple SRs addressing the same clinical question.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eProtocol\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eThe study protocol can be found on the Open Science Framework at\u0026nbsp;\u003ca href=\"https://osf.io/nbcta/\"\u003ehttps://osf.io/nbcta/\u003c/a\u003e. Approval from the University British Columbia Ethics Board was obtained in the form of an exemption (ID H20-02013). The reporting of this survey is in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES; \u003cstrong\u003eAppendix\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eA\u003c/strong\u003e)\u0026nbsp;[17]. Important definitions are found in \u003cstrong\u003eBox 1\u003c/strong\u003e.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cdiv style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:solid windowtext 1.0pt;padding:1.0pt 1.0pt 1.0pt 1.0pt;'\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003eBox 1. Important definitions\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eDecision maker\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eWe define \u0026ldquo;decision makers\u0026rdquo; as individuals who are likely to be able to use research results to make informed decisions about health policies, programs and/or clinical practices\u0026nbsp;[18]. A decision maker can be, but is not limited to, a health practitioner, a policymaker, an educator, a health care administrator, a community leader or an individual in a health charity, patient group, private sector organization or media outlet\u0026nbsp;[18]. The following groups were considered decision makers:\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Health practitioner (individuals that provide care, e.g., nurses, physicians, pharmacists, mental health counsellors, community-based workers)\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Patient, caregiver, family member, patient and consumer advocacy organization representative, community leader\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Policymaker (government representative, public funding agency representative, healthcare/hospital administrator, Clinical Practice Guideline developer, Health Technology Assessment [HTA] developer)\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Educator\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Industry representative (e.g., drug/device manufacturers)\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Researcher and/or academic\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Information scientist/medical librarian\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Journal editor, publishers, news media\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026bull; Student, trainee, postdoctoral fellow, graduate student/post graduate trainee/undergoing practicum in a clinical program or focused on health policy or research\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eEvidence-informed decision-making\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eEvidence-informed decision-making stresses that the best available evidence from research should inform decisions, as well as other factors such as context, public opinion, equity, feasibility of implementation, affordability, sustainability, and acceptability to stakeholders. It is a systematic and transparent approach that applies structured and replicable methods to identify, appraise, and make use of evidence across decision-making processes, including for implementation.\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eSystematic review\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eA systematic review attempts to collate all study-specific evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimising bias, thus providing more reliable findings from which conclusions can be drawn and decisions made\u0026nbsp;[2].\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eMeta-analysis\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eTraditional meta-analysis is a statistical method to combine the results from two or more primary studies (e.g. randomised controlled trials, cohort studies), to produce a point estimate of an effect and measures of the precision of that estimate\u0026nbsp;[2].\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eNetwork meta-analysis (NMA)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u0026ldquo;Any set of studies that links three or more interventions via direct comparisons forms a network of interventions. In a network of interventions there can be multiple ways to make indirect comparisons between the interventions. These are comparisons that have not been made directly within studies, and they can be estimated using mathematical combinations of the direct intervention effect estimates available\u0026nbsp;[2].\u0026rdquo; A network is composed by at least three nodes (interventions or comparators) and these are connected (graphically depicted as lines/edges) when at least one study compares the underlying two interventions - the direct comparisons.\u0026nbsp;Reviews that intend to compare multiple treatments with an NMA but then find that the expectations or assumptions are violated (e.g. the network is \u0026lsquo;disconnected\u0026rsquo;, studies are too heterogeneous to combine, underlying assumptions of the method are not met), and hence an NMA is not possible or optimal, are also considered in our definition\u0026nbsp;[19]\u003cstrong\u003e\u003cem\u003e.\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eDiscordance\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eDiscordance is when SRs with similar public health, or policy eligibility criteria (as expressed in PICO) report different results or conclusions for the same outcome. We define discordant results as differences based on the methodological decisions SR authors make, or different interpretations or judgments about these results\u0026nbsp;[16].\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eRisk of bias assessment in the systematic review level\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eA risk of bias assessment evaluates limitations in the way in which the results were planned, analysed, and presented. If these methods are inappropriate, the validity of the findings can be compromised. Bias may also be introduced when interpreting the results to draw conclusions. Conclusions may include \u0026lsquo;spin\u0026rsquo; (e.g. biased mis-representation of the evidence, perhaps to facilitate publication) or (erroneous) mis-interpretation of the evidence\u0026nbsp;[20]. Ideally, potential biases identified in the results of the SR might be acknowledged and addressed appropriately when drawing conclusions\u0026nbsp;[21]. Similarly, a well conducted SR draws conclusions that are appropriate to the included evidence and therefore is free of bias even when the primary studies included in the review have high risk of bias.\u0026nbsp;On the basis of the risk of bias assessment,\u0026nbsp;supported by balanced reporting of SR/meta-analysis (MA) interpretation of findings, relevance of included studies to the SR/MA\u0026rsquo; question, a final consideration is performed on whether the SR/MA as a whole is at \u0026lsquo;low\u0026rsquo;, \u0026lsquo;high\u0026rsquo;, or \u0026lsquo;unclear\u0026rsquo; risk of bias.\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eReporting comprehensiveness\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eThe PRISMA\u0026nbsp;(Preferred Reporting Items for Systematic Reviews and Meta-Analyses)\u0026nbsp;standard encourages reporting completeness or comprehensiveness when authors write up the results of their SRs prior to publication\u0026nbsp;[22]. Often a PRISMA checklist is required when submitting a systematic review to a peer reviewed journal for consideration.\u0026nbsp;A review can be well conducted, but poorly reported; or poorly conducted, but well reported (even if methods were poor). The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network\u0026nbsp;[23]is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines.\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003e\u003cem\u003eQuality of conduct (systematic reviews)\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eMethodological quality is about how well the research is conducted according to established guidance (e.g. Cochrane Handbook\u0026nbsp;[2], JBI Manual ). The Assessing the Methodological Quality of SRs (AMSTAR) measurement tool was designed to appraise the quality of conduct of SRs\u0026nbsp;[24]. AMSTAR has been validated and proven popular as a simple means of assessing the quality of reviews\u0026nbsp;[25, 26]. A recently updated version (AMSTAR-2) was published in 2017\u0026nbsp;[27].\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u003cstrong\u003eCertainty of the evidence assessment\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003eAn assessment of the certainty of evidence is defined as any of evaluation of the strength of the evidence such as the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach\u0026nbsp;[28], criteria for credibility assessment, and other approaches used to grade the overall body of the evidence.\u0026nbsp;GRADE is a well-established approach to assess the certainty of evidence based on the following criteria: risk of bias of the primary studies, imprecision, indirectness, inconsistency, and publication bias. GRADE is designed for assessing the certainty of the evidence deriving from primary studies.\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;border:none;padding:0in;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/div\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eSurvey Design\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eOur investigative team used a cross-sectional survey design, informed by established approaches for conducting needs assessments and the Dillman approach for conducting online surveys\u0026nbsp;[29].\u0026nbsp;The survey was conducted using Qualtrics (Qualtrics Labs, Provo, UT, USA)\u0026nbsp;[30, 31]. No incentive or compensation was offered to respondents. Survey responses were anonymous. Personal identifying information was only collected on a voluntary basis from respondents who wished to be contacted about the survey\u0026rsquo;s results. Informed consent was implied when participants ticked a consent box on the first survey page.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eWe created an English-language survey with 21 questions, primarily closed-ended in nature (full survey questions in \u003cstrong\u003eAppendix B\u003c/strong\u003e). \u0026nbsp;The survey questions were sub-divided into three parts: (a) demographics, (b) experiences and barriers to choosing SRs when more than one exists on the same topic, and (c) data elements to consider when choosing the SRs from multiple on the same topic. Respondents were allowed to skip questions they did not wish to answer and were able to review and change their answers prior to submitting their responses.\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eWe were particularly interested in gaining insight on the specific features a SR decision makers would consider when comparing and selecting amongst multiple SRs, as these features would be crucial to inform priority areas for an AI tool. We recognized the potential for \u0026ldquo;leading\u0026rdquo; or influencing survey respondents by providing a curated list of SR features our steering group deemed relevant in advance of their independent responses. To protect against this influence, we created two similar questions (Q11 and 12a-c), which were randomly allocated to half of the respondents: the first (Q11) was a multiple-choice drop-down list of elements to choose from, and the second (Q12a-c), included a short case study summarizing the characteristics, methods, and results from three similar SRs on the same topic, ultimately asking the respondent to choose which SR(s) they would use to inform a discussion with a patient, and which features of the SR led to their decision.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eEight academics piloted the survey and modified it iteratively to improve clarity, face validity, and content validity.\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eSample size\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eThe sample size was calculated to detect mean difference of 50%\u0026nbsp;between two factor levels with 90% power\u0026nbsp;[32]. To detect this difference, a sample of 440 decision makers was required, assuming a standard deviation of 1.6 points (based on similar surveys\u0026nbsp;[33, 34]) and a 5% significance level. We assumed that contacting quadruple (i.e. 1760) the number of decision makers would be sufficient to recruit the required number\u0026nbsp;(assuming a 25% response rate),\u0026nbsp;allowing for failed email addresses and non-response.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eDistribution of the Survey\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eWe aimed to survey individuals from organizations or institutions which produce SRs, as well as decision makers of all types who use SRs.\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eWe developed an email list of SR-producing groups including Cochrane Multiple Treatments Methods Group, Guidelines International Network, JBI (formerly the Joanna Briggs Institute), Campbell Collaboration, U.S. Agency for Healthcare Research \u0026amp; Quality\u0026rsquo;s Evidence-Based Practice Centre program, Centre for Reviews and Dissemination, Canadian Agency for Drugs and Technologies in Health [CADTH], Evidence for Policy and Practice Information and Co-ordinating Centre [EPPI-Centre],\u0026nbsp;Clinical Epidemiology program\u0026nbsp;at the Ottawa Hospital Research Institute, the GRADE group). These potential survey participants were sent an email describing the purpose of the study, requesting their participation, and providing a link to the survey.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eWe leveraged the professional contacts from our steering committee and distributed the survey to an additional 28 organisations anonymously (\u003cstrong\u003eAppendix C)\u003c/strong\u003e. We also included participants from a UBC Methods Speaker Series on evidence synthesis methods (\u003ca href=\"https://www.ti.ubc.ca/2022/01/01/methods-speaker-series-2022/\"\u003ehttps://www.ti.ubc.ca/2023/01/10/methods-speaker-series-2023/\u003c/a\u003e). In addition, we advertised through the e-newsletters of Knowledge Translation Canada, SPOR (Strategy for Patient-Oriented Research) Evidence Alliance and Therapeutics Initiative. We also contacted professional healthcare organisations (e.g. Canadian Association for Physiotherapists) to promote the survey in their newsletters.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eA distribution plan was followed to disseminate and advertise the survey. The Dillman approach\u0026nbsp;[29]\u0026nbsp;suggests repeated contact to boost responses, which we followed by sending out three reminders to email recipients, and repeated advertisement through social media outlets. Anonymous links were included in LinkedIn and Twitter posts which were circulated through targeted Twitter accounts, such as the Knowledge Translation Program, SPOR Evidence Alliance, and the Therapeutics Initiative. Tweets were retweeted amongst followers. We used twitter cards (i.e. advertisements with pictures) and targeted hashtags to increase awareness of the survey. We also advertised through two\u0026nbsp;LinkedIn accounts.\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eTiming\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eThe survey ran from July 19 to August 19, 2022. Qualtrics email reminders were scheduled at two-week intervals throughout this period to unfinished or non-respondents. We estimated that the survey would take approximately 10 minutes of a respondent\u0026rsquo;s time.\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eData analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003ePrior to data analysis, the responses were transferred from Qualtrics to MS Excel.\u0026nbsp;Questionnaires that were terminated before completion, were included in analyses, but those that were entirely blank were excluded. We measured the time respondents took to fill in a questionnaire regardless of whether it was complete.\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003eDescriptive statistics were calculated for each closed response question, including count, frequency, with denominators taken as the number who provided a response to the question. One researcher coded responses to the open-ended questions or comments independently by identifying themes. The free-form responses were then presented descriptively as counts and frequencies in an identical fashion to closed responses. We stratified the analysis by type of decision maker as presented in Box 1. When a respondent indicated that they were more than one type of decision maker, we calculated the response for all their respondent types. For example, if they identified as both a practitioner and researcher, and responded yes to question 1, we counted a yes for both practitioner and researcher types. We compared the results of the two randomly presented questions.\u0026nbsp;\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eRecruitment results\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 3158 email invitations were sent to advertise the survey. Of these, 197 emails failed to reach the recipients due to incorrect addresses, no longer at the related job post, etc., resulting in a total of 2,961 email invitations successfully delivered (\u003cstrong\u003eFigure 1\u003c/strong\u003e). After consolidating duplicates (n = 25) and blank responses (n = 83), a total of 684 survey responses were included in the analysis. Most respondents completed the survey by clicking and completing an anonymous link distributed over social media and e-newsletters (n = 450), compared to those who responded through the Qualtrics email link (n = 234). As per our sample size calculation, we expected a\u0026nbsp;25% response rate to email invitations but only achieved 7.9%.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOf the 684 respondents, 462 (67.5%) answered all the survey questions, and 97 (14.2%) completed less than 50% of the questions. For those who completed the survey, the median response time was 11 minutes.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDemographics and characteristics of respondents\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe majority of surveyed decision makers identified as researchers (58.2%), practitioners (37.1%), students/trainees (19.2%), and policymakers (13.5%), and many respondents identified as more than one role. For example, of the 62 (13.5%) policymakers, 37 (59.7%) also identified as a researcher, 14 (22.6%) as a practitioner, 9 (14.5%) as a journal editor, and 4 (6.4%) as a patient. The majority of respondents lived in North America\u0026nbsp;(52.8%)\u0026nbsp;and Europe\u0026nbsp;(34.2%). When comparing survey responses by role we focus here on those categorized as researchers, practitioners, and policymakers, as these were well-represented decision maker subgroups who serve unique knowledge user roles. Full characteristics of respondents are summarized in \u003cstrong\u003eAppendix D, Table 1\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe vast majority of respondents reported they were familiar with SRs (621/684 [90.8%]). Two survey questions were included to further gauge the respondents\u0026rsquo; understanding of SRs (\u003cstrong\u003eAppendix D, Table 2 Q2-3\u003c/strong\u003e). The first question asked if PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)\u0026nbsp;was used to assess the methodological quality of a SR. This question was ambiguous as PRISMA is a reporting checklist to determine the comprehensiveness of \u003cem\u003ereporting\u003c/em\u003e of a published SR manuscript (and not the methodological quality). However, the majority (335/542 [61.8%]) correctly agreed that PRISMA was not used to assess methodological quality [of conduct] of a SR. The second question asked respondents to identify\u0026nbsp;limitations that occurred at the level of the SR. Slightly more than half (308/547 [56.3%)]\u0026nbsp;correctly answered that all options except risk of bias of the primary studies applied. More than a third (199/547 [36.4%]) incorrectly answered that all but selective reporting of results and analyses applied.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eExperiences and barriers to choosing SRs when more than one exists on the same topic\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eRespondents (n=558) often (64.5%) or sometimes (32.6%) sought out SRs as a source of evidence in their decision making (never did=2.9%; \u003cstrong\u003eAppendix D, Table 3\u003c/strong\u003e). Respondents (n=538) reported facing a situation where they found more than one SR on a given topic\u0026nbsp;sometimes (54.8%)\u0026nbsp;or often (43.1%) (\u003cstrong\u003eTable 1 Q6\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eJust over half (50.8%) responded that they struggled to choose the most valid and trustworthy SR amongst multiple SRs on the same topic (\u003cstrong\u003eTable 1 Q10\u003c/strong\u003e). Overall, the most common barrier to making this decision was reported to be a lack of time to fully read and evaluate each SR (55.2%) (\u003cstrong\u003eTable 1 Q10\u003c/strong\u003e). Other frequent barriers were related to variation in the quality of conduct of SRs (54.2%), differences in results and conclusions across SRs (49.7%), variation in the primary studies included in the SRs (44.6%), and slightly different clinical focuses of the SRs (43.1%) \u003cstrong\u003e(Table 1 Q10)\u003c/strong\u003e. \u0026nbsp;Of interest, when asked why they struggled to pick one SR, 35.6% (180/505) of respondents said it was because there was insufficient data from titles and abstracts to assess relevance to their question. Additionally, 27% of respondents recognized their inexperience assessing the methodological quality of SRs as being a barrier.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe reported approach to choosing the most appropriate SR tended to vary, depending on the type of decision maker (\u003cstrong\u003eTable 1 Q8\u003c/strong\u003e):\u003c/p\u003e\n\u003cul class=\"decimal_type\"\u003e\n \u003cli\u003ePractitioners most often (75/170 [44.1%]) chose the most recently published SR(s) that were relevant to their topic. About one quarter (45/170 [26.4%]) found as many as they could that were relevant to their topic and then reviewed them all, and one tenth (18/170 [10.6%]) chose the SR from the highest impact factor journal.\u003c/li\u003e\n \u003cli\u003ePolicymakers (34/62 [54.8%]) and researchers (111/266 [41.7%]) most often reported finding as many SRs as possible relevant to their topic of interest, and reviewed them all.\u0026nbsp;\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe majority of decision makers (335/385 [87.0%]) responded that they would use a free, automated, AI-informed, evidence-based online tool to assist in choosing the best SR(s) among multiple on the same question, if it were available (\u003cstrong\u003eFigure 2\u003c/strong\u003e). Of respondent subgroups, practitioners reported the highest interest (90.5%), followed by researchers (86.9%), then policymakers (79.5%).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eData elements to consider when choosing the SRs best suited to my needs from multiple on the same topic\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe two survey questions asking respondents to identify important SR features either via a pre-populated list (Q11) or determined through free-form response to a case study (Q12a-c) were randomised to 274 and 278 respondents, and answered by\u0026nbsp;274 and 186 respondents, respectively. See \u003cstrong\u003eAppendix D\u003c/strong\u003e \u003cstrong\u003eTable 4\u0026nbsp;\u003c/strong\u003efor complete answers.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eQ11 \u0026ndash; \u003cem\u003epre-defined features provided via drop-down menu\u003c/em\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe relevance of the SR\u0026rsquo;s research question to the respondent\u0026rsquo;s clinical question or learning needs was selected most frequently as an important feature (83.6%), followed by the methodological quality and reproducibility of the SR (79.2%). The SR search strategy was also identified a key consideration, with recency of the SR search date (74.8%), and comprehensive search strategy (69.0%) frequently selected. Other features more than half of respondents recognized as important to consider were: (i) the relevance of clinical outcomes (65.7%), (ii) having a risk of bias assessment conducted for primary studies (60.9%), (iii) a published protocol or pre-registration for the SR (59.1%), and (iv) consideration for the types of studies included (i.e. randomized controlled trials versus non-randomized) (55.1%).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eQ12 \u0026ndash; \u003cem\u003efree-form responses identifying features considered when choosing between SRs relevant to a case\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe case study was based on the\u0026nbsp;clinical question \u0026ldquo;Is acupuncture effective/efficacious and safe for women with primary dysmenorrhoea.\u0026rdquo; The PICO, characteristics, and features of three SRs (Lui 2017\u0026nbsp;[35]; Yu 2017\u0026nbsp;[36]; Woo 2018\u0026nbsp;[37]) were presented in a table. Respondents chose the SR by Woo 2018 (86.6%) most frequently based on its strengths and weaknesses (\u003cstrong\u003eAppendix D, Figure 4 Q12a\u003c/strong\u003e). Four out of ten policymakers indicated that they would use all three (44.8%) in their decision making.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWhen asked which criteria helped the respondents decide, the most common response involved using a hierarchy of features, not just one feature, to choose between SRs (or elements of SRs) (73.1%). The next most common consideration was the number of studies or patients included (45.2%). This response was most common amongst practitioners, about two-thirds, compared with\u0026nbsp;one third of researchers, and one tenth of policymakers.\u0026nbsp;The recency of the SR search date (29.6%), the risk of bias of the primary studies being assessed (28.5%), and the methodological quality of the SR (26.3%) were the next most common responses.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAdditional features identified by free-form responses and not included in the pre-populated elements (Q11) were: heterogeneity, and the process for selecting, extracting and assessing studies.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\u003cstrong\u003eTable 1: Considerations when there are multiple SRs on the same topic\u003c/strong\u003e\u003c/p\u003e\n\u003ctable style=\"width:639.3pt;margin-left:5.4pt;border-collapse:collapse;border:none;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 114.85pt;border: 1pt solid windowtext;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003eItem\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 147.4pt;border-width: 1pt 1pt 1pt medium;border-style: solid solid solid none;border-color: windowtext windowtext windowtext currentcolor;border-image: none;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003eResponses\u0026nbsp;\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: 1pt 1pt 1pt medium;border-style: solid solid solid none;border-color: windowtext windowtext windowtext currentcolor;border-image: none;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003eALL\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: 1pt 1pt 1pt medium;border-style: solid solid solid none;border-color: windowtext windowtext windowtext currentcolor;border-image: none;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003ePolicymaker\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: 1pt 1pt 1pt medium;border-style: solid solid solid none;border-color: windowtext windowtext windowtext currentcolor;border-image: none;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003ePractitioner\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: 1pt 1pt 1pt medium;border-style: solid solid solid none;border-color: windowtext windowtext windowtext currentcolor;border-image: none;background: rgb(217, 226, 243);padding: 0in 5.4pt;height: 31.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003eResearcher\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\" style=\"width: 114.85pt;border-width: medium 1pt 1pt;border-style: none solid solid;border-color: currentcolor windowtext windowtext;border-image: none;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eQ6. How often have you faced a situation where you find more than one SR on a given topic of interest to you?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eNever\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=538)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=62)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=167)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=266)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e12 (2.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e0 (0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e4 (2.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 12.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e5 (1.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eSometimes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e295 (54.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e30 (48.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e106 (63.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.35pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e123 (46.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eOften\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e232 (43.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e31 (50.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e57 (34.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e138 (51.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"5\" style=\"width: 114.85pt;border-width: medium 1pt 1pt;border-style: none solid solid;border-color: currentcolor windowtext windowtext;border-image: none;padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eQ8. When you encounter multiple SRs on the same topic how do you choose the one(s) most likely to address your clinical/public health/policy question or your learning needs?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI typically choose the first one I find that is relevant to my topic\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=552)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=62)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=170)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 14.7pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=266)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e13 (2.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e1 (1.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e2 (1.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e4 (1.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI find as many as I can that are relevant to my topic and then review them all\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e207 (37.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e34 (54.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e45 (26.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e111 (41.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI typically choose the most recently published one(s) that are relevant to my topic\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e171 (31.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e7 (11.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e75 (44.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e60 (22.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI typically choose the one from the highest impact factor journal\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e34 (6.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e1 (1.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e18 (10.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 27.9pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e10 (3.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"4\" style=\"width: 114.85pt;border-width: medium 1pt 1pt;border-style: none solid solid;border-color: currentcolor windowtext windowtext;border-image: none;padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eQ9. When you have encountered multiple SRs on the same topic, which of the following statements resonates most with you?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI can usually identify the SR(s) best suited to my needs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=548)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=62)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=170)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 13.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=264)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e268 (48.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e35 (56.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e56 (32.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e152 (57.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI sometimes struggle to identify the SR(s) that are best suited to my needs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e238 (43.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e24 (38.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e94 (55.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e101 (38.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eI often struggle to identify the SR(s) best suited to my needs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e41 (7.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e3 (4.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e19 (11.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 26.3pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e10 (3.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"11\" style=\"width: 114.85pt;border-width: medium 1pt 1pt;border-style: none solid solid;border-color: currentcolor windowtext windowtext;border-image: none;padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eQ10.\u003cem\u003e\u0026nbsp;\u003c/em\u003eIf/when you struggle to choose the SR(s) best suited to your needs, the barriers to you being able to make this decision are\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"2\" style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eInsufficient data from titles and abstracts to assess relevance to my question\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=505)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=60)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=165)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;background: rgb(231, 230, 230);padding: 0in 5.4pt;height: 11.8pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cstrong\u003e\u003cspan style=\"color:black;\"\u003e(n=251)\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 33.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e180 (35.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 33.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e21 (35.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 33.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e55 (33.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 33.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e77 (30.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eInexperience with assessing the methodological quality of (or biases in) SRs\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e140 (27.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e9 (15%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e72 (43.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e30 (12.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eNot enough time to read each SR in full to evaluate all the options\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e279 (55.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e23 (38.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e110 (66.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 28.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e119 (47.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 15.05pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eYou don\u0026apos;t trust the conclusions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 15.05pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e56 (11.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 15.05pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e11 (18.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 15.05pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e21 (12.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 15.05pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e34 (13.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 24.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eDifferent results and conclusions across the SRs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 24.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e251 (49.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 24.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e28 (46.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 24.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e94 (57.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 24.55pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e130 (51.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.65pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eVariation in the quality of how the SRs were conducted\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.65pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e274 (54.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.65pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e34 (56.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.65pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e79 (47.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 25.65pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e146 (58.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eVariation in searches across the SRs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e172 (34.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e23 (38.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e46 (27.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 13.45pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e95 (37.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 35.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eVariation in included primary studies across the SRs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 35.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e225 (44.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 35.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e30 (50.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 35.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e63 (38.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 35.1pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e120 (47.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 31.15pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eVariation in how across the SRs results were synthesized\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 31.15pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e194 (38.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 31.15pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e28 (46.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 31.15pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e51 (30.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 31.15pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e106 (42.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 147.4pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 2.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003eSlightly different clinical focus between SRs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 108.6pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 2.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e218 (43.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 85.05pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 2.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e27 (45.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89.85pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 2.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e74 (44.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 93.55pt;border-width: medium 1pt 1pt medium;border-style: none solid solid none;border-color: currentcolor windowtext windowtext currentcolor;padding: 0in 5.4pt;height: 2.5pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e107 (42.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp style='margin:0in;font-size:15px;font-family:\"Calibri\",sans-serif;margin-left:5.4pt;'\u003e\u003cspan style=\"font-size:12px;\"\u003e*Numbers do not add up to 100% because respondents may have chosen more than one response option, and the majority of respondents identified as more than one type of decision maker (e.g. researcher and patient)\u003c/span\u003e\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eWe surveyed\u0026nbsp;policymakers, practitioners, researchers, learners and other respondent types to understand how they use SRs to inform decision making, and how they compare and select one or more SRs when there are multiple addressing the same clinical, public health or policy question. These individuals, who demonstrated good baseline knowledge of SRs, often\u0026nbsp;sought out SRs as a source of evidence in their work and decision making. They were also\u0026nbsp;frequently\u0026nbsp;faced with a situation where they find more than one SR on a given topic of interest.\u0026nbsp;Nearly half of all respondents have struggled to choose the most valid and trustworthy SR, most often due to lack of time, and owing to varying methodological quality of identified SRs, variability in the primary studies included, and differences in their results and conclusions. The proposed use of an AI tool to assist in comparing multiple SRs on the same topic was well-accepted by survey respondents. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWhen comparing SRs on the same topic, themes of important characteristics were related to the relevance of the SR to their PICO question of interest, the robustness of the literature search and included studies, the recency, and the methodological quality. However, the answers varied by type of decision maker, where healthcare\u0026nbsp;practitioners more often\u0026nbsp;chose the most recently published review(s) relevant to their topic, and policymakers and researchers most often reviewed all the relevant SRs on their topic of interest based on a hierarchy of criteria. This may indicate that\u0026nbsp;practitioners are happy with a decision support\u0026nbsp;tool to compare features of SRs which presents the \u0026ldquo;bottom line\u0026rdquo; synthesis or ranking of the SR evidence, but policymakers and researchers want the distilled information of all the presented reviews from which to make their own methodological judgments.\u003cbr\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAnother identified theme was that there is usually not one single best SR to ultimately choose. While a review may have a good AMSTAR-2 quality rating\u0026nbsp;[27], it still may contain important flaws like failing to report patient important outcomes (e.g. adverse events). It may also miss data and information relevant to the decision maker. The three SRs in our case study contained different primary studies and publication dates, making their comparison especially difficult. To get the most out of the data for their decision making, respondents (especially policymakers) commented that they often would review and read all the SRs on a topic and assess the strengths and weaknesses before making any decisions.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStrengths and limitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA strength of our research was that we conducted it in accordance with an \u003cem\u003ea priori\u003c/em\u003e published protocol. We combined newsletter, email distribution lists, and social media to reach a wide range of decision makers from across the globe. We attempted to maximise the response rate by sending email reminders and repeating messages through social media.\u0026nbsp;Social media circulation as a distribution method proved fruitful although we had no control over exposure. This is the first study to our knowledge that explores this topic.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eA limitation was that\u0026nbsp;we were expecting a 25% response rate to email invitations but achieved \u0026lt;10%.\u0026nbsp;Another limitation is that we were unable to calculate a\u0026nbsp;true survey response rate\u0026nbsp;since two thirds (n=450) responded through social media links which were anonymous. Survey fatigue is a significant issue with this form of research and a multimodal approach to maximize reach, even at the expense of being able to calculate a response rate, was deemed necessary.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eResponse bias in our sample was also a major limitation as decision makers working in higher income countries were over-represented. Additionally, the sample represented in this survey, namely individuals involved in guideline development and policymaking, may have been more likely to have responded. Another limitation is that our targeted emails and social media advertisement may have missed important decision makers that use systematic reviews.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eImplication for practice, policy and knowledge translation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWith the number of SRs on the same topic growing exponentially each year, we can predict that the challenge of decision makers struggling to compare and choose between SR evidence will continue to increase. Our survey suggested that currently, over one third\u0026nbsp;of decision makers\u0026nbsp;reviewed only the data in the title and abstract when making their choice of SR (as opposed to the full text)\u0026nbsp;when faced with clinical or policy decisions. Titles and abstracts may not contain enough information to make informed choices between SRs, and the full text publications should be sought to enable methodological quality/risk of bias assessment and a full comparison of the strengths and weaknesses across SRs on the same topic.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIf a health care decision is informed by SR evidence that is low methodological quality and where inappropriate methods were used, this risks negative patient care outcomes\u0026nbsp;[5, 38-40]. An example of misleading results from SRs with meta-analysis is when ivermectin, an antiparasitic medication, was widely promoted across the world for preventing and treating COVID-19\u0026nbsp;[39, 41-43]. A search (Aug 5, 2022) for Ivermectin for COVID-19 yielded 149 SRs in the PubMed database. Many of the SRs are discordant\u0026nbsp;[39]\u0026nbsp;and vary in methodological quality. A few of these meta-analyses were found to contain impossible numbers, unexplainable cohort mismatches, inconsistent timepoints, and substantial methodological flaws. One of these meta-analyses, submitted as a preprint (https://www.researchsquare.com/article/rs-100956/v2), has since been withdrawn, whereas a published meta-analysis (\u003ca href=\"https://academic.oup.com/ofid/article/8/11/ofab358/6316214\"\u003ehttps://academic.oup.com/ofid/article/8/11/ofab358/6316214\u003c/a\u003e) has been retracted after it was found to include fraudulent data (note that we intentionally omitted a formal reference to these studies). Despite these serious concerns, millions of doses of ivermectin have already been given to treat or prevent COVID-19 globally, potentially risking unnecessary toxicity and depleting supply where it is otherwise indicated. The biased results from these and other poorly designed and reported SRs can mislead decision-making at all levels\u0026nbsp;[44-46]. A user-friendly tool which would efficiently highlight SR methodology weaknesses would help facilitate efficient and appropriately informed health care decisions.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFuture research\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe have identified a role for an AI tool to help decision makers more efficiently compare and choose SR evidence, and preferred SR features be the focus of the tool. Preferences for AI assistance appears to vary depending on the type of decision-maker, and we strive to develop a tool that is suitable to the needs of all users. The WISEST AI tool, in development, will present the scope, strengths and limitations of all the SRs found on the users\u0026rsquo; topic, as well as a methodological quality ranking\u0026nbsp;[47]. Users can then compare the different SRs and make their own clinical and methodological judgment about which SRs are most suited to their needs. The WISEST\u0026nbsp;AI tool will aim to provide a \u0026ldquo;supporting\u0026rdquo; role for decision makers when comparing and choosing between reviews and not a substitution role\u0026nbsp;[47].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOur survey identified that there is a considerable proportion of SR users who feel they have inadequate assessment skills to adequately assess SR methodological quality. The availability of the WISEST AI tool will help distill relevant SR information but is not intended to replace critical assessment or judgment. Appropriate training coupled with the user-friendly tool is necessary to overcome this knowledge gap.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003ePolicymakers, practitioners and researchers often sought out SRs as a source of evidence in their decision making, and often encountered more than one SR on a given topic of interest. Just over half of those surveyed struggled to choose the most valid and trustworthy SR amongst multiple. These difficulties related to lack of time, and difficulty comparing different SRs when they vary in methodological quality and characteristics. When comparing SRs on the same topic, relevance to the question of interest, robustness and recency of the search, and methodological quality of the SR were most important to respondents. The development and implementation of an AI tool to rapidly highlight the features, strengths and weaknesses of SRs will help address these challenges and facilitate healthcare decision making.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding source\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe received no funding to conduct this survey.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe have no conflicts of interest to declare with regards to this survey and project.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePatient involvement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePatients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDeclarations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEthics was obtained from the University of British Columbia Behavioural Research Ethics Board (ID H20-02013).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003e\u0026nbsp;\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData sharing and availability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data is contained in the manuscript and appendices.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSUPPLEMENTAL FILES\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAppendices A-D\u003c/p\u003e"},{"header":"References","content":"\u003cp\u003e1.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Sackett, D.L., \u003cem\u003eEvidence-based medicine.\u003c/em\u003e Semin Perinatol, 1997. \u003cstrong\u003e21\u003c/strong\u003e(1): p. 3-5.\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Higgins, J.P., et al., \u003cem\u003eCochrane handbook for systematic reviews of interventions\u003c/em\u003e. 2019: John Wiley \u0026amp; Sons.\u003c/p\u003e\n\u003cp\u003e3.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Bornmann, L. and R. Mutz, \u003cem\u003eGrowth rates of modern science: A bibliometric analysis based on the number of publications and cited references.\u003c/em\u003e Journal of the Association for Information Science and Technology, 2015. \u003cstrong\u003e66\u003c/strong\u003e(11): p. 2215-2222.\u003c/p\u003e\n\u003cp\u003e4.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Bastian, H., P. Glasziou, and I. Chalmers, \u003cem\u003eSeventy-five trials and eleven systematic reviews a day: how will we ever keep up?\u003c/em\u003e PLoS medicine, 2010. \u003cstrong\u003e7\u003c/strong\u003e(9): p. e1000326.\u003c/p\u003e\n\u003cp\u003e5.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Ioannidis, J.P., \u003cem\u003eThe Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses.\u003c/em\u003e Milbank Q, 2016. \u003cstrong\u003e94\u003c/strong\u003e(3): p. 485-514.\u003c/p\u003e\n\u003cp\u003e6.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Taito, S., et al., \u003cem\u003eAssessment of the Publication Trends of COVID-19 Systematic Reviews and Randomized Controlled Trials.\u003c/em\u003e Annals of Clinical Epidemiology, 2021. \u003cstrong\u003e3\u003c/strong\u003e(2): p. 56-58.\u003c/p\u003e\n\u003cp\u003e7.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Hoffmann, F., et al., \u003cem\u003eNearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019.\u003c/em\u003e J Clin Epidemiol, 2021. \u003cstrong\u003e138\u003c/strong\u003e: p. 1-11.\u003c/p\u003e\n\u003cp\u003e8.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Page, M.J. and D. Moher, \u003cem\u003eMass production of systematic reviews and meta‐analyses: an exercise in mega‐silliness?\u003c/em\u003e The Milbank Quarterly, 2016. \u003cstrong\u003e94\u003c/strong\u003e(3): p. 515.\u003c/p\u003e\n\u003cp\u003e9.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Moher, D., \u003cem\u003eThe problem of duplicate systematic reviews\u003c/em\u003e. 2013, British Medical Journal Publishing Group.\u003c/p\u003e\n\u003cp\u003e10.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lunny, C., et al., \u003cem\u003eBibliometric study of \u0026apos;overviews of systematic reviews\u0026apos; of health interventions: Evaluation of prevalence, citation and journal impact factor.\u003c/em\u003e Res Synth Methods, 2022. \u003cstrong\u003e13\u003c/strong\u003e(1): p. 109-120.\u003c/p\u003e\n\u003cp\u003e11.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Bougioukas, K.I., et al., \u003cem\u003eGlobal mapping of overviews of systematic reviews in healthcare published between 2000 and 2020: a bibliometric analysis.\u003c/em\u003e Journal of Clinical Epidemiology, 2021. \u003cstrong\u003e137\u003c/strong\u003e: p. 58-72.\u003c/p\u003e\n\u003cp\u003e12.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Jadad, A.R., D.J. Cook, and G.P. Browman, \u003cem\u003eA guide to interpreting discordant systematic reviews.\u003c/em\u003e Cmaj, 1997. \u003cstrong\u003e156\u003c/strong\u003e(10): p. 1411-6.\u003c/p\u003e\n\u003cp\u003e13.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Li, Q., et al., \u003cem\u003eMinimally invasive versus open surgery for acute Achilles tendon rupture: a systematic review of overlapping meta-analyses.\u003c/em\u003e J Orthop Surg Res, 2016. \u003cstrong\u003e11\u003c/strong\u003e(1): p. 65.\u003c/p\u003e\n\u003cp\u003e14.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Mascarenhas, R., et al., \u003cem\u003eIs double-row rotator cuff repair clinically superior to single-row rotator cuff repair: a systematic review of overlapping meta-analyses.\u003c/em\u003e Arthroscopy, 2014. \u003cstrong\u003e30\u003c/strong\u003e(9): p. 1156-65.\u003c/p\u003e\n\u003cp\u003e15.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Zhao, J.G., J. Wang, and L. Long, \u003cem\u003eSurgical Versus Conservative Treatments for Displaced Midshaft Clavicular Fractures: A Systematic Review of Overlapping Meta-Analyses.\u003c/em\u003e Medicine (Baltimore), 2015. \u003cstrong\u003e94\u003c/strong\u003e(26): p. e1057.\u003c/p\u003e\n\u003cp\u003e16.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lunny, C., Thirugnanasampanthar, S.S., Kanji, S., Pieper, D., Whitelaw, S., Tasnim, S., Reid, E., Zhang, J.H.J., Kalkat, B., Chi, Y., \u0026nbsp;Abdoulrezzak, R., Zheng, D.W.W., Pangka, L., Wang, D.X.R., Safavi, P., Sooch, A., Kang, K.T., Ferri, N., Nelson, H., Tricco, A.C., \u003cem\u003eHow can clinicians choose between conflicting and discordant systematic reviews? A replication study of the Jadad algorithm.\u003c/em\u003e BMC Med Res Methodol 2022. \u003cstrong\u003e22\u003c/strong\u003e(27): p. https://rdcu.be/c3yM9.\u003c/p\u003e\n\u003cp\u003e17.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Eysenbach, G., \u003cem\u003eImproving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES).\u003c/em\u003e J Med Internet Res, 2004. \u003cstrong\u003e6\u003c/strong\u003e(3): p. e34.\u003c/p\u003e\n\u003cp\u003e18.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;CIHR, \u003cem\u003eKnowledge User Engagement\u003c/em\u003e. 2016, Canadian Institute for Health Research.\u003c/p\u003e\n\u003cp\u003e19.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lunny, C., et al., \u003cem\u003eKnowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool).\u003c/em\u003e BMJ Evid Based Med, 2022.\u003c/p\u003e\n\u003cp\u003e20.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Boutron, I. and P. Ravaud, \u003cem\u003eMisrepresentation and distortion of research in biomedical literature.\u003c/em\u003e Proceedings of the National Academy of Sciences, 2018. \u003cstrong\u003e115\u003c/strong\u003e(11): p. 2613.\u003c/p\u003e\n\u003cp\u003e21.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Whiting, P., et al., \u003cem\u003eROBIS: a new tool to assess risk of bias in systematic reviews was developed.\u003c/em\u003e Journal of clinical epidemiology, 2016. \u003cstrong\u003e69\u003c/strong\u003e: p. 225-234.\u003c/p\u003e\n\u003cp\u003e22.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Page, M.J., et al., \u003cem\u003eThe PRISMA 2020 statement: an updated guideline for reporting systematic reviews.\u003c/em\u003e Systematic reviews, 2021. \u003cstrong\u003e10\u003c/strong\u003e(1): p. 1-11.\u003c/p\u003e\n\u003cp\u003e23.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;EQUATOR Network. \u003cem\u003eEQUATOR (Enhancing the QUAlity and Transparency of health Research) Network. https://www.equator-network.org/about-us/\u003c/em\u003e. 2022.\u003c/p\u003e\n\u003cp\u003e24.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Shea, B.J., et al., \u003cem\u003eDevelopment of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.\u003c/em\u003e BMC medical research methodology, 2007. \u003cstrong\u003e7\u003c/strong\u003e(1): p. 10.\u003c/p\u003e\n\u003cp\u003e25.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lorenz, R.C., et al., \u003cem\u003eA psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool.\u003c/em\u003e Journal of clinical epidemiology, 2019. \u003cstrong\u003e114\u003c/strong\u003e: p. 133-140.\u003c/p\u003e\n\u003cp\u003e26.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Pollock, M., R.M. Fernandes, and L. Hartling, \u003cem\u003eEvaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions.\u003c/em\u003e BMC Medical Research Methodology, 2017. \u003cstrong\u003e17\u003c/strong\u003e(1): p. 1-13.\u003c/p\u003e\n\u003cp\u003e27.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Shea, B.J., et al., \u003cem\u003eAMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.\u003c/em\u003e bmj, 2017. \u003cstrong\u003e358\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003e28.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Balshem, H., et al., \u003cem\u003eGRADE guidelines: 3. Rating the quality of evidence.\u003c/em\u003e Journal of clinical epidemiology, 2011. \u003cstrong\u003e64\u003c/strong\u003e(4): p. 401-406.\u003c/p\u003e\n\u003cp\u003e29.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Dillman, D.A., \u003cem\u003eMail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide\u003c/em\u003e. 2011: John Wiley \u0026amp; Sons.\u003c/p\u003e\n\u003cp\u003e30.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Gupta, K., \u003cem\u003eA practical guide to needs assessment\u003c/em\u003e. 2011: John Wiley \u0026amp; Sons.\u003c/p\u003e\n\u003cp\u003e31.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Dillman, D.A., \u003cem\u003eMail and internet surveys\u003c/em\u003e. 2nd edition ed. 2007, Hoboken, New Jersey: John Wiley \u0026amp; Sons Inc. 18.\u003c/p\u003e\n\u003cp\u003e32.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lwanga, S.K., S. Lemeshow, and W.H. Organization, \u003cem\u003eSample size determination in health studies: a practical manual\u003c/em\u003e. 1991: World Health Organization.\u003c/p\u003e\n\u003cp\u003e33.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Keating, J.L., et al., \u003cem\u003eProviding services for acute low-back pain: a survey of Australian physiotherapists.\u003c/em\u003e Manual Therapy, 2016. \u003cstrong\u003e22\u003c/strong\u003e: p. 145-152.\u003c/p\u003e\n\u003cp\u003e34.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Walker, B.F., et al., \u003cem\u003eManagement of people with acute low-back pain: a survey of Australian chiropractors.\u003c/em\u003e Chiropr Man Therap, 2011. \u003cstrong\u003e19\u003c/strong\u003e(1): p. 29.\u003c/p\u003e\n\u003cp\u003e35.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Liu, T., et al., \u003cem\u003eAcupuncture for Primary Dysmenorrhea: A Meta-analysis of Randomized Controlled Trials.\u003c/em\u003e Alternative Therapies in Health \u0026amp; Medicine, 2017. \u003cstrong\u003e23\u003c/strong\u003e(7).\u003c/p\u003e\n\u003cp\u003e36.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Yu, S.-y., et al., \u003cem\u003eElectroacupuncture is beneficial for primary dysmenorrhea: the evidence from meta-analysis of randomized controlled trials.\u003c/em\u003e Evidence-Based Complementary and Alternative Medicine, 2017. \u003cstrong\u003e2017\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003e37.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Woo, H.L., et al., \u003cem\u003eThe efficacy and safety of acupuncture in women with primary dysmenorrhea: a systematic review and meta-analysis.\u003c/em\u003e Medicine, 2018. \u003cstrong\u003e97\u003c/strong\u003e(23).\u003c/p\u003e\n\u003cp\u003e38.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Harris, R.G., E.P. Neale, and I. Ferreira, \u003cem\u003eWhen poorly conducted systematic reviews and meta-analyses can mislead: a critical appraisal and update of systematic reviews and meta-analyses examining the effects of probiotics in the treatment of functional constipation in children.\u003c/em\u003e Am J Clin Nutr, 2019. \u003cstrong\u003e110\u003c/strong\u003e(1): p. 177-195.\u003c/p\u003e\n\u003cp\u003e39.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Llanaj, E. and T. Muka, \u003cem\u003eMisleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis.\u003c/em\u003e J Clin Med, 2022. \u003cstrong\u003e11\u003c/strong\u003e(14).\u003c/p\u003e\n\u003cp\u003e40.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lucenteforte, E., et al., \u003cem\u003eDiscordances originated by multiple meta-analyses on interventions for myocardial infarction: a systematic review.\u003c/em\u003e J Clin Epidemiol, 2015. \u003cstrong\u003e68\u003c/strong\u003e(3): p. 246-56.\u003c/p\u003e\n\u003cp\u003e41.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Hill, A., et al., \u003cem\u003eIvermectin for the prevention of COVID-19: addressing potential bias and medical fraud.\u003c/em\u003e J Antimicrob Chemother, 2022. \u003cstrong\u003e77\u003c/strong\u003e(5): p. 1413-1416.\u003c/p\u003e\n\u003cp\u003e42.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Lawrence, J.M., et al., \u003cem\u003eThe lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable.\u003c/em\u003e Nat Med, 2021. \u003cstrong\u003e27\u003c/strong\u003e(11): p. 1853-1854.\u003c/p\u003e\n\u003cp\u003e43.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;O\u0026apos;Math\u0026uacute;na, D.P., \u003cem\u003eIvermectin and the Integrity of Healthcare Evidence During COVID-19.\u003c/em\u003e Front Public Health, 2022. \u003cstrong\u003e10\u003c/strong\u003e: p. 788972.\u003c/p\u003e\n\u003cp\u003e44.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Mhaskar, R., et al., \u003cem\u003eCritical appraisal skills are essential to informed decision-making.\u003c/em\u003e Indian J Sex Transm Dis AIDS, 2009. \u003cstrong\u003e30\u003c/strong\u003e(2): p. 112-9.\u003c/p\u003e\n\u003cp\u003e45.\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Petticrew, M., \u003cem\u003eWhy certain systematic reviews reach uncertain conclusions.\u003c/em\u003e Bmj, 2003. \u003cstrong\u003e326\u003c/strong\u003e(7392): p. 756-8.\u003c/p\u003e\n\u003cp\u003e46. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Page, M.J., et al., \u003cem\u003eBias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.\u003c/em\u003e Cochrane Database of Systematic Reviews, 2014(10).\u003c/p\u003e\n\u003cp\u003e47. \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Lunny, C., Thirugnanasampanthar, S.S., Kanji, S., Ferri, N., Thabet, P., Pieper, D., Tasnim, S., Nelson, H., Reid, E., Zhang, J.H.J., Kalkat, B., Chi, Y., Thompson, J., Abdoulrezzak, R., Zheng, D.W.W., Pangka, L., Wang, D.X.R., Safavi, P., Sooch, A., Kang, K., Whitelaw, S., Tricco, A.C., \u003cem\u003eProtocol and plan for the development of the automated algorithm for choosing the best systematic review. Available at: https://osf.io/nbcta/\u003c/em\u003e. 2021.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"St Michael's Hospital","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"systematic reviews, evidence synthesis, implementation science, decision makers, knowledge translation, research methodology","lastPublishedDoi":"10.21203/rs.3.rs-2416773/v2","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2416773/v2","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground\u003c/b\u003e\u003c/p\u003e \u003cp\u003e Systematic reviews (SRs) are being published at an accelerated rate. Decision makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision makers (e.g., practitioners, policymakers, researchers) use SRs to inform decision making, and to explore the role of a proposed AI tool to assist in critical appraisal and choosing amongst SRs.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods\u003c/b\u003e\u003c/p\u003e \u003cp\u003eWe developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults\u003c/b\u003e\u003c/p\u003e \u003cp\u003eOf the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students, and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR amongst multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%), or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusions\u003c/b\u003e\u003c/p\u003e \u003cp\u003eRespondents often sought out SRs as a source of evidence in their decision making, and often encountered more than one SR on a given topic of interest. Many decision makers struggled to choose the most trustworthy SR amongst multiple, related to a lack of time and difficulty comparing SRs varying in methodological quality. An AI tool to facilitate comparison of the relevance of SRs, the search, and methodological quality, would help users efficiently choose amongst SRs and make healthcare decisions.\u003c/p\u003e","manuscriptTitle":"Can artificial intelligence help decision makers navigate the growing body of systematic review evidence? A cross-sectional survey","msid":"","msnumber":"","nonDraftVersions":[{"code":2,"date":"2024-01-10 16:24:35","doi":"10.21203/rs.3.rs-2416773/v2","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}},{"code":1,"date":"2023-01-10 15:18:59","doi":"10.21203/rs.3.rs-2416773/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"56bccf68-3090-4664-b0aa-ff828e86bf5f","owner":[],"postedDate":"January 10th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":28048856,"name":"Epidemiology"}],"tags":[],"updatedAt":"2023-01-10T15:18:59+00:00","versionOfRecord":[],"versionCreatedAt":"2024-01-10 16:24:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v2","identity":"rs-2416773","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2416773","identity":"rs-2416773","version":["v2"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}