Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias

Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias Audinga-Dea Hazewinkel, Elinor Curnow, Kate Tilling, Rachael Ann Hughes This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9283156/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background Failure to appropriately account for selection bias may lead to erroneous conclusions. Risk-of-bias tools are recommended (in systematic reviews of medical studies) to identify potential selection bias, and quantitative bias analyses (QBA) are recommended to quantitatively assess the sensitivity of a (meta-analysis or individual) study’s conclusions to plausible assumptions about the selection mechanism. While risk-of-bias tools are widely available, their effective use is complicated by the large number of available instruments with varying granularity and a high reliance on investigator expertise. QBAs are not routinely implemented, due to a lack of knowledge about accessible methods and software. Methods We conducted a scoping review of software implementations of risk-of-bias tools and QBA methods, for sample selection bias, published from January 2004 through August 2025. Inclusion criteria were validated risk-of-bias tools and software not requiring adaptation (i.e., code changes) before application, still available in 2025, and accompanied by documentation. Key properties of each risk-of-bias and software tool were identified. Results We identified 24 risk-of-bias tools. All consisted of one to five signalling questions relating to selection bias and covered a wide range of study designs (e.g., randomised controlled trials, non-randomised and diagnostic accuracy studies). Question granularity varied and fewer than half of tools included a dedicated selection bias domain, with only three providing structured guidance for synthesizing the evidence from these questions into a final risk judgment on selection bias. For QBA, we identified nine software programs (one web-based, five R packages and three Stata commands). Six programs were only applicable when estimating the effect of a binary exposure on a binary outcome. Three programs implemented a probabilistic QBA, one a multidimensional QBA, and one a tipping point QBA. The remaining programs required the user to supply their own code to fully implement a QBA. Conclusions While many risk-of-bias tools are available, the degree of guidance provided varies drastically, and the overall assessment of selection bias depends heavily on the investigator's interpretation. Greater provision of QBA software to selection bias, along with detailed QBA guidelines, would facilitate the wider uptake of QBA among future studies. Causal inference Quantitative bias analysis Review Risk-of-bias tool Selection bias Sensitivity analysis Software Figures Figure 1 1. Background The main aim of many epidemiologic studies and randomised trials is to estimate the causal effect of an exposure (or treatment) on an outcome in some target population. Inference about the causal exposure effect (here onwards, shortened to exposure effect ) may be invalid when the sample included in the analysis (i.e., the analytic sample ) is a non-representative sample of the target population. This could be due to selection into the study (non-random sampling of the target population or non-participation of individuals invited into the study) or due to selection out of the study (e.g., participant dropout, loss to follow-up, or missing data). In general, bias due to selection into the study is far more challenging to identify and to correct for than selection out of the study because there is (virtually) no information available on the unselected participants [ 1 ]. Consequently, limited guidance is available on how to deal with bias due to selection into the study, in contrast to the numerous reviews and tutorial-style papers available on missing data (e.g., [ 2 – 6 ]). To address this gap in the literature, this paper focuses on selection into the study. In the absence of data on the unselected individuals, risk-of-bias tools may be used in systematic reviews of medical studies to identify potential selection bias [ 7 – 9 ]. At the analysis stage, a quantitative bias analysis (QBA; also known as a sensitivity analysis) is recommended to quantitatively assess the sensitivity of a (meta-analysis or individual) study’s conclusions to plausible assumptions about the selection bias [ 1 , 10 ]. Qualitative risk-of-bias tools have been primarily designed and employed for appraising the methodological quality of studies in systematic reviews. Despite this primary use, these tools are increasingly accepted for investigating bias in individual studies [ 7 ]. A wide range of risk-of-bias tools exists, covering diverse study types, from general designs like randomised trials and cohort studies, to highly specific areas like animal or mental health studies. These tools differ significantly in depth, ranging from simple checklists to complex, algorithm-based instruments with extensive guidance. A systematic overview of such tools was published by Ma et al. in 2020 [ 8 ]. However, no systematic review has yet catalogued how this diverse collection of qualitative tools specifically assesses the risk of selection bias. From here onward, risk-of-bias tools refers to those aspects of risk-of-bias tools that examine selection bias. Three systematic reviews of analyses of real-data studies, published between 2010 and mid-2023, found that QBA to selection bias was far less common than QBA to unmeasured confounding and misclassification [ 11 – 13 ]. This may be partly attributed to a lack of knowledge about accessible software since there are no reviews of software implementing a QBA to selection bias. Currently, the limited information on software includes: the appendix of Shi et al [ 14 ], which lists the software (or links to supplied code) of QBA statistical methods applicable to summary-level data; Infante-Rivard and Cusson [ 15 ] summarise the Stata command episens [ 16 ], and the appendix of Griffith et al [ 17 ] lists four software programs on quantifying selection bias, of which only two are still available (AscRtain [ 18 ] and Bias app [ 19 ]). Also, the website accompanying Fox et al’s seminal textbook on QBA [ 10 ] provides spreadsheets and software code to replicate QBA example analyses. From here onward, QBA refers to QBA to selection bias. To address these gaps in the literature, we have conducted a scoping review of risk-of-bias tools and publicly available software implementations of QBAs, describing the key properties of each software program. We begin, in section 2 , with a brief overview of selection bias, risk-of-bias tools, and QBA. The methods and results of the scoping review are described in sections 3 and 4 , respectively, and we conclude with a discussion in section 5 . 2. Overview of risk-of-bias tools and QBA to selection bias In this section we briefly introduce statistical terms and concepts mentioned by the articles and documents included in our review. 2.1 Selection bias We want to estimate the effect of an exposure (or treatment) \(\:X\) on an outcome \(\:Y\) in some target population, where the \(\:Y-X\) association may be confounded by measured covariates \(\:C\) . We use a sampling design to select the study sample from our target population. All individuals of the study sample are invited to participate and those who agree will form the analytic sample. Data on \(\:X,\) \(\:Y,\) and \(\:C\) are only recorded on individuals in the analytic sample and are used to estimate the exposure effect. We assume that in the absence of selection bias we obtain an unbiased estimate of the exposure effect. We consider the overall selection bias arising from a systematic difference between the true exposure effect in the target population and the estimate obtained from the analytic sample (we shall refer to as the naïve estimate ) [ 20 , 21 ]. It can arise when the study sample is unrepresentative of the target population (e.g., due to an inappropriate sampling design) or when the analytic sample is unrepresentative of the target population (e.g., due to non-participation of invited individuals). Note that we do not distinguish between different types of selection bias defined according to the stage of the selection process (see [ 21 – 23 ] for more details). Also, in line with the articles and documents we reviewed, we do not distinguish between type I selection bias (due to restricting to one or more levels of a collider or a descendent of a collider) and type II selection (due to restricting to one or more levels of an effect measure modifier) [ 21 ]. For a detailed discussion on the concept of selection bias in epidemiology and the different selection mechanisms that can arise see [ 20 – 25 ]. 2.2 Risk-of-bias tools Risk-of-bias tools aim to assess the quality of studies through a series of qualitative signalling questions, which may be presented as a checklist or be organised in different domains, with each domain focussing on a different source of bias (e.g., selection bias, confounding bias, missing data bias, etc.). These tools have been designed to aid in appraising the risk-of-bias for individual studies that are included in a systematic review, or to help decide on the inclusion or exclusion of a study. They can, however, also be used to assess the risk-of-bias in an individual study, and it is to that end that we describe these tools here. Each tool contains signalling questions to determine whether different types of biases may be present. 2.3 Quantitative bias analysis A QBA quantifies the likely magnitude and direction of the selection bias under different plausible assumptions about the selection mechanism (assuming no other sources of bias). Generally, a QBA requires a model (known as a bias model) for the observed data, Y , X and C , and selection process \(\:S\) . The bias model will include one or more parameters \(\:\varphi\:\) (known as bias or sensitivity parameters) which govern the magnitude and direction of the bias. For example, in the simple setting of binary \(\:X\) and \(\:Y\) , \(\:\varphi\:\) could be the probabilities of selection into the analytic sample for each of the four strata defined by \(\:X\) and \(\:Y\) . Since \(\:\varphi\:\) cannot be estimated from the observed data, information about the likely values of \(\:\varphi\:\) must be obtained from external sources (such as external validation studies, published literature, or expert opinion). Setting \(\:\varphi\:\) to specified values enables estimation of the remaining parameters of the bias model and can provide an estimate of the exposure effect adjusted for selection bias (here onward, called the bias-adjusted exposure effect estimate ). By changing the values of \(\:\varphi\:\) , a QBA estimates the bias-adjusted exposure effect under different assumptions about the selection process. There are two broad classes of QBA methods: deterministic and probabilistic [ 10 ]. A deterministic QBA (sometimes called a multidimensional QBA) specifies a range of values for each bias parameter of \(\:\varphi\:\) and then calculates the bias-adjusted exposure effect estimate for each plausible combination of values. Typically, the results are displayed as a plot or table of the bias-adjusted estimates against different values of \(\:\varphi\:\) . A tipping point analysis is a deterministic QBA where interest only lies in the values of \(\:\varphi\:\) that correspond to a change of study conclusions. A probabilistic QBA specifies a prior probability distribution for \(\:\varphi\:\) to explicitly model the user’s assumptions about which value-combinations of \(\:\varphi\:\) are most likely to occur and to incorporate their uncertainty about \(\:\varphi\:\) . The output is a distribution of bias-adjusted estimates which can be summarised to give a point estimate (i.e., the most likely bias-adjusted exposure effect estimate under the QBA’s assumptions) and an interval estimate to reflect the uncertainty due to the selection bias and sampling variability. A probabilistic QBA can be implemented as a fully Bayesian analysis (where the prior for \(\:\varphi\:\) is combined with the likelihood function for the data) or as a Monte Carlo sensitivity analysis (where values for \(\:\varphi\:\:\) are directly sampled from their prior distributions and then used to obtain estimates of the bias-adjusted exposure effect) [ 26 , 27 ]. 3. Methods 3.1 Summary of the search strategy We conducted a scoping review of software implementations of risk-of-bias tools and QBAs made publicly available before 31st August 2025. We defined “software” to be either a web tool or software code that (i) was not specific to a particular data example (i.e., we excluded examples of code from empirical analyses that required code adaptation before application to another example), (ii) had a user interface, (iii) was accompanied by documentation detailing the software’s features, and (iv) was available for use on 31st August 2025. We conducted three searches: (1) a search of the published literature to identify risk-of-bias tools and software implementations of QBA, (2) a search of R packages implementing a QBA, available on the Comprehensive R archive Network (CRAN), and (3) a search of Stata commands implementing a QBA, including inbuilt commands and user-written commands available on the Boston College Statistical Software Components (SSC) archive or via Stata’s net command. Each search was conducted in three stages. In stage 1, we identified published articles or software documents that mentioned “selection bias” and “risk-of-bias tools” or “selection bias” and “bias analysis”, and their synonyms, in either the title, keywords, abstract or software description (see boxes 1–8 in the Supplementary materials for the exact search terms). 3.2 Stage 1 We conducted a search of the published literature to identify software implementations of risk-of-bias tools and of QBA methods. We used Web of Science to search 10 databases (Web of Science core collection, BIOSIS citation index, Derwent innovations index, grants index, KCI Korean journal database, MEDLINE, policy citation index, preprint citation index, ProQuest dissertations & theses citation index, SciELO citation index). The Web of Science search query was designed to exclude articles solely focused on answering applied questions because they contain limited information on the statistical methodology used. Specifically, the Web of Science search query consisted of three substages: Substage 1.1 included articles that mentioned (in the title, abstract or keywords) terms (and their synonyms) relating to “software” or “developing a method” along with terms relating to “selection bias” and “risk-of-bias tools” or “selection bias” and “bias analysis” (Supplement box 1). Substage 1.2 excluded articles with a title containing terms relating to “meta-analysis” or “systematic review” and articles published in the Cochrane Database of Systematic reviews (Supplement box 2). Substage 1.3 added back in any articles excluded from substage 1.2 that mentioned in the title terms relating to “meta-analysis” or “systematic review” and “bias” or “method” (Supplement box 3). We checked that substage 1.1 did not exclude any relevant methodological papers by rerunning the search excluding the software terms and examining the full text of 50 randomly selected articles that were not identified by substage 1.1; none of these were relevant papers that should have been included. Substage 1.3 was necessary to ensure we only excluded applied meta-analyses and systematic reviews. For more details see Supplement Section 1.1. The R and Stata searches were used to identify software implementations of QBA methods only. We used R packages packagefinder (version 0.3.5) [ 28 ] and pkgsearch (version 3.1.5) [ 29 ] to search the CRAN database, and the RePEc IDEAS website [ 30 ] and the Stata search command (version 19 [ 31 ]) to identify inbuilt commands (using option manual ) and user-written Stata commands (using option net ). For more details see Supplement Sections 1.2 and 1.3. 3.3 Stage 2 After excluding any duplicate entries from stage 1, we proceeded to stage 2 where the abstracts of published articles or software descriptions were independently reviewed by two reviewers to determine if they were eligible for data extraction, with any disagreements resolved by consensus. Eligible abstracts of published articles either (i) introduced a new risk-of-bias tool for selection bias, (ii) introduced a new method or software implementing a QBA to selection bias, or (iii) compared or reviewed existing risk-of-bias or QBA methodology in the context of selection bias. Eligible software descriptions of R packages and Stata commands introduced software implementations of a QBA to selection bias. Ineligible abstracts or software descriptions were (i) meeting abstracts, (ii) articles where a QBA was not conducted but mentioned as further work, (iii) articles or software descriptions about a risk-of-bias tool or QBA software for other sources of bias (e.g., unmeasured confounding, missing data, information bias), and (iv) articles solely focused on answering applied questions. 3.4 Stage 3 In stage 3, we assessed the full text of the included articles. For the risk-of-bias tools, we extracted signalling questions relating to selection bias, and for QBA software we extracted information about the analysis of interest, the QBA method, and features of the software implementation. Also, we performed a concurrent reference list search on these full texts to identify additional relevant risk-of-bias tools and QBA software. 4. Results 4.1 Study selection Figure 1 shows that our review identified 24 risk-of-bias tools and nine QBA software programs (additional details given in Supplement Section 2 and Supplementary Fig. 1). The Web of Science search identified 1085 abstracts, of which we excluded 669 because they were either systematic reviews or meta-analyses aimed at answering an applied question, and 5 were duplicates or conference abstracts. We excluded 329 of the 411 screened abstracts and read the full text of the remaining 82 articles. A further 68 articles were excluded, of which 11 provided software code that did not satisfy our definition of a software program. Among the 14 included articles, 3 described software programs for QBA, while 11 described qualitative risk-of-bias tools. A reference search identified 13 additional qualitative risk-of-bias tools. The R and Stata searches identified a further 2 and 4 software programs, respectively. A detailed breakdown of the selection steps and intermediate results for each search are given in Supplement Sections 2.1, 2.2 and 2.3 for the Web of Science search, R search, and Stata search, respectively. 4.2 Risk-of-bias tools Table 1 gives an overview of the 24 available tools, ordered according to study design, and, within this, alphabetically by first author name. Most of these tools have been developed for a specific type of study design, ranging from randomised clinical trials to non-randomised studies of interventions, such as cohort and case-control studies, to prevalence studies and diagnostic accuracy studies. Four tools (Faillie et al.’ s checklist for drug adverse events [ 32 ], Luijken et al. ’s checklist for operative interventions [ 33 ], Well et al .’s Newcastle-Ottowa scale for cohort and case control studies [ 34 ]) and the OHAT risk-of-bias rating tool [ 35 ] contained signalling questions specific to multiple study designs. Table 1 Qualitative tools for risk of bias assessment Tool Detail on target studies Selection bias specific domain Outcome Name or descriptor of the qualitative assessment tool Additional detail on target studies Does the tool contain a selection bias specific domain? Number of signalling questions (SQs) pertaining to selection bias and method of assessment Randomised controlled trials Checklist for drug AEs [ 32 ] (Faillie et al. , 2017) RCTs and observational studies of drug adverse events. Yes 4 SQs specific to RCTs, rated yes/ no/ unclear/ not applicable Checklist for operative interventions [ 33 ] (Luijken et al. , 2022) RCTs and observational studies for operative interventions in orthopaedic trauma surgery No (selection bias specific SQs are given in domains Confounding and Missing data and selection bias 4 SQs specific to RCTs, rated yes/ probably yes/ probably no/ no/ not informative. Per SQ, the rating is translated to an assessment of good/ moderate/ poor methodology. JBI Checklist for RCTs [ 56 ] (Barker et al. , 2023) Individually-randomised, parallel-group trials No 3 SQs, rated yes/ no/ unclear/ not applicable. OHAT risk of bias rating tool [ 35 ] (US National Toxicology Program) Animal and human RCTs, cohort, case control and cross-sectional studies Yes 2 SQs, rated probably/definitely low risk of bias, probably/definitely high risk of bias RoB-2 [ 9 ] (Sterne et al. , 2019) Individually-randomised, parallel-group trials Yes (SQs are given in domain Risk of bias arising from the randomization process ) 3 SQs rated yes/ probably yes/ probably no/ no/ not informative. Provides an algorithm for combining evidence from the 3 SQs into an overall risk of bias judgement of low risk/some concerns/ high risk. SYRCLE’s RoB tool [ 39 ](Hooijmans et al. , 2014) Animal intervention studies Yes (SQs are given in domain Bias due inadequate randomisation and lack of blinding ) 3 SQs, rated yes/no/unclear. Non-randomised studies of the effects of interventions or exposures Checklist for operative interventions [ 33 ] (Luijken et al. , 2022) RCTs and observational studies for operative interventions in orthopaedic trauma surgery No (selection bias specific SQs are given in the joint domain Missing data and selection bias ) 2 SQs specific to observational studies, rated yes/ probably yes/ probably no/ no/ not informative. Per SQ, the rating is translated to an assessment of good/ moderate/ poor methodology. JBI checklist for quasi-experimental studies [ 57 ] (Barker et al. , 2024) Experimental studies without random allocation No 2 SQs, rated yes/ no/ unclear/ not applicable MINORS [ 58 ] (Slim et al. , 2003) Non-randomised studies (without and with comparator) No 2 SQs, scored 0 (unreported), 1 (reported but inadequate) and 2 (reported and adequate) OHAT risk of bias rating tool [ 35 ] (US National Toxicology Program) Animal and human RCTs, cohort, case control and cross-sectional studies Yes 2 SQs, rated probably/definitely low risk of bias, probably/definitely high risk of bias RoBANS-2 [ 46 ] (Seo et al. , 2023) Non-randomised studies of interventions N/A (each SQ makes up its own domain) 2 SQs, rated as indicative of a low/ high/ unclear risk of bias ROBINS-E [ 38 ] (Higgins et al. , 2024) Non-randomised studies of exposures Yes (SQs are given in domain Bias in selection of participants into the study ) 5 SQs rated yes/ probably yes/ probably no/ no/ not informative. Provides an algorithm for combining evidence from the 5 SQs into an overall risk of bias judgement of low/moderate/ serious/ critical/ not informative ROBINS-I [ 37 ] (Sterne et al. , 2016) Non-randomised studies of interventions Cohort studies Checklist for drug AEs [ 32 ] (Faillie et al. , 2017) RCTs and observational studies of drug adverse events. Yes 3 SQs specific to cohort studies, rated yes/ no/ unclear JBI checklist for cohort studies [ 59 ] (Barker et al. , 2025) Cohort studies No 1 SQ, rated yes/ no/ unclear/ not applicable Newcastle-Ottowa Scale (NOS) [ 34 ] (Wells et al. , 2011) Cohort studies and case-control studies Yes 4 SQs with multiple choice answers (2 to 4 per question), with, for each SQ, a star indicating an acceptable answer (1 to 2 stars per SQ), and a maximum of four stars indicating low risk of selection bias. Case control studies Checklist for drug AEs [ 32 ] (Faillie et al. , 2017) RCTs and observational studies of drug adverse events. Yes 3 SQs specific to case control studies, rated yes/ no/ unclear JBI checklist for case/control studies. [ 60 ] (JBI website) Case/control studies No 2 SQs, rated yes/ no/ unclear/ not applicable Newcastle-Ottowa Scale (NOS) [ 34 ] (Wells et al. , 2011) Cohort studies and case-control studies Yes 4 SQs with multiple choice answers (2 or 3 per question), with a star collected for each best answer, and a maximum of four stars indicating low risk of selection bias. Cross-sectional studies JBI Checklist for analytical cross-sectional studies [ 61 ] (JBI website) Analytical cross-sectional studies No 1 SQ, rated yes/ no/ unclear/ not applicable Newcastle-Ottowa scale for cross-sectional studies (NOS-xs) [ADD REF Carra, 2025] Analytical cross-sectional studies and prevalence studies No 1 SQ with 4 multiple choice answers, with 2 deemed acceptable JBI Checklist for prevalence studies [ 62 ] (Munn et al. , 2020) Prevalence studies No 2 SQs, rated yes/ no/ unclear/ not applicable RoB-PrevMH [ 63 ] (Tonia et al. , 2023) Studies measuring the prevalence of mental health disorders Yes 2 SQs, rated high/ low/ unclear risk of bias. RoB-SPEO [ 64 ] (Pega et al. , 2020) Studies measuring the prevalence of exposure to occupational risk factors N/A (each SQ makes up its own domain) 2 SQs, rated low/ probably low/ probably high/ no information Case series JBI Checklist for case series [ 65 ] (Munn et al. , 2020) Case series No 3 SQs, rated yes/ no/ unclear/ not applicable Survey Risk-of-bias tool for urban planning surveys [ 66 ] (Ravensbergen & El-Geneidy, 2023) Urban planning surveys Yes 2 SQs, jointly assessed as weak/moderate/strong evidence against selection bias Diagnostic accuracy studies JBI checklist for diagnostic accuracy studies [ 67 ] (Cambell et al. , 2015) Diagnostic accuracy studies No 3 SQs rated yes/ no/ unclear/ not applicable QUADAS-2 [ 68 ] (Whiting et al. , 2011) Diagnostic accuracy studies Yes 3 SQs rated yes/ no/ unclear, with an overall rating of low/ high/ unclear risk of bias. Symptom and Performance Validity Studies RoB-spv (Puente-López et al. , 2025) Psychology studies using simulation or criterion-group designs to assess the performance of a validity test in classifying genuine vs feigned presentations No 4 SQs rated yes/ probably yes/ probably no/ no/ not informative (for simulation designs), with an additional 4 SQs if a clinical comparison group is included 6 SQs rated yes/ probably yes/ probably no/ no/ not informative (for criterion group designs) For 11 tools, signalling questions relating to selection bias were presented in a specific ‘selection bias’ domain, with multiple signalling questions pertaining to selection bias. In others, selection-bias specific questions were included in the ‘confounding bias’ domain or grouped with missing data questions (n = 1) or presented separately (n = 12). In tools specific to RCTs – where selection bias stems from errors in the allocation process [ 36 ], relevant signalling questions were also identified in domains focussing on randomisation and blinding. A signalling question is answered with a rating such as ‘yes’/‘no’/‘unclear’, and most tools provide clear guidance on these ratings (e.g., a ‘yes’ implies risk of selection bias). Only three tools (RoB-2 [ 9 ] for RCTs and ROBINS-I/ROBINS-E [ 37 , 38 ] for non-randomised studies of interventions/exposures) provide an algorithm for combining the evidence of the signalling questions into an overall risk of selection bias judgment. The remaining tools either do not contain a dedicated selection-bias domain (n = 13), or do, but leave such a final assessment up to the discretion of the investigator (n = 8). Supplementary Table S1 provides an alphabetical comparison of the identified qualitative risk-of-bias tools, detailing their operational components. This includes the level of guidance provided for question assessment, the specific rating outcome structure used for signalling questions, and, where relevant, relationship to other tools. Many of the available tools build on previous ones, taking existing signalling questions and adding new ones where relevant, or adapting existing questions to a specific type of study (e.g., existing signalling questions for RCTs may be modified to suit RCTs of animal intervention studies). In particular, many later studies cite the Rob-2 tool (or its predecessor RoB) or the ROBINS-I/E tools, developed by the Cochrane Collaboration. When examining specific signalling questions per study design, overlap is found between many studies. Supplementary table S2 provides a comprehensive, categorized compilation of the selection bias signalling questions across all 24 tools. Questions are organized by study design and similarity, alongside an interpretation explaining the link between the question and the selection bias mechanism. Additionally, we have provided an interpretation on how specific questions relate to selection bias. A number of tools are aimed at specific subtypes of study, e.g. animal RCTs [ 39 ], operative interventions [ 33 ], etc. While the signalling questions are largely comparable to those of more general tools, such as RoB-2 and ROBINS-I, they contain subject-specific examples for interpreting the signalling questions, which may prove helpful. 4.3 QBA software programs Table 2 summarises the key features of the nine QBA software programs identified by our review: five implemented in R, three in Stata, and one as a web tool. Table 2 Software programs implementing a quantitative bias analysis to selection bias Analysis of interest Quantitative bias analysis Software platform Name Outcome Exposure Estimand Covariates a No. bias parameters Graphical plot Other sources of bias Population mean or proportion R nrba [ 69 , 70 ] Continuous, binary Not applicable Mean, proportion Not applicable 1 None None Exposure effect Web tool apisensr [ 25 ] Binary Binary Risk ratio, odds ratio None 1 or 4 Histogram, b DAG Confounding, misclassification R episensr [ 25 , 71 ] Binary Binary Risk ratio, odds ratio None 1 or 4 Histogram, forest plot, DAG Confounding, misclassification R EValue [ 41 , 72 ] Binary, time-to-event Binary, categorical, continuous d Risk ratio, d odds ratio, d hazards ratio None 1, 2, or 4 None Confounding, misclassification R multibias [ 73 , 74 ] Binary Binary Odds ratio Yes 3 Forest plot Confounding, misclassification R SelectionBias [ 40 , 43 , 75 , 76 ] Binary Binary d Risk difference, d risk ratio None 0, 2, 6 None None Stata biasepi Binary Binary Risk ratio, risk difference, odds ratio None 4 None Confounding, misclassification Stata episens [ 16 ] Binary Binary Odds ratio None 1or 4 Histogram Confounding, misclassification Stata rori [ 77 ] Binary Binary, categorical Proportion, Odds ratio None \(\:\ge\:1\) None None a: Of a QBA to selection bias only. b: Directed Acyclic Graph. c: Exponential distribution. d: In the total population and selected population. 4.3.1 Types of analysis models R package nrba is only applicable when the estimand of interest is a population mean or proportion. The R package outputs estimates of the selection bias and includes other functions to check for non-random selection into the study (such as a comparison of the study’s estimated means to benchmarks from an external source). Note that the web version of nrba does not conduct a QBA and so was excluded here. The remaining eight programs evaluate the sensitivity of an exposure effect estimate to selection bias. Note that apisensr is the web-based version of R package episensr but with slightly reduced functionality. All eight programs implement a QBA for a binary outcome and binary exposure. R package EValue and Stata command rori can be used in other analysis scenarios such as a categorical or continuous exposure, or a time-to-event outcome. R package multibias includes an option to specify covariates required to adjust for measured confounding. For the remaining seven programs (web tool apisensr , R packages episensr, EValue , and SelectionBias , and Stata commands biasepi , episens , and rori ) adjustment for measured confounding can be achieved by conducting the QBA within strata defined by the covariates (or within levels of a propensity score that summarises the measured confounders) or by applying the QBA to data weighted by the inverse of a propensity score [ 10 , 40 , 41 ]. None of these software programs include any features to help the user conduct a stratified or weighted QBA. 4.3.2 Bias adjustment methods Software programs apisensr , episensr , biasepi and episens calculate the bias-adjusted estimate using the same bias formula [ 42 ], where the four bias parameters are selection probabilities (e.g., probability of selection among exposed cases) which for some programs can be alternatively expressed as a single selection odds ratio. R package multibias adjusts for selection bias using weights generated from a logistic regression model. Stata command rori has a single bias parameter, the relative odds ratio, which is the ratio of the odds ratio among the selected participants divided by the odds ratio in the source population. For a binary exposure, rori computes the relative odds ratio from a \(\:2\times\:2\) tabulation of the exposure and outcome among the selected participants and the same \(\:2\times\:2\) tabulation in the source population. Lastly, R packages EValue and SelectionBias both implement a QBA using the Smith and VanderWeele (SV) bounds for selection bias and are applicable when inference is in the total population and in a selected population [ 41 ]. However, there are several differences between these two R packages: (1) EValue reports the minimum value of the bias parameters (when all equal) to change a study conclusion (e.g., give a null point estimate or an estimate opposite in sign to the naïve point estimate) whereas SelectionBias reports the lower and upper bound for the bias-adjusted estimate given user-specified values of the bias parameters. (2) SelectionBias reports sharp lower and upper SV bounds (i.e., tightest possible bounds given the necessary assumptions of the QBA). (3) SelectionBias also provides alternatives to the SV bounds; namely the assumption-free bound, the generalised assumption-free bound and the counterfactual assumption-free bound [ 43 ]. 4.3.3 Specialist QBA features R package nrba includes features to implement a multidimensional QBA such as enabling the user to specify multiple values for the bias parameter and outputting a table of the multiple estimates of the bias-adjusted mean or proportion. However, nrba does not include any features to identify bias parameter values that correspond to a pre-specified tipping point. In contrast, R package EValue implements a tipping point analysis but not a multidimensional QBA (i.e., does not output any bias-adjusted estimates). Instead, EValue reports the minimum value(s) of the bias parameter(s) required to change a study conclusion. Software programs apisensr , episensr , and episens include features to implement a probabilistic QBA using the Monte Carlo Sensitivity Analysis approach [ 10 ], outputting a histogram of the frequency distribution of bias-adjusted estimates (and also draws from the bias parameters’ prior distributions for episens and episensr ). Unfortunately, R packages multibias and SelectionBias and Stata commands biasepi and rori do not include any features to conduct a multidimensional or probabilistic QBA or a tipping point analysis. Consequently, the user is required to write their own code to conduct a full QBA using these programs. For example, to implement a multidimensional QBA the user must write code to repeatedly call the software program for different values of the bias parameter(s), store the multiple bias-adjusted estimates and present the results in a graphical plot or table. 4.3.4 Other specialist features Other specialist features include: (1) web tool apisensr and R package episensr generate directed acyclic graphs illustrating selection caused by M-bias (a form of collider bias [ 1 ]). (2) R packages multibias and episensr output a forest plot of the results of the naïve estimate and QBAs, and (3) multibias includes an option to specify the values of the bias parameters using an external validation dataset. 4.3.5 Multiple QBA Six software programs also implement a QBA to other sources of bias and implement a multiple QBA (i.e., assess sensitivity to the combined impact of multiple sources of bias). Software programs apisensr , episensr , EValue , multibias , biasepi , and episens can be used to conduct a QBA to unmeasured confounding, misclassification of the exposure, or misclassification of the outcome, or a multiple QBA to any combination of selection bias, unmeasured confounding, and misclassification. Note that apisensr , episensr , EValue , biasepi , and episens sequentially adjust for the multiple sources of bias in the reverse order in which they occurred in the data (e.g., adjust for outcome misclassification within the selected sample and then adjust for selection bias). In contrast, multibias simultaneously adjusts for multiple sources of bias, avoiding the need to specify the order in which the biases occur. 5. Discussion We have conducted an up-to-date review of software implementations of risk-of-bias tools and QBA to selection bias. 5.1 Risk-of-bias tools review In our review, we identified a subset of risk-of-bias tools through a literature search, supplemented with a cross-reference search, restricting our focus to tools that contain at least one dedicated signalling question pertaining to selection bias We examined the selection bias components of these 24 tools—which varied from single questions to multi-item domains—and detailed their target study types, assessment methods, and levels of guidance. To the best of our knowledge, no similar report of the selection bias components of qualitative risk-of-bias tools has previously been published. While qualitative risk-of-bias tools can be a valuable resource for assessing the risk of selection bias, or, more generally, the overall methodological quality of a study, this is subject to the correct application of the tool. A methodological systematic review of 124 applied systematic reviews [ 44 ] found that the ROBINS-I tool was frequently applied incorrectly, and noted that ROBINS-I is a complex tool requiring extensive methodological understanding, which is too frequently applied by authors lacking the necessary expertise. Various studies have been published on inter-rater agreement for tools such as ROBINS-I and NOS [ 45 ], RoBANS [ 46 ], ROB-2 [ 47 ], and the original ROB tool (the predecessor of ROB-2) [ 48 ], with studies reporting limited [ 47 ], moderate [ 46 , 48 ] and good agreement [ 45 ] between reviewers. For ROB-2, the authors found substantial improvement in the reliability of the tool after developing topic-specific implementation instructions. Overall, when reported, agreement on selection-bias specific signalling questions and domains, while varying, was generally judged to be ‘fair’. It should be noted that all assessments were performed by reviewers with prior methodological and/or systematic review experience and, in some cases, tool-specific training [ 45 ]. A recent publication by Sotiropoulos et al. [ 49 ] provides methodological guidance for assessing the different bias domains of the RoB-2 tool when there is access to individual-level patient data, instead of summary-level data as is common for systematic reviews. A challenge to the uptake of qualitative bias tools for assessing the risk of (selection) bias is the volume of tools and their variation in the granularity and degree of guidance for signalling questions, which increases reliance on investigator expertise. To help researchers synthesize this information, we have provided a comprehensive overview of selection bias signalling questions per study design (Supplementary Table 2). 5.2 QBA software review Our review identified nine software programs of QBA to selection bias, applicable to summary-level and individual-level data. Our identification of only a small number of tools is in keeping with a recent review (published in 2024) of QBA methods applicable to summary-level data [ 14 ] which identified only a single software program (R package EValue ) and previous systematic reviews of applied analyses which found that QBA studies were far less likely to conduct a QBA to selection bias compared to unmeasured confounding and misclassification [ 11 – 13 ]. This has partly been attributable to the lack of available methodology for QBA to selection bias [ 14 ]. Most of the software programs we identified are only applicable for a binary outcome and exposure. We did not find any programs applicable for estimation of an exposure effect when the outcome is continuous or categorical. Also, none of the programs applicable for estimation of a bias-adjusted exposure effect included any specialist features to aid the user in carrying out a multidimensional QBA (i.e., such as estimation of the bias-adjusted exposure effect for all value-combinations of the bias parameters and graphical presentation of the multiple estimates to aid interpretation). Some published QBA methods were excluded because the provided software code did not meet our software criteria (e.g., no user interface, not accompanied by documentation, or required code adaption before application to another example). For example, R function nisb() for calculating indices of selection bias for means and/or proportions estimated from non-probability samples [ 50 , 51 ]; (undocumented) R package scbounds for calculation of bounds for a population mean estimated from a non-probability sample [ 52 ]; Flanders and Ye provide R code for their bound of bias arising from M-bias [ 53 ]; and Gelman and Carpenter provide R and Stan code for their Bayesian analyses of testing for a rare disease with unknown specificity and sensitivity and using data from an unrepresentative sample of the target population [ 54 ]. 5.3 Strengths and limitations Our study covers more than 20-year period, from 1 January 2004 up to the end of August 2025, capturing all available risk-of-bias tools and QBA software programs for individual-level data and summary-level data. Also, in order to capture tools and applications across different research fields, our search strategy used a diverse range of terminology for sample selection bias. One limitation of our search strategy is its restriction to the published literature and searchable databases for Stata and R implementations. Thus, our review may have excluded risk-of-bias tools and software programs that are only available from a search of the internet or from databases of other software environments. Note that our review of the unpublished literature focused on Stata and R because they are widely used among epidemiologists and medical statistician, with freely available user-written applications [ 55 ]. On balance, we believe that our search strategy mimics the way in which an applied researcher would source their software and so our results reflect the tools and software accessible to most researchers. A second limitation is that we do not provide information on the quality of the risk-of-bias tools or software programs. However, our review’s inclusion criteria did exclude all risk-of-bias tools that were still under construction or not fully validated and all undocumented or partially developed QBA software programs. Thirdly, our review did not address publication bias. Although publication bias involves a form of selection, it occurs at the synthesis level (selection of studies for publication) rather than at the study level. Our review was strictly limited to the assessment of selection bias at the study level, focusing on the internal validity of individual studies. 6. Conclusions While there are many qualitative risk-of-bias tools that assess selection bias for a variety of study designs and areas of research, there is considerable variation in granularity and guidance, and their successful application requires sufficient methodological expertise. To facilitate wider uptake of QBA, greater provision of QBA software to selection bias is needed, along with detailed QBA guidelines. Abbreviations QBA quantitative bias analysis CRAN Comprehensive R archive Network SSC Statistical Software Components Declarations Ethics approval and consent to participate# Not applicable Consent for publication Not applicable Availability of data and materials Data sharing is not applicable to this article as no datasets were generated or analysed during the current study. Competing interests The authors declare that they have no competing interests. Funding AH received research support from AstraZeneca. KT, RAH and EC work in the MRC Integrative Epidemiology Unit which is supported by the UK Medical Research Council and the University of Bristol (MC_UU_00032/2). RAH was supported by a Sir Henry Dale Fellowship that was jointly funded by the Wellcome Trust and the Royal Society (grant 215408/Z/19/Z). Authors’ contributions RAH proposed and designed the project. AH, EC, and RAH conducted the scoping review. All authors drafted the manuscript and reviewed and approved the final version of the manuscript. Acknowledgements Not applicable. References Smith LH. Selection Mechanisms and Their Consequences: Understanding and Addressing Selection Bias. Curr Epidemiol Rep. 2020;7(4):179–89. Madley-Dowd P, et al. Using directed acyclic graphs to determine whether multiple imputation or subsample-multiple imputation estimates of an exposure-outcome association are unbiased. Am J Epidemiol. 2025;195(2):505–14. Enders CK. Missing data: An update on the state of the art. Psychol Methods. 2025;30(2):322–39. Lee KJ, et al. Framework for the treatment and reporting of missing data in observational studies: The Treatment And Reporting of Missing data in Observational Studies framework. J Clin Epidemiol. 2021;134:79–88. Little RJ. Missing Data Analysis. Ann Rev Clin Psychol. 2024;20(20):149–73. Hughes RA, et al. Accounting for missing data in statistical analyses: multiple imputation is not always the answer. Int J Epidemiol. 2019;48(4):1294–304. Grubic N, et al. Breaking Down Bias: A Methodological Primer on Identifying, Evaluating, and Mitigating Bias in Cardiovascular Research. Can J Cardiol. 2025;41(5):996–1009. Ma L-L, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Military Med Res. 2020;7(1):1–11. Sterne JAC, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. Fox MP, MacLehose RF, Lash TL. Applying Quantitative Bias Analysis to Epidemiologic Data. New York: Springer; 2021. Hunnicutt JN, et al. Probabilistic bias analysis in pharmacoepidemiology and comparative effectiveness research: a systematic review. Pharmacoepidemiol Drug Saf. 2016;25(12):1343–53. Petersen JM, et al. A systematic review of quantitative bias analysis applied to epidemiological research. Int J Epidemiol. 2021;50(5):1708–30. Akbaş KE, Hark BD. Evaluation of quantitative bias analysis in epidemiological research: A systematic review from 2010 to mid-2023. J Eval Clin Pract. 2024;30(7):1413–21. Shi X, et al. Quantitative bias analysis methods for summary-level epidemiologic data in the peer-reviewed literature: a systematic review. J Clin Epidemiol. 2024;175:111507. Infante-Rivard C, Cusson A. Reflection on modern methods: selection bias—a review of recent developments. Int J Epidemiol. 2018;47(5):1714–22. Orsini N, Bellocco R, Greenland S. EPISENS: Stata module for basic sensitivity analysis of epidemiological results; in Statistical Software Components. 2008, Boston College Department of Economics,. [Accessed 25 March 2026]; Available from: https://ideas.repec.org/c/boc/bocode/s456792.html Griffith GJ, et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nat Commun. 2020;11(1):5749. Griffith GJ. AscRtain . [Accessed 21 January 2026]; Available from: http://apps.mrcieu.ac.uk/ascrtain/ Grönwold R, Palmer TM, Tilling K. Bias App . [Accessed 25 March 2026]; Available from: https://remlapmot.shinyapps.io/bias-app/#section-selection-bias Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15(5):615–25. Lu H, et al. Toward a Clearer Definition of Selection Bias When Estimating Causal Effects. Epidemiology. 2022;33(5):699–706. Lu H, et al. The evolution of selection bias in the recent epidemiologic literature—a selective overview. Am J Epidemiol. 2024;194(3):580–4. Mathur MB, Shpitser I. Simple graphical rules for assessing selection bias in general-population and selected-sample treatment effects. Am J Epidemiol. 2024;194(1):267–77. Hernán MA. Invited Commentary: Selection Bias Without Colliders. Am J Epidemiol. 2017;185(11):1048–50. Banack HR, Smith SN, Bodnar LM. Application of a Web-based Tool for Quantitative Bias Analysis: The Example of Misclassification Due to Self-reported Body Mass Index. Epidemiology. 2024;35(3):359–67. McCandless LC, Gustafson P. A comparison of Bayesian and Monte Carlo sensitivity analysis for unmeasured confounding. Stat Med. 2017;36(18):2887–901. Kawabata E, et al. Accounting for bias due to outcome data missing not at random: comparison and illustration of two approaches to probabilistic bias analysis: a simulation study. BMC Med Res Methodol. 2024;24(1):278. Zuckarelli J, packagefinder. Comfortable Search for R Packages on CRAN, Either Directly from the R Console or with an R Studio Add-in; in. 2023. [Accessed 25 March 2026]; Available from: https://cran.r-project.org/web/packages/packagefinder/index.html Csárdi G. and M. Salmon. pkgsearch: Search and Query CRAN R Packages; in. 2025. [Accessed 25 March 2026]; Available from: https://cran.r-project.org/web/packages/pkgsearch/index.html IDEAS/RePEc. Economic and Finance Research [Accessed 25 March 2026]; Available from: https://ideas.repec.org/ StataCorp. Stata Statistical Software; in. StataCorp LLC. [Accessed; Available from; 2025. Faillie J-L, et al. A new risk of bias checklist applicable to randomized trials, observational studies, and systematic reviews was developed and validated to be used for systematic reviews focusing on drug adverse events. J Clin Epidemiol. 2017;86:168–75. Luijken K, et al. How to assess applicability and methodological quality of comparative studies of operative interventions in orthopedic trauma surgery. Eur J Trauma Emerg Surg. 2022;48(6):4943–53. Wells G et al. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses . 2011 [Accessed 25 March 2026]; Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp National Toxicology Program. OHAT Risk of Bias Rating Tool for Human and Animal Studies. 2015 [Accessed 25 March 2026]; Available from: https://ntp.niehs.nih.gov/sites/default/files/ntp/ohat/pubs/riskofbiastool_508.pdf Kahan BC, Rehal S, Cro S. Risk of selection bias in randomised trials. Trials. 2015;16:405. Sterne JA, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. Higgins JPT, et al. A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E). Environ Int. 2024;186:108602. Hooijmans CR, et al. SYRCLE's risk of bias tool for animal studies. BMC Med Res Methodol. 2014;14:43. Zetterstrom S, Waernbaum I. SelectionBias: Calculates Bounds for the Selection Bias for Binary Treatment and Outcome Variables; in. 2024. [Accessed 25 March 2026]; Available from: https://cran.r-project.org/web/packages/SelectionBias/index.html Smith LH, VanderWeele TJ. Bounding Bias Due to Selection. Epidemiology. 2019;30(4):509–16. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25(6):1107–16. Zetterstrom S. Bounds for selection bias using outcome probabilities. Epidemiol Methods, 2024. 13s2). Igelström E, et al. Cochrane's risk of bias tool for non-randomized studies (ROBINS-I) is frequently misapplied: A methodological systematic review. J Clin Epidemiol. 2021;140:22–32. Zhang Y, et al. The ROBINS-I and the NOS had similar reliability but differed in applicability: A random sampling observational studies of systematic reviews/meta-analysis. J Evid Based Med. 2021;14(2):112–22. Seo HJ, et al. RoBANS 2: A Revised Risk of Bias Assessment Tool for Nonrandomized Studies of Interventions. Korean J Fam Med. 2023;44(5):249–60. Minozzi S, et al. Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022;141:99–105. Daly A, et al. Tolerating bad health research (part 2): still as many bad trials, but more good ones too. Trials. 2025;26(1):110. Sotiropoulos JX et al. Individual participant data informed risk of bias assessments for randomized controlled trials in systematic reviews and meta-analyses. J Clin Epidemiol, 2025. 185. Little RJA, et al. Measures of the Degree of Departure from Ignorable Sample Selection. J Surv Stat Methodol. 2019;8(5):932–64. Andridge RR, et al. Indices of Non-Ignorable Selection Bias for Proportions Estimated from Non-Probability Samples. J Royal Stat Soc Ser C: Appl Stat. 2019;68(5):1465–83. Miratrix LW, Wager S, Zubizarreta JR. Shape-constrained partial identification of a population mean under unknown probabilities of sample selection. Biometrika. 2017;105(1):103–14. Flanders WD, Ye D. Limits for the Magnitude of M-bias and Certain Other Types of Structural Selection Bias. Epidemiology. 2019;30(4):501–8. Gelman A, Carpenter B. Bayesian Analysis of Tests with Unknown Specificity and Sensitivity. J R Stat Soc Ser C Appl Stat. 2020;69(5):1269–83. Sriram R. Top 10 Statistical Tools Used in Medical Research . 2020 [Accessed 25 March 2026]; Available from: https://www.kolabtree.com/blog/top-10-statistical-tools-used-in-medical-research/ Barker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evid Synth. 2023;21(3):494–506. Barker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies. JBI Evid Synth. 2024;22(3):378–88. Slim K, et al. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg. 2003;73(9):712–6. Barker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for cohort studies. JBI Evid Synth. 2025;23(3):441–53. Joanna Briggs Institute. JBI Critical Appraisal Checklist for Case Control Studies. 2017 [Accessed 25 March 2026]; Available from: https://jbi.global/sites/default/files/2019-05/JBI_Critical_Appraisal-Checklist_for_Case_Control_Studies2017_0.pdf Joanna Briggs Institute. JBI Critical Appraisal Checklist for Analytical Cross Sectional Studies. 2020 [Accessed 25 March 2026]; Available from: https://jbi.global/sites/default/files/2020-08/Checklist_for_Analytical_Cross_Sectional_Studies.pdf Munn Z, et al. Chap. 5: Systematic reviews of prevalence and incidence . In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. Editors: JBI; 2020. Tonia T et al. Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH). BMJ Ment Health, 2023. 26(1). Pega F, et al. RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environ Int. 2020;135:105039. Munn Z, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evid Synth. 2020;18(10):2127–33. Ravensbergen L, El-Geneidy A. Toward Evidence-Based Urban Planning. J Am Plann Association. 2023;89(3):389–98. Campbell JM, et al. Diagnostic test accuracy: methods for systematic review and meta-analysis. Int J Evid Based Healthc. 2015;13(3):154–62. Whiting PF, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36. Schneider B, et al. nrba: Methods for Conducting Nonresponse Bias Analysis; in. Westat, Inc. [Accessed; Available from; 2023. Lohr SL, Riddles MK, Morganstein D. Tests for evaluating nonresponse bias in surveys. Surv Methodol. 2016;42(2):195–218. Haine D. The episensr package: Basic sensitivity analysis of epidemiological results; in. 2025. [Accessed 25 March 2026]; Available from: https://dhaine.codeberg.page/episensr/ Mathur MB et al. EValue: Sensitivity Analyses for Unmeasured Confounding and Other Biases in Observational Studies and Meta-Analyses; in. 2021. [Accessed Accessed 25 March 2026]; Available from: https://cran.r-project.org/web/packages/EValue/index.html Brendel P, Torres A, Arah OA. Simultaneous adjustment of uncontrolled confounding, selection bias and misclassification in multiple-bias modelling. Int J Epidemiol. 2023;52(4):1220–30. Brendel Pmultibias. Multiple Bias Analysis in Causal Inference; in. 2025. [Accessed 25 March 2026]; Available from: https://cran.r-project.org/web/packages/multibias/index.html Zetterstrom S, Waernbaum I. Selection bias and multiple inclusion criteria in observational studies. Epidemiol Methods, 2022. 11(1). Zetterstrom S, Sjölander A, Waernbaum I. Investigations of sharp bounds for causal effects under selection bias. Stat Methods Med Res. 2025;34(12):2270–83. Nohr EA, Liew Z. How to investigate and adjust for selection bias in cohort studies. Acta Obstet Gynecol Scand. 2018;97(4):407–16. Additional Declarations No competing interests reported. Supplementary Files SupplementsoftwarereviewselectionbiasHazewinkel.pdf Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 15 May, 2026 Reviewers agreed at journal 03 May, 2026 Reviewers invited by journal 21 Apr, 2026 Editor invited by journal 07 Apr, 2026 Editor assigned by journal 05 Apr, 2026 Submission checks completed at journal 05 Apr, 2026 First submitted to journal 31 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9283156","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":615451574,"identity":"f8c4ffcb-d9b5-46e1-8e99-22d150eef56e","order_by":0,"name":"Audinga-Dea Hazewinkel","email":"","orcid":"","institution":"London School of Hygiene \u0026 Tropical Medicine","correspondingAuthor":false,"prefix":"","firstName":"Audinga-Dea","middleName":"","lastName":"Hazewinkel","suffix":""},{"id":615451575,"identity":"5a01ad28-e444-48a6-b7a6-324a43e108f5","order_by":1,"name":"Elinor Curnow","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Elinor","middleName":"","lastName":"Curnow","suffix":""},{"id":615451576,"identity":"41f746ec-9357-4fbe-8f54-ed3a1d55c1ed","order_by":2,"name":"Kate Tilling","email":"","orcid":"","institution":"University of Bristol","correspondingAuthor":false,"prefix":"","firstName":"Kate","middleName":"","lastName":"Tilling","suffix":""},{"id":615451577,"identity":"b651fa9b-2b5f-492f-b270-a4d0cd2a1f97","order_by":3,"name":"Rachael Ann Hughes","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABKUlEQVRIiWNgGAWjYHACNiC2YDCAMGwSGJhBggVgKWY8WiRgWtJAWhgbQFxitRxOABL4tZi3H3724EeFBIO5RPq1Bx/+nM8zb2d//uCDAYM8fwOPsQEWLTJn0swNe85IMFjOyCk3nNl2u1jmMI9h4wwDBsMZB3iME7BokWDIYZPgbQM67EZOmjRvw+3EGcw8jM08BgyMGxh4jA9g08L/hk3y7z+olj9/zgG1sD8EabHHqUUihw1oOEhL+jFpBrYDQC0MhiAtiSAtWB0m8cxMWuaYBI/BGaB1vW3JxRLMPIYzZxhIJM84zFaMzfsS/MnPJN/U2MgZHE9/JvHjj12eBP/xBx8+VNjY9rc3b5bAFspQwANEKEZK4I5IBGB/QFDJKBgFo2AUjEwAAEEGWKUn0c2zAAAAAElFTkSuQmCC","orcid":"","institution":"University of Bristol","correspondingAuthor":true,"prefix":"","firstName":"Rachael","middleName":"Ann","lastName":"Hughes","suffix":""}],"badges":[],"createdAt":"2026-03-31 17:38:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9283156/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9283156/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106093750,"identity":"3f25db6a-42db-4a9d-926b-c4fb869eaf5b","added_by":"auto","created_at":"2026-04-03 11:38:57","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":150306,"visible":true,"origin":"","legend":"\u003cp\u003eFlowchart of the selection process for the Web of Science search, the Stata search and the R search\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-9283156/v1/96dad77f9c702a9e3f2992ed.png"},{"id":106095617,"identity":"7076a2f0-7f6a-49d4-908f-fe18a703c43d","added_by":"auto","created_at":"2026-04-03 11:50:08","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1404097,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9283156/v1/a4e0bfe8-7226-48f4-ba82-e4a6b51efe62.pdf"},{"id":105990099,"identity":"7de9e787-3b6a-4a72-b3c3-df83e7e00a18","added_by":"auto","created_at":"2026-04-02 08:14:56","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":636969,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementsoftwarereviewselectionbiasHazewinkel.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9283156/v1/55a2a91c6c161c09aec0e583.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias","fulltext":[{"header":"1. Background","content":"\u003cp\u003eThe main aim of many epidemiologic studies and randomised trials is to estimate the causal effect of an exposure (or treatment) on an outcome in some target population. Inference about the causal exposure effect (here onwards, shortened to \u003cem\u003eexposure effect\u003c/em\u003e) may be invalid when the sample included in the analysis (i.e., the \u003cem\u003eanalytic sample\u003c/em\u003e) is a non-representative sample of the target population. This could be due to selection into the study (non-random sampling of the target population or non-participation of individuals invited into the study) or due to selection out of the study (e.g., participant dropout, loss to follow-up, or missing data). In general, bias due to selection into the study is far more challenging to identify and to correct for than selection out of the study because there is (virtually) no information available on the unselected participants [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Consequently, limited guidance is available on how to deal with bias due to selection into the study, in contrast to the numerous reviews and tutorial-style papers available on missing data (e.g., [\u003cspan additionalcitationids=\"CR3 CR4 CR5\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]). To address this gap in the literature, this paper focuses on selection into the study.\u003c/p\u003e \u003cp\u003eIn the absence of data on the unselected individuals, risk-of-bias tools may be used in systematic reviews of medical studies to identify potential selection bias [\u003cspan additionalcitationids=\"CR8\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. At the analysis stage, a quantitative bias analysis (QBA; also known as a sensitivity analysis) is recommended to quantitatively assess the sensitivity of a (meta-analysis or individual) study\u0026rsquo;s conclusions to plausible assumptions about the selection bias [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eQualitative risk-of-bias tools have been primarily designed and employed for appraising the methodological quality of studies in systematic reviews. Despite this primary use, these tools are increasingly accepted for investigating bias in individual studies [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. A wide range of risk-of-bias tools exists, covering diverse study types, from general designs like randomised trials and cohort studies, to highly specific areas like animal or mental health studies. These tools differ significantly in depth, ranging from simple checklists to complex, algorithm-based instruments with extensive guidance. A systematic overview of such tools was published by Ma et al. in 2020 [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. However, no systematic review has yet catalogued how this diverse collection of qualitative tools specifically assesses the risk of selection bias. From here onward, \u003cem\u003erisk-of-bias tools\u003c/em\u003e refers to those aspects of risk-of-bias tools that examine selection bias.\u003c/p\u003e \u003cp\u003eThree systematic reviews of analyses of real-data studies, published between 2010 and mid-2023, found that QBA to selection bias was far less common than QBA to unmeasured confounding and misclassification [\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. This may be partly attributed to a lack of knowledge about accessible software since there are no reviews of software implementing a QBA to selection bias. Currently, the limited information on software includes: the appendix of Shi et al [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], which lists the software (or links to supplied code) of QBA statistical methods applicable to summary-level data; Infante-Rivard and Cusson [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] summarise the Stata command \u003cem\u003eepisens\u003c/em\u003e [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e], and the appendix of Griffith et al [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] lists four software programs on quantifying selection bias, of which only two are still available (AscRtain [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] and Bias app [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]). Also, the website accompanying Fox et al\u0026rsquo;s seminal textbook on QBA [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] provides spreadsheets and software code to replicate QBA example analyses. From here onward, \u003cem\u003eQBA\u003c/em\u003e refers to QBA to selection bias.\u003c/p\u003e \u003cp\u003eTo address these gaps in the literature, we have conducted a scoping review of risk-of-bias tools and publicly available software implementations of QBAs, describing the key properties of each software program. We begin, in section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, with a brief overview of selection bias, risk-of-bias tools, and QBA. The methods and results of the scoping review are described in sections \u003cspan refid=\"Sec6\" class=\"InternalRef\"\u003e3\u003c/span\u003e and \u003cspan refid=\"Sec11\" class=\"InternalRef\"\u003e4\u003c/span\u003e, respectively, and we conclude with a discussion in section \u003cspan refid=\"Sec20\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e"},{"header":"2. Overview of risk-of-bias tools and QBA to selection bias","content":"\u003cp\u003eIn this section we briefly introduce statistical terms and concepts mentioned by the articles and documents included in our review.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Selection bias\u003c/h2\u003e \u003cp\u003eWe want to estimate the effect of an exposure (or treatment) \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:X\\)\u003c/span\u003e\u003c/span\u003e on an outcome \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Y\\)\u003c/span\u003e\u003c/span\u003e in some target population, where the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Y-X\\)\u003c/span\u003e\u003c/span\u003e association may be confounded by measured covariates \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:C\\)\u003c/span\u003e\u003c/span\u003e. We use a sampling design to select the study sample from our target population. All individuals of the study sample are invited to participate and those who agree will form the analytic sample. Data on \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:X,\\)\u003c/span\u003e\u003c/span\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Y,\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:C\\)\u003c/span\u003e\u003c/span\u003e are only recorded on individuals in the analytic sample and are used to estimate the exposure effect. We assume that in the absence of selection bias we obtain an unbiased estimate of the exposure effect.\u003c/p\u003e \u003cp\u003eWe consider the overall selection bias arising from a systematic difference between the true exposure effect in the target population and the estimate obtained from the analytic sample (we shall refer to as the \u003cem\u003ena\u0026iuml;ve estimate\u003c/em\u003e) [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. It can arise when the study sample is unrepresentative of the target population (e.g., due to an inappropriate sampling design) or when the analytic sample is unrepresentative of the target population (e.g., due to non-participation of invited individuals). Note that we do not distinguish between different types of selection bias defined according to the stage of the selection process (see [\u003cspan additionalcitationids=\"CR22\" citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] for more details). Also, in line with the articles and documents we reviewed, we do not distinguish between type I selection bias (due to restricting to one or more levels of a collider or a descendent of a collider) and type II selection (due to restricting to one or more levels of an effect measure modifier) [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. For a detailed discussion on the concept of selection bias in epidemiology and the different selection mechanisms that can arise see [\u003cspan additionalcitationids=\"CR21 CR22 CR23 CR24\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eRisk-of-bias tools\u003c/span\u003e\u003c/h2\u003e \u003cp\u003eRisk-of-bias tools aim to assess the quality of studies through a series of qualitative signalling questions, which may be presented as a checklist or be organised in different domains, with each domain focussing on a different source of bias (e.g., selection bias, confounding bias, missing data bias, etc.). These tools have been designed to aid in appraising the risk-of-bias for individual studies that are included in a systematic review, or to help decide on the inclusion or exclusion of a study. They can, however, also be used to assess the risk-of-bias in an individual study, and it is to that end that we describe these tools here. Each tool contains signalling questions to determine whether different types of biases may be present.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eQuantitative bias analysis\u003c/span\u003e\u003c/h2\u003e \u003cp\u003eA QBA quantifies the likely magnitude and direction of the selection bias under different plausible assumptions about the selection mechanism (assuming no other sources of bias). Generally, a QBA requires a model (known as a bias model) for the observed data, \u003cem\u003eY\u003c/em\u003e, \u003cem\u003eX\u003c/em\u003e and \u003cem\u003eC\u003c/em\u003e, and selection process \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:S\\)\u003c/span\u003e\u003c/span\u003e. The bias model will include one or more parameters \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e (known as bias or sensitivity parameters) which govern the magnitude and direction of the bias. For example, in the simple setting of binary \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:X\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Y\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e could be the probabilities of selection into the analytic sample for each of the four strata defined by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:X\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Y\\)\u003c/span\u003e\u003c/span\u003e. Since \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e cannot be estimated from the observed data, information about the likely values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e must be obtained from external sources (such as external validation studies, published literature, or expert opinion). Setting \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e to specified values enables estimation of the remaining parameters of the bias model and can provide an estimate of the exposure effect adjusted for selection bias (here onward, called the \u003cem\u003ebias-adjusted exposure effect estimate\u003c/em\u003e). By changing the values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e, a QBA estimates the bias-adjusted exposure effect under different assumptions about the selection process.\u003c/p\u003e \u003cp\u003eThere are two broad classes of QBA methods: deterministic and probabilistic [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. A deterministic QBA (sometimes called a multidimensional QBA) specifies a range of values for each bias parameter of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e and then calculates the bias-adjusted exposure effect estimate for each plausible combination of values. Typically, the results are displayed as a plot or table of the bias-adjusted estimates against different values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e. A tipping point analysis is a deterministic QBA where interest only lies in the values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e that correspond to a change of study conclusions. A probabilistic QBA specifies a prior probability distribution for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e to explicitly model the user\u0026rsquo;s assumptions about which value-combinations of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e are most likely to occur and to incorporate their uncertainty about \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e. The output is a distribution of bias-adjusted estimates which can be summarised to give a point estimate (i.e., the most likely bias-adjusted exposure effect estimate under the QBA\u0026rsquo;s assumptions) and an interval estimate to reflect the uncertainty due to the selection bias and sampling variability. A probabilistic QBA can be implemented as a fully Bayesian analysis (where the prior for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\)\u003c/span\u003e\u003c/span\u003e is combined with the likelihood function for the data) or as a Monte Carlo sensitivity analysis (where values for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varphi\\:\\:\\)\u003c/span\u003e\u003c/span\u003eare directly sampled from their prior distributions and then used to obtain estimates of the bias-adjusted exposure effect) [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Methods","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Summary of the search strategy\u003c/h2\u003e \u003cp\u003eWe conducted a scoping review of software implementations of risk-of-bias tools and QBAs made publicly available before 31st August 2025. We defined \u0026ldquo;software\u0026rdquo; to be either a web tool or software code that (i) was not specific to a particular data example (i.e., we excluded examples of code from empirical analyses that required code adaptation before application to another example), (ii) had a user interface, (iii) was accompanied by documentation detailing the software\u0026rsquo;s features, and (iv) was available for use on 31st August 2025.\u003c/p\u003e \u003cp\u003eWe conducted three searches: (1) a search of the published literature to identify risk-of-bias tools and software implementations of QBA, (2) a search of R packages implementing a QBA, available on the Comprehensive R archive Network (CRAN), and (3) a search of Stata commands implementing a QBA, including inbuilt commands and user-written commands available on the Boston College Statistical Software Components (SSC) archive or via Stata\u0026rsquo;s \u003cem\u003enet\u003c/em\u003e command.\u003c/p\u003e \u003cp\u003eEach search was conducted in three stages. In stage 1, we identified published articles or software documents that mentioned \u0026ldquo;selection bias\u0026rdquo; and \u0026ldquo;risk-of-bias tools\u0026rdquo; or \u0026ldquo;selection bias\u0026rdquo; and \u0026ldquo;bias analysis\u0026rdquo;, and their synonyms, in either the title, keywords, abstract or software description (see boxes 1\u0026ndash;8 in the Supplementary materials for the exact search terms).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Stage 1\u003c/h2\u003e \u003cp\u003eWe conducted a search of the published literature to identify software implementations of risk-of-bias tools and of QBA methods. We used Web of Science to search 10 databases (Web of Science core collection, BIOSIS citation index, Derwent innovations index, grants index, KCI Korean journal database, MEDLINE, policy citation index, preprint citation index, ProQuest dissertations \u0026amp; theses citation index, SciELO citation index). The Web of Science search query was designed to exclude articles solely focused on answering applied questions because they contain limited information on the statistical methodology used. Specifically, the Web of Science search query consisted of three substages: Substage 1.1 included articles that mentioned (in the title, abstract or keywords) terms (and their synonyms) relating to \u0026ldquo;software\u0026rdquo; or \u0026ldquo;developing a method\u0026rdquo; along with terms relating to \u0026ldquo;selection bias\u0026rdquo; and \u0026ldquo;risk-of-bias tools\u0026rdquo; or \u0026ldquo;selection bias\u0026rdquo; and \u0026ldquo;bias analysis\u0026rdquo; (Supplement box 1). Substage 1.2 excluded articles with a title containing terms relating to \u0026ldquo;meta-analysis\u0026rdquo; or \u0026ldquo;systematic review\u0026rdquo; and articles published in the Cochrane Database of Systematic reviews (Supplement box 2). Substage 1.3 added back in any articles excluded from substage 1.2 that mentioned in the title terms relating to \u0026ldquo;meta-analysis\u0026rdquo; or \u0026ldquo;systematic review\u0026rdquo; \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eand\u003c/span\u003e \u0026ldquo;bias\u0026rdquo; or \u0026ldquo;method\u0026rdquo; (Supplement box 3). We checked that substage 1.1 did not exclude any relevant methodological papers by rerunning the search excluding the software terms and examining the full text of 50 randomly selected articles that were not identified by substage 1.1; none of these were relevant papers that should have been included. Substage 1.3 was necessary to ensure we only excluded applied meta-analyses and systematic reviews. For more details see Supplement Section 1.1.\u003c/p\u003e \u003cp\u003eThe R and Stata searches were used to identify software implementations of QBA methods only. We used R packages \u003cem\u003epackagefinder\u003c/em\u003e (version 0.3.5) [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] and \u003cem\u003epkgsearch\u003c/em\u003e (version 3.1.5) [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] to search the CRAN database, and the RePEc IDEAS website [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] and the Stata \u003cem\u003esearch\u003c/em\u003e command (version 19 [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e]) to identify inbuilt commands (using option \u003cem\u003emanual\u003c/em\u003e) and user-written Stata commands (using option \u003cem\u003enet\u003c/em\u003e). For more details see Supplement Sections 1.2 and 1.3.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Stage 2\u003c/h2\u003e \u003cp\u003eAfter excluding any duplicate entries from stage 1, we proceeded to stage 2 where the abstracts of published articles or software descriptions were independently reviewed by two reviewers to determine if they were eligible for data extraction, with any disagreements resolved by consensus. Eligible abstracts of published articles either (i) introduced a new risk-of-bias tool for selection bias, (ii) introduced a new method or software implementing a QBA to selection bias, or (iii) compared or reviewed existing risk-of-bias or QBA methodology in the context of selection bias. Eligible software descriptions of R packages and Stata commands introduced software implementations of a QBA to selection bias. Ineligible abstracts or software descriptions were (i) meeting abstracts, (ii) articles where a QBA was not conducted but mentioned as further work, (iii) articles or software descriptions about a risk-of-bias tool or QBA software for other sources of bias (e.g., unmeasured confounding, missing data, information bias), and (iv) articles solely focused on answering applied questions.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Stage 3\u003c/h2\u003e \u003cp\u003eIn stage 3, we assessed the full text of the included articles. For the risk-of-bias tools, we extracted signalling questions relating to selection bias, and for QBA software we extracted information about the analysis of interest, the QBA method, and features of the software implementation. Also, we performed a concurrent reference list search on these full texts to identify additional relevant risk-of-bias tools and QBA software.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Results","content":"\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Study selection\u003c/h2\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows that our review identified 24 risk-of-bias tools and nine QBA software programs (additional details given in Supplement Section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e and Supplementary Fig.\u0026nbsp;1). The Web of Science search identified 1085 abstracts, of which we excluded 669 because they were either systematic reviews or meta-analyses aimed at answering an applied question, and 5 were duplicates or conference abstracts. We excluded 329 of the 411 screened abstracts and read the full text of the remaining 82 articles. A further 68 articles were excluded, of which 11 provided software code that did not satisfy our definition of a software program. Among the 14 included articles, 3 described software programs for QBA, while 11 described qualitative risk-of-bias tools. A reference search identified 13 additional qualitative risk-of-bias tools. The R and Stata searches identified a further 2 and 4 software programs, respectively. A detailed breakdown of the selection steps and intermediate results for each search are given in Supplement Sections 2.1, \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e and \u003cspan refid=\"Sec5\" class=\"InternalRef\"\u003e2.3\u003c/span\u003e for the Web of Science search, R search, and Stata search, respectively.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Risk-of-bias tools\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e gives an overview of the 24 available tools, ordered according to study design, and, within this, alphabetically by first author name. Most of these tools have been developed for a specific type of study design, ranging from randomised clinical trials to non-randomised studies of interventions, such as cohort and case-control studies, to prevalence studies and diagnostic accuracy studies. Four tools (Faillie \u003cem\u003eet al.\u0026rsquo;\u003c/em\u003es checklist for drug adverse events [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e], Luijken \u003cem\u003eet al.\u003c/em\u003e\u0026rsquo;s checklist for operative interventions [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e], Well \u003cem\u003eet al\u003c/em\u003e.\u0026rsquo;s Newcastle-Ottowa scale for cohort and case control studies [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]) and the OHAT risk-of-bias rating tool [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] contained signalling questions specific to multiple study designs.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eQualitative tools for risk of bias assessment\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTool\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDetail on target studies\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSelection bias specific domain\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eOutcome\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eName or descriptor of the qualitative assessment tool\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdditional detail on target studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDoes the tool contain a selection bias specific domain?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNumber of signalling questions (SQs) pertaining to selection bias and method of assessment\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eRandomised controlled trials\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChecklist for drug AEs [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Faillie \u003cem\u003eet al.\u003c/em\u003e, 2017)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRCTs and observational studies of drug adverse events.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4 SQs specific to RCTs, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChecklist for operative interventions [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Luijken \u003cem\u003eet al.\u003c/em\u003e, 2022)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRCTs and observational studies for operative interventions in orthopaedic trauma surgery\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo (selection bias specific SQs are given in domains \u003cem\u003eConfounding\u003c/em\u003e and \u003cem\u003eMissing data and selection bias\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4 SQs specific to RCTs, rated yes/ probably yes/ probably no/ no/ not informative. Per SQ, the rating is translated to an assessment of good/ moderate/ poor methodology.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI Checklist for RCTs [\u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Barker \u003cem\u003eet al.\u003c/em\u003e, 2023)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIndividually-randomised, parallel-group trials\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs, rated yes/ no/ unclear/ not applicable.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOHAT risk of bias rating tool [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(US National Toxicology Program)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnimal and human RCTs, cohort, case control and cross-sectional studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated probably/definitely low risk of bias, probably/definitely high risk of bias\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRoB-2 [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Sterne \u003cem\u003eet al.\u003c/em\u003e, 2019)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIndividually-randomised, parallel-group trials\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes (SQs are given in domain \u003cem\u003eRisk of bias arising from the randomization process\u003c/em\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs rated yes/ probably yes/ probably no/ no/ not informative. Provides an algorithm for combining evidence from the 3 SQs into an overall risk of bias judgement of low risk/some concerns/ high risk.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSYRCLE\u0026rsquo;s RoB tool [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e](Hooijmans \u003cem\u003eet al.\u003c/em\u003e, 2014)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnimal intervention studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes (SQs are given in domain \u003cem\u003eBias due inadequate randomisation and lack of blinding\u003c/em\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs, rated yes/no/unclear.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eNon-randomised studies of the effects of interventions or exposures\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChecklist for operative interventions [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Luijken \u003cem\u003eet al.\u003c/em\u003e, 2022)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRCTs and observational studies for operative interventions in orthopaedic trauma surgery\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo (selection bias specific SQs are given in the joint domain \u003cem\u003eMissing data and selection bias\u003c/em\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs specific to observational studies, rated yes/ probably yes/ probably no/ no/ not informative. Per SQ, the rating is translated to an assessment of good/ moderate/ poor methodology.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI checklist for quasi-experimental studies [\u003cspan citationid=\"CR57\" class=\"CitationRef\"\u003e57\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Barker \u003cem\u003eet al.\u003c/em\u003e, 2024)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eExperimental studies without random allocation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMINORS [\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Slim \u003cem\u003eet al.\u003c/em\u003e, 2003)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNon-randomised studies (without and with comparator)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, scored 0 (unreported), 1 (reported but inadequate) and 2 (reported and adequate)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOHAT risk of bias rating tool [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(US National Toxicology Program)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnimal and human RCTs, cohort, case control and cross-sectional studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated probably/definitely low risk of bias, probably/definitely high risk of bias\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRoBANS-2 [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Seo \u003cem\u003eet al.\u003c/em\u003e, 2023)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNon-randomised studies of interventions\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A (each SQ makes up its own domain)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated as indicative of a low/ high/ unclear risk of bias\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eROBINS-E [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Higgins \u003cem\u003eet al.\u003c/em\u003e, 2024)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNon-randomised studies of exposures\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eYes (SQs are given in domain \u003cem\u003eBias in selection of participants into the study\u003c/em\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e5 SQs rated yes/ probably yes/ probably no/ no/ not informative. Provides an algorithm for combining evidence from the 5 SQs into an overall risk of bias judgement of low/moderate/ serious/ critical/ not informative\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eROBINS-I [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Sterne \u003cem\u003eet al.\u003c/em\u003e, 2016)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNon-randomised studies of interventions\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCohort studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChecklist for drug AEs [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Faillie \u003cem\u003eet al.\u003c/em\u003e, 2017)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRCTs and observational studies of drug adverse events.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs specific to cohort studies, rated yes/ no/ unclear\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI checklist for cohort studies [\u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Barker \u003cem\u003eet al.\u003c/em\u003e, 2025)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCohort studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1 SQ, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNewcastle-Ottowa Scale (NOS) [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Wells \u003cem\u003eet al.\u003c/em\u003e, 2011)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCohort studies and case-control studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4 SQs with multiple choice answers (2 to 4 per question), with, for each SQ, a star indicating an acceptable answer (1 to 2 stars per SQ), and a maximum of four stars indicating low risk of selection bias.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCase control studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eChecklist for drug AEs [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Faillie \u003cem\u003eet al.\u003c/em\u003e, 2017)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRCTs and observational studies of drug adverse events.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs specific to case control studies, rated yes/ no/ unclear\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI checklist for case/control studies. [\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(JBI website)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCase/control studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNewcastle-Ottowa Scale (NOS) [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Wells \u003cem\u003eet al.\u003c/em\u003e, 2011)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCohort studies and case-control studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4 SQs with multiple choice answers (2 or 3 per question), with a star collected for each best answer, and a maximum of four stars indicating low risk of selection bias.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCross-sectional studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI Checklist for analytical cross-sectional studies [\u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e61\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(JBI website)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnalytical cross-sectional studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1 SQ, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNewcastle-Ottowa scale for cross-sectional studies (NOS-xs) [ADD REF Carra, 2025]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAnalytical cross-sectional studies and prevalence studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1 SQ with 4 multiple choice answers, with 2 deemed acceptable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI Checklist for prevalence studies [\u003cspan citationid=\"CR62\" class=\"CitationRef\"\u003e62\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Munn \u003cem\u003eet al.\u003c/em\u003e, 2020)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePrevalence studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRoB-PrevMH [\u003cspan citationid=\"CR63\" class=\"CitationRef\"\u003e63\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Tonia \u003cem\u003eet al.\u003c/em\u003e, 2023)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStudies measuring the prevalence of mental health disorders\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated high/ low/ unclear risk of bias.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRoB-SPEO [\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Pega \u003cem\u003eet al.\u003c/em\u003e, 2020)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStudies measuring the prevalence of exposure to occupational risk factors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A (each SQ makes up its own domain)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, rated low/ probably low/ probably high/ no information\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eCase series\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI Checklist for case series [\u003cspan citationid=\"CR65\" class=\"CitationRef\"\u003e65\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Munn \u003cem\u003eet al.\u003c/em\u003e, 2020)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCase series\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs, rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSurvey\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRisk-of-bias tool for urban planning surveys [\u003cspan citationid=\"CR66\" class=\"CitationRef\"\u003e66\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Ravensbergen \u0026amp; El-Geneidy, 2023)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUrban planning surveys\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2 SQs, jointly assessed as weak/moderate/strong evidence against selection bias\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eDiagnostic accuracy studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJBI checklist for diagnostic accuracy studies [\u003cspan citationid=\"CR67\" class=\"CitationRef\"\u003e67\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Cambell \u003cem\u003eet al.\u003c/em\u003e, 2015)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDiagnostic accuracy studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs rated yes/ no/ unclear/ not applicable\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eQUADAS-2 [\u003cspan citationid=\"CR68\" class=\"CitationRef\"\u003e68\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e(Whiting \u003cem\u003eet al.\u003c/em\u003e, 2011)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDiagnostic accuracy studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 SQs rated yes/ no/ unclear, with an overall rating of low/ high/ unclear risk of bias.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSymptom and Performance Validity Studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRoB-spv\u003c/p\u003e \u003cp\u003e(Puente-L\u0026oacute;pez \u003cem\u003eet al.\u003c/em\u003e, 2025)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePsychology studies using simulation or criterion-group designs to assess the performance of a validity test in classifying genuine vs feigned presentations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4 SQs rated yes/ probably yes/ probably no/ no/ not informative (for simulation designs), with an additional 4 SQs if a clinical comparison group is included\u003c/p\u003e \u003cp\u003e6 SQs rated yes/ probably yes/ probably no/ no/ not informative (for criterion group designs)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFor 11 tools, signalling questions relating to selection bias were presented in a specific \u0026lsquo;selection bias\u0026rsquo; domain, with multiple signalling questions pertaining to selection bias. In others, selection-bias specific questions were included in the \u0026lsquo;confounding bias\u0026rsquo; domain or grouped with missing data questions (n\u0026thinsp;=\u0026thinsp;1) or presented separately (n\u0026thinsp;=\u0026thinsp;12). In tools specific to RCTs \u0026ndash; where selection bias stems from errors in the allocation process [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e], relevant signalling questions were also identified in domains focussing on randomisation and blinding. A signalling question is answered with a rating such as \u0026lsquo;yes\u0026rsquo;/\u0026lsquo;no\u0026rsquo;/\u0026lsquo;unclear\u0026rsquo;, and most tools provide clear guidance on these ratings (e.g., a \u0026lsquo;yes\u0026rsquo; implies risk of selection bias). Only three tools (RoB-2 [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] for RCTs and ROBINS-I/ROBINS-E [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e] for non-randomised studies of interventions/exposures) provide an algorithm for combining the evidence of the signalling questions into an overall risk of selection bias judgment. The remaining tools either do not contain a dedicated selection-bias domain (n\u0026thinsp;=\u0026thinsp;13), or do, but leave such a final assessment up to the discretion of the investigator (n\u0026thinsp;=\u0026thinsp;8).\u003c/p\u003e \u003cp\u003eSupplementary Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e provides an alphabetical comparison of the identified qualitative risk-of-bias tools, detailing their operational components. This includes the level of guidance provided for question assessment, the specific rating outcome structure used for signalling questions, and, where relevant, relationship to other tools. Many of the available tools build on previous ones, taking existing signalling questions and adding new ones where relevant, or adapting existing questions to a specific type of study (e.g., existing signalling questions for RCTs may be modified to suit RCTs of animal intervention studies). In particular, many later studies cite the Rob-2 tool (or its predecessor RoB) or the ROBINS-I/E tools, developed by the Cochrane Collaboration. When examining specific signalling questions per study design, overlap is found between many studies. Supplementary table S2 provides a comprehensive, categorized compilation of the selection bias signalling questions across all 24 tools. Questions are organized by study design and similarity, alongside an interpretation explaining the link between the question and the selection bias mechanism. Additionally, we have provided an interpretation on how specific questions relate to selection bias. A number of tools are aimed at specific subtypes of study, e.g. animal RCTs [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e], operative interventions [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e], etc. While the signalling questions are largely comparable to those of more general tools, such as RoB-2 and ROBINS-I, they contain subject-specific examples for interpreting the signalling questions, which may prove helpful.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e4.3 QBA software programs\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e summarises the key features of the nine QBA software programs identified by our review: five implemented in R, three in Stata, and one as a web tool.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eSoftware programs implementing a quantitative bias analysis to selection bias\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"10\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c6\" namest=\"c3\"\u003e \u003cp\u003eAnalysis of interest\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c10\" namest=\"c8\"\u003e \u003cp\u003eQuantitative bias analysis\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSoftware platform\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eName\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eOutcome\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eExposure\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eEstimand\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eCovariates\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003csup\u003ea\u003c/sup\u003eNo. bias parameters\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eGraphical plot\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eOther sources\u003c/p\u003e \u003cp\u003eof bias\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"10\" nameend=\"c10\" namest=\"c1\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003ePopulation mean or proportion\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003enrba\u003c/em\u003e [\u003cspan citationid=\"CR69\" class=\"CitationRef\"\u003e69\u003c/span\u003e, \u003cspan citationid=\"CR70\" class=\"CitationRef\"\u003e70\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eContinuous, binary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eNot applicable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eMean, proportion\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNot applicable\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"10\" nameend=\"c10\" namest=\"c1\"\u003e \u003cp\u003e\u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eExposure effect\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWeb tool\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eapisensr\u003c/em\u003e [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eRisk ratio,\u003c/p\u003e \u003cp\u003eodds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1 or 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eHistogram, \u003csup\u003eb\u003c/sup\u003eDAG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eepisensr\u003c/em\u003e\u003c/p\u003e \u003cp\u003e[\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR71\" class=\"CitationRef\"\u003e71\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eRisk ratio,\u003c/p\u003e \u003cp\u003eodds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1 or 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eHistogram, forest plot, DAG\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eEValue\u003c/em\u003e\u003c/p\u003e \u003cp\u003e[\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e, \u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary,\u003c/p\u003e \u003cp\u003etime-to-event\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary, categorical, continuous\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003csup\u003ed\u003c/sup\u003eRisk ratio,\u003c/p\u003e \u003cp\u003e\u003csup\u003ed\u003c/sup\u003eodds ratio, \u003csup\u003ed\u003c/sup\u003ehazards ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1, 2, or 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003emultibias\u003c/em\u003e\u003c/p\u003e \u003cp\u003e[\u003cspan citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e, \u003cspan citationid=\"CR74\" class=\"CitationRef\"\u003e74\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eOdds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eForest plot\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eSelectionBias\u003c/em\u003e\u003c/p\u003e \u003cp\u003e[\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan citationid=\"CR75\" class=\"CitationRef\"\u003e75\u003c/span\u003e, \u003cspan citationid=\"CR76\" class=\"CitationRef\"\u003e76\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003csup\u003ed\u003c/sup\u003eRisk difference, \u003csup\u003ed\u003c/sup\u003erisk ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0, 2, 6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStata\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003ebiasepi\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eRisk ratio,\u003c/p\u003e \u003cp\u003erisk difference, odds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStata\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eepisens\u003c/em\u003e [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eOdds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1or 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eHistogram\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eConfounding, misclassification\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStata\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003erori\u003c/em\u003e [\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBinary\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eBinary, categorical\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eProportion,\u003c/p\u003e \u003cp\u003eOdds ratio\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\ge\\:1\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eNone\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"10\"\u003ea: Of a QBA to selection bias only. b: Directed Acyclic Graph. c: Exponential distribution. d: In the total population and selected population.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e4.3.1 Types of analysis models\u003c/h2\u003e \u003cp\u003eR package \u003cem\u003enrba\u003c/em\u003e is only applicable when the estimand of interest is a population mean or proportion. The R package outputs estimates of the selection bias and includes other functions to check for non-random selection into the study (such as a comparison of the study\u0026rsquo;s estimated means to benchmarks from an external source). Note that the web version of \u003cem\u003enrba\u003c/em\u003e does not conduct a QBA and so was excluded here.\u003c/p\u003e \u003cp\u003eThe remaining eight programs evaluate the sensitivity of an exposure effect estimate to selection bias. Note that \u003cem\u003eapisensr\u003c/em\u003e is the web-based version of R package \u003cem\u003eepisensr\u003c/em\u003e but with slightly reduced functionality. All eight programs implement a QBA for a binary outcome and binary exposure. R package \u003cem\u003eEValue\u003c/em\u003e and Stata command \u003cem\u003erori\u003c/em\u003e can be used in other analysis scenarios such as a categorical or continuous exposure, or a time-to-event outcome.\u003c/p\u003e \u003cp\u003eR package \u003cem\u003emultibias\u003c/em\u003e includes an option to specify covariates required to adjust for measured confounding. For the remaining seven programs (web tool \u003cem\u003eapisensr\u003c/em\u003e, R packages \u003cem\u003eepisensr, EValue\u003c/em\u003e, and \u003cem\u003eSelectionBias\u003c/em\u003e, and Stata commands \u003cem\u003ebiasepi\u003c/em\u003e, \u003cem\u003eepisens\u003c/em\u003e, and \u003cem\u003erori\u003c/em\u003e) adjustment for measured confounding can be achieved by conducting the QBA within strata defined by the covariates (or within levels of a propensity score that summarises the measured confounders) or by applying the QBA to data weighted by the inverse of a propensity score [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. None of these software programs include any features to help the user conduct a stratified or weighted QBA.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section3\"\u003e \u003ch2\u003e4.3.2 Bias adjustment methods\u003c/h2\u003e \u003cp\u003eSoftware programs \u003cem\u003eapisensr\u003c/em\u003e, \u003cem\u003eepisensr\u003c/em\u003e, \u003cem\u003ebiasepi\u003c/em\u003e and \u003cem\u003eepisens\u003c/em\u003e calculate the bias-adjusted estimate using the same bias formula [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e], where the four bias parameters are selection probabilities (e.g., probability of selection among exposed cases) which for some programs can be alternatively expressed as a single selection odds ratio. R package \u003cem\u003emultibias\u003c/em\u003e adjusts for selection bias using weights generated from a logistic regression model.\u003c/p\u003e \u003cp\u003eStata command \u003cem\u003erori\u003c/em\u003e has a single bias parameter, the relative odds ratio, which is the ratio of the odds ratio among the selected participants divided by the odds ratio in the source\u003c/p\u003e \u003cp\u003epopulation. For a binary exposure, \u003cem\u003erori\u003c/em\u003e computes the relative odds ratio from a \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:2\\times\\:2\\)\u003c/span\u003e\u003c/span\u003e tabulation of the exposure and outcome among the selected participants and the same \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:2\\times\\:2\\)\u003c/span\u003e\u003c/span\u003e tabulation in the source population. Lastly, R packages \u003cem\u003eEValue\u003c/em\u003e and \u003cem\u003eSelectionBias\u003c/em\u003e both implement a QBA using the Smith and VanderWeele (SV) bounds for selection bias and are applicable when inference is in the total population and in a selected population [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. However, there are several differences between these two R packages: (1) \u003cem\u003eEValue\u003c/em\u003e reports the minimum value of the bias parameters (when all equal) to change a study conclusion (e.g., give a null point estimate or an estimate opposite in sign to the na\u0026iuml;ve point estimate) whereas \u003cem\u003eSelectionBias\u003c/em\u003e reports the lower and upper bound for the bias-adjusted estimate given user-specified values of the bias parameters. (2) \u003cem\u003eSelectionBias\u003c/em\u003e reports sharp lower and upper SV bounds (i.e., tightest possible bounds given the necessary assumptions of the QBA). (3) \u003cem\u003eSelectionBias\u003c/em\u003e also provides alternatives to the SV bounds; namely the assumption-free bound, the generalised assumption-free bound and the counterfactual assumption-free bound [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section3\"\u003e \u003ch2\u003e4.3.3 Specialist QBA features\u003c/h2\u003e \u003cp\u003eR package \u003cem\u003enrba\u003c/em\u003e includes features to implement a multidimensional QBA such as enabling the user to specify multiple values for the bias parameter and outputting a table of the multiple estimates of the bias-adjusted mean or proportion. However, \u003cem\u003enrba\u003c/em\u003e does not include any features to identify bias parameter values that correspond to a pre-specified tipping point. In contrast, R package \u003cem\u003eEValue\u003c/em\u003e implements a tipping point analysis but not a multidimensional QBA (i.e., does not output any bias-adjusted estimates). Instead, \u003cem\u003eEValue\u003c/em\u003e reports the minimum value(s) of the bias parameter(s) required to change a study conclusion.\u003c/p\u003e \u003cp\u003eSoftware programs \u003cem\u003eapisensr\u003c/em\u003e, \u003cem\u003eepisensr\u003c/em\u003e, and \u003cem\u003eepisens\u003c/em\u003e include features to implement a probabilistic QBA using the Monte Carlo Sensitivity Analysis approach [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], outputting a histogram of the frequency distribution of bias-adjusted estimates (and also draws from the bias parameters\u0026rsquo; prior distributions for \u003cem\u003eepisens\u003c/em\u003e and \u003cem\u003eepisensr\u003c/em\u003e).\u003c/p\u003e \u003cp\u003eUnfortunately, R packages \u003cem\u003emultibias\u003c/em\u003e and \u003cem\u003eSelectionBias\u003c/em\u003e and Stata commands \u003cem\u003ebiasepi\u003c/em\u003e and \u003cem\u003erori\u003c/em\u003e do not include any features to conduct a multidimensional or probabilistic QBA or a tipping point analysis. Consequently, the user is required to write their own code to conduct a full QBA using these programs. For example, to implement a multidimensional QBA the user must write code to repeatedly call the software program for different values of the bias parameter(s), store the multiple bias-adjusted estimates and present the results in a graphical plot or table.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec18\" class=\"Section3\"\u003e \u003ch2\u003e4.3.4 Other specialist features\u003c/h2\u003e \u003cp\u003eOther specialist features include: (1) web tool \u003cem\u003eapisensr\u003c/em\u003e and R package \u003cem\u003eepisensr\u003c/em\u003e generate directed acyclic graphs illustrating selection caused by M-bias (a form of collider bias [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]). (2) R packages \u003cem\u003emultibias\u003c/em\u003e and \u003cem\u003eepisensr\u003c/em\u003e output a forest plot of the results of the na\u0026iuml;ve estimate and QBAs, and (3) \u003cem\u003emultibias\u003c/em\u003e includes an option to specify the values of the bias parameters using an external validation dataset.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec19\" class=\"Section3\"\u003e \u003ch2\u003e4.3.5 Multiple QBA\u003c/h2\u003e \u003cp\u003eSix software programs also implement a QBA to other sources of bias and implement a multiple QBA (i.e., assess sensitivity to the combined impact of multiple sources of bias). Software programs \u003cem\u003eapisensr\u003c/em\u003e, \u003cem\u003eepisensr\u003c/em\u003e, \u003cem\u003eEValue\u003c/em\u003e, \u003cem\u003emultibias\u003c/em\u003e, \u003cem\u003ebiasepi\u003c/em\u003e, and \u003cem\u003eepisens\u003c/em\u003e can be used to conduct a QBA to unmeasured confounding, misclassification of the exposure, or misclassification of the outcome, or a multiple QBA to any combination of selection bias, unmeasured confounding, and misclassification. Note that \u003cem\u003eapisensr\u003c/em\u003e, \u003cem\u003eepisensr\u003c/em\u003e, \u003cem\u003eEValue\u003c/em\u003e, \u003cem\u003ebiasepi\u003c/em\u003e, and \u003cem\u003eepisens\u003c/em\u003e sequentially adjust for the multiple sources of bias in the reverse order in which they occurred in the data (e.g., adjust for outcome misclassification within the selected sample and then adjust for selection bias). In contrast, \u003cem\u003emultibias\u003c/em\u003e simultaneously adjusts for multiple sources of bias, avoiding the need to specify the order in which the biases occur.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"5. Discussion","content":"\u003cp\u003eWe have conducted an up-to-date review of software implementations of risk-of-bias tools and QBA to selection bias.\u003c/p\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003e5.1 Risk-of-bias tools review\u003c/h2\u003e \u003cp\u003eIn our review, we identified a subset of risk-of-bias tools through a literature search, supplemented with a cross-reference search, restricting our focus to tools that contain at least one dedicated signalling question pertaining to selection bias We examined the selection bias components of these 24 tools\u0026mdash;which varied from single questions to multi-item domains\u0026mdash;and detailed their target study types, assessment methods, and levels of guidance. To the best of our knowledge, no similar report of the selection bias components of qualitative risk-of-bias tools has previously been published.\u003c/p\u003e \u003cp\u003eWhile qualitative risk-of-bias tools can be a valuable resource for assessing the risk of selection bias, or, more generally, the overall methodological quality of a study, this is subject to the correct application of the tool. A methodological systematic review of 124 applied systematic reviews [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e] found that the ROBINS-I tool was frequently applied incorrectly, and noted that ROBINS-I is a complex tool requiring extensive methodological understanding, which is too frequently applied by authors lacking the necessary expertise. Various studies have been published on inter-rater agreement for tools such as ROBINS-I and NOS [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e], RoBANS [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e], ROB-2 [\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e], and the original ROB tool (the predecessor of ROB-2) [\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e], with studies reporting limited [\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e], moderate [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e, \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e] and good agreement [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e] between reviewers. For ROB-2, the authors found substantial improvement in the reliability of the tool after developing topic-specific implementation instructions. Overall, when reported, agreement on selection-bias specific signalling questions and domains, while varying, was generally judged to be \u0026lsquo;fair\u0026rsquo;. It should be noted that all assessments were performed by reviewers with prior methodological and/or systematic review experience and, in some cases, tool-specific training [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e]. A recent publication by Sotiropoulos \u003cem\u003eet al.\u003c/em\u003e [\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e] provides methodological guidance for assessing the different bias domains of the RoB-2 tool when there is access to individual-level patient data, instead of summary-level data as is common for systematic reviews.\u003c/p\u003e \u003cp\u003eA challenge to the uptake of qualitative bias tools for assessing the risk of (selection) bias is the volume of tools and their variation in the granularity and degree of guidance for signalling questions, which increases reliance on investigator expertise. To help researchers synthesize this information, we have provided a comprehensive overview of selection bias signalling questions per study design (Supplementary Table\u0026nbsp;2).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003e5.2 QBA software review\u003c/h2\u003e \u003cp\u003eOur review identified nine software programs of QBA to selection bias, applicable to summary-level and individual-level data. Our identification of only a small number of tools is in keeping with a recent review (published in 2024) of QBA methods applicable to summary-level data [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] which identified only a single software program (R package \u003cem\u003eEValue\u003c/em\u003e) and previous systematic reviews of applied analyses which found that QBA studies were far less likely to conduct a QBA to selection bias compared to unmeasured confounding and misclassification [\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. This has partly been attributable to the lack of available methodology for QBA to selection bias [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eMost of the software programs we identified are only applicable for a binary outcome and exposure. We did not find any programs applicable for estimation of an exposure effect when the outcome is continuous or categorical. Also, none of the programs applicable for estimation of a bias-adjusted exposure effect included any specialist features to aid the user in carrying out a multidimensional QBA (i.e., such as estimation of the bias-adjusted exposure effect for all value-combinations of the bias parameters and graphical presentation of the multiple estimates to aid interpretation).\u003c/p\u003e \u003cp\u003eSome published QBA methods were excluded because the provided software code did not meet our software criteria (e.g., no user interface, not accompanied by documentation, or required code adaption before application to another example). For example, R function \u003cem\u003enisb()\u003c/em\u003e for calculating indices of selection bias for means and/or proportions estimated from non-probability samples [\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]; (undocumented) R package \u003cem\u003escbounds\u003c/em\u003e for calculation of bounds for a population mean estimated from a non-probability sample [\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e]; Flanders and Ye provide R code for their bound of bias arising from M-bias [\u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e]; and Gelman and Carpenter provide R and Stan code for their Bayesian analyses of testing for a rare disease with unknown specificity and sensitivity and using data from an unrepresentative sample of the target population [\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec23\" class=\"Section2\"\u003e \u003ch2\u003e5.3 Strengths and limitations\u003c/h2\u003e \u003cp\u003eOur study covers more than 20-year period, from 1 January 2004 up to the end of August 2025, capturing all available risk-of-bias tools and QBA software programs for individual-level data and summary-level data. Also, in order to capture tools and applications across different research fields, our search strategy used a diverse range of terminology for sample selection bias. One limitation of our search strategy is its restriction to the published literature and searchable databases for Stata and R implementations. Thus, our review may have excluded risk-of-bias tools and software programs that are only available from a search of the internet or from databases of other software environments. Note that our review of the unpublished literature focused on Stata and R because they are widely used among epidemiologists and medical statistician, with freely available user-written applications [\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e]. On balance, we believe that our search strategy mimics the way in which an applied researcher would source their software and so our results reflect the tools and software accessible to most researchers.\u003c/p\u003e \u003cp\u003eA second limitation is that we do not provide information on the quality of the risk-of-bias tools or software programs. However, our review\u0026rsquo;s inclusion criteria did exclude all risk-of-bias tools that were still under construction or not fully validated and all undocumented or partially developed QBA software programs.\u003c/p\u003e \u003cp\u003eThirdly, our review did not address publication bias. Although publication bias involves a form of selection, it occurs at the synthesis level (selection of studies for publication) rather than at the study level. Our review was strictly limited to the assessment of selection bias at the study level, focusing on the internal validity of individual studies.\u003c/p\u003e \u003c/div\u003e"},{"header":"6. Conclusions","content":"\u003cp\u003eWhile there are many qualitative risk-of-bias tools that assess selection bias for a variety of study designs and areas of research, there is considerable variation in granularity and guidance, and their successful application requires sufficient methodological expertise. To facilitate wider uptake of QBA, greater provision of QBA software to selection bias is needed, along with detailed QBA guidelines.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eQBA\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003equantitative bias analysis\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCRAN\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eComprehensive R archive Network\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSSC\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eStatistical Software Components\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003ch3\u003eEthics approval and consent to participate#\u003c/h3\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003ch3\u003eConsent for publication\u003c/h3\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003ch3\u003eAvailability of data and materials\u003c/h3\u003e\n\u003cp\u003eData sharing is not applicable to this article as no datasets were generated or analysed during the current study.\u003c/p\u003e\n\u003ch3\u003eCompeting interests\u003c/h3\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003ch3\u003eFunding\u003c/h3\u003e\n\u003cp\u003eAH received research support from AstraZeneca. KT, RAH and EC work in the MRC Integrative Epidemiology Unit which is supported by\u0026nbsp;the\u0026nbsp;UK Medical Research Council and\u0026nbsp;the\u0026nbsp;University of Bristol (MC_UU_00032/2). RAH was supported by a Sir Henry Dale Fellowship that was jointly funded by the Wellcome Trust and the Royal Society (grant 215408/Z/19/Z).\u003c/p\u003e\n\u003ch3\u003eAuthors\u0026rsquo; contributions\u003c/h3\u003e\n\u003cp\u003eRAH proposed and designed the project. AH, EC, and RAH conducted the scoping review. \u0026nbsp;All authors drafted the manuscript and reviewed and approved the final version of the manuscript.\u003c/p\u003e\n\u003ch3\u003eAcknowledgements\u003c/h3\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eSmith LH. Selection Mechanisms and Their Consequences: Understanding and Addressing Selection Bias. Curr Epidemiol Rep. 2020;7(4):179\u0026ndash;89.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMadley-Dowd P, et al. Using directed acyclic graphs to determine whether multiple imputation or subsample-multiple imputation estimates of an exposure-outcome association are unbiased. Am J Epidemiol. 2025;195(2):505\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEnders CK. Missing data: An update on the state of the art. Psychol Methods. 2025;30(2):322\u0026ndash;39.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee KJ, et al. Framework for the treatment and reporting of missing data in observational studies: The Treatment And Reporting of Missing data in Observational Studies framework. J Clin Epidemiol. 2021;134:79\u0026ndash;88.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLittle RJ. Missing Data Analysis. Ann Rev Clin Psychol. 2024;20(20):149\u0026ndash;73.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHughes RA, et al. Accounting for missing data in statistical analyses: multiple imputation is not always the answer. Int J Epidemiol. 2019;48(4):1294\u0026ndash;304.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGrubic N, et al. Breaking Down Bias: A Methodological Primer on Identifying, Evaluating, and Mitigating Bias in Cardiovascular Research. Can J Cardiol. 2025;41(5):996\u0026ndash;1009.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMa L-L, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Military Med Res. 2020;7(1):1\u0026ndash;11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSterne JAC, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFox MP, MacLehose RF, Lash TL. Applying Quantitative Bias Analysis to Epidemiologic Data. New York: Springer; 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHunnicutt JN, et al. Probabilistic bias analysis in pharmacoepidemiology and comparative effectiveness research: a systematic review. Pharmacoepidemiol Drug Saf. 2016;25(12):1343\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePetersen JM, et al. A systematic review of quantitative bias analysis applied to epidemiological research. Int J Epidemiol. 2021;50(5):1708\u0026ndash;30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAkbaş KE, Hark BD. Evaluation of quantitative bias analysis in epidemiological research: A systematic review from 2010 to mid-2023. J Eval Clin Pract. 2024;30(7):1413\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShi X, et al. Quantitative bias analysis methods for summary-level epidemiologic data in the peer-reviewed literature: a systematic review. J Clin Epidemiol. 2024;175:111507.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eInfante-Rivard C, Cusson A. Reflection on modern methods: selection bias\u0026mdash;a review of recent developments. Int J Epidemiol. 2018;47(5):1714\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOrsini N, Bellocco R, Greenland S. EPISENS: Stata module for basic sensitivity analysis of epidemiological results; in Statistical Software Components. 2008, Boston College Department of Economics,. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ideas.repec.org/c/boc/bocode/s456792.html\u003c/span\u003e\u003cspan address=\"https://ideas.repec.org/c/boc/bocode/s456792.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGriffith GJ, et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nat Commun. 2020;11(1):5749.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGriffith GJ. \u003cem\u003eAscRtain\u003c/em\u003e. [Accessed 21 January 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://apps.mrcieu.ac.uk/ascrtain/\u003c/span\u003e\u003cspan address=\"http://apps.mrcieu.ac.uk/ascrtain/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGr\u0026ouml;nwold R, Palmer TM, Tilling K. \u003cem\u003eBias App\u003c/em\u003e. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://remlapmot.shinyapps.io/bias-app/#section-selection-bias\u003c/span\u003e\u003cspan address=\"https://remlapmot.shinyapps.io/bias-app/#section-selection-bias\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHern\u0026aacute;n MA, Hern\u0026aacute;ndez-D\u0026iacute;az S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15(5):615\u0026ndash;25.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLu H, et al. Toward a Clearer Definition of Selection Bias When Estimating Causal Effects. Epidemiology. 2022;33(5):699\u0026ndash;706.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLu H, et al. The evolution of selection bias in the recent epidemiologic literature\u0026mdash;a selective overview. Am J Epidemiol. 2024;194(3):580\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMathur MB, Shpitser I. Simple graphical rules for assessing selection bias in general-population and selected-sample treatment effects. Am J Epidemiol. 2024;194(1):267\u0026ndash;77.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHern\u0026aacute;n MA. Invited Commentary: Selection Bias Without Colliders. Am J Epidemiol. 2017;185(11):1048\u0026ndash;50.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBanack HR, Smith SN, Bodnar LM. Application of a Web-based Tool for Quantitative Bias Analysis: The Example of Misclassification Due to Self-reported Body Mass Index. Epidemiology. 2024;35(3):359\u0026ndash;67.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcCandless LC, Gustafson P. A comparison of Bayesian and Monte Carlo sensitivity analysis for unmeasured confounding. Stat Med. 2017;36(18):2887\u0026ndash;901.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKawabata E, et al. Accounting for bias due to outcome data missing not at random: comparison and illustration of two approaches to probabilistic bias analysis: a simulation study. BMC Med Res Methodol. 2024;24(1):278.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZuckarelli J, packagefinder. Comfortable Search for R Packages on CRAN, Either Directly from the R Console or with an R Studio Add-in; in. 2023. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cran.r-project.org/web/packages/packagefinder/index.html\u003c/span\u003e\u003cspan address=\"https://cran.r-project.org/web/packages/packagefinder/index.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCs\u0026aacute;rdi G. and M. Salmon. pkgsearch: Search and Query CRAN R Packages; in. 2025. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cran.r-project.org/web/packages/pkgsearch/index.html\u003c/span\u003e\u003cspan address=\"https://cran.r-project.org/web/packages/pkgsearch/index.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIDEAS/RePEc. \u003cem\u003eEconomic and Finance Research\u003c/em\u003e [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ideas.repec.org/\u003c/span\u003e\u003cspan address=\"https://ideas.repec.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStataCorp. Stata Statistical Software; in. StataCorp LLC. [Accessed; Available from; 2025.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFaillie J-L, et al. A new risk of bias checklist applicable to randomized trials, observational studies, and systematic reviews was developed and validated to be used for systematic reviews focusing on drug adverse events. J Clin Epidemiol. 2017;86:168\u0026ndash;75.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLuijken K, et al. How to assess applicability and methodological quality of comparative studies of operative interventions in orthopedic trauma surgery. Eur J Trauma Emerg Surg. 2022;48(6):4943\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWells G et al. \u003cem\u003eThe Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses\u003c/em\u003e. 2011 [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.ohri.ca/programs/clinical_epidemiology/oxford.asp\u003c/span\u003e\u003cspan address=\"http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNational Toxicology Program. OHAT Risk of Bias Rating Tool for Human and Animal Studies. 2015 [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ntp.niehs.nih.gov/sites/default/files/ntp/ohat/pubs/riskofbiastool_508.pdf\u003c/span\u003e\u003cspan address=\"https://ntp.niehs.nih.gov/sites/default/files/ntp/ohat/pubs/riskofbiastool_508.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKahan BC, Rehal S, Cro S. Risk of selection bias in randomised trials. Trials. 2015;16:405.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSterne JA, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHiggins JPT, et al. A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E). Environ Int. 2024;186:108602.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHooijmans CR, et al. SYRCLE's risk of bias tool for animal studies. BMC Med Res Methodol. 2014;14:43.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZetterstrom S, Waernbaum I. SelectionBias: Calculates Bounds for the Selection Bias for Binary Treatment and Outcome Variables; in. 2024. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cran.r-project.org/web/packages/SelectionBias/index.html\u003c/span\u003e\u003cspan address=\"https://cran.r-project.org/web/packages/SelectionBias/index.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSmith LH, VanderWeele TJ. Bounding Bias Due to Selection. Epidemiology. 2019;30(4):509\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGreenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25(6):1107\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZetterstrom S. Bounds for selection bias using outcome probabilities. Epidemiol Methods, 2024. 13s2).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIgelstr\u0026ouml;m E, et al. Cochrane's risk of bias tool for non-randomized studies (ROBINS-I) is frequently misapplied: A methodological systematic review. J Clin Epidemiol. 2021;140:22\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang Y, et al. The ROBINS-I and the NOS had similar reliability but differed in applicability: A random sampling observational studies of systematic reviews/meta-analysis. J Evid Based Med. 2021;14(2):112\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeo HJ, et al. RoBANS 2: A Revised Risk of Bias Assessment Tool for Nonrandomized Studies of Interventions. Korean J Fam Med. 2023;44(5):249\u0026ndash;60.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMinozzi S, et al. Reliability of the revised Cochrane risk-of-bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022;141:99\u0026ndash;105.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDaly A, et al. Tolerating bad health research (part 2): still as many bad trials, but more good ones too. Trials. 2025;26(1):110.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSotiropoulos JX et al. Individual participant data informed risk of bias assessments for randomized controlled trials in systematic reviews and meta-analyses. J Clin Epidemiol, 2025. 185.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLittle RJA, et al. Measures of the Degree of Departure from Ignorable Sample Selection. J Surv Stat Methodol. 2019;8(5):932\u0026ndash;64.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndridge RR, et al. Indices of Non-Ignorable Selection Bias for Proportions Estimated from Non-Probability Samples. J Royal Stat Soc Ser C: Appl Stat. 2019;68(5):1465\u0026ndash;83.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiratrix LW, Wager S, Zubizarreta JR. Shape-constrained partial identification of a population mean under unknown probabilities of sample selection. Biometrika. 2017;105(1):103\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFlanders WD, Ye D. Limits for the Magnitude of M-bias and Certain Other Types of Structural Selection Bias. Epidemiology. 2019;30(4):501\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGelman A, Carpenter B. Bayesian Analysis of Tests with Unknown Specificity and Sensitivity. J R Stat Soc Ser C Appl Stat. 2020;69(5):1269\u0026ndash;83.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSriram R. \u003cem\u003eTop 10 Statistical Tools Used in Medical Research\u003c/em\u003e. 2020 [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.kolabtree.com/blog/top-10-statistical-tools-used-in-medical-research/\u003c/span\u003e\u003cspan address=\"https://www.kolabtree.com/blog/top-10-statistical-tools-used-in-medical-research/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evid Synth. 2023;21(3):494\u0026ndash;506.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies. JBI Evid Synth. 2024;22(3):378\u0026ndash;88.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSlim K, et al. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg. 2003;73(9):712\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarker TH, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for cohort studies. JBI Evid Synth. 2025;23(3):441\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoanna Briggs Institute. JBI Critical Appraisal Checklist for Case Control Studies. 2017 [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://jbi.global/sites/default/files/2019-05/JBI_Critical_Appraisal-Checklist_for_Case_Control_Studies2017_0.pdf\u003c/span\u003e\u003cspan address=\"https://jbi.global/sites/default/files/2019-05/JBI_Critical_Appraisal-Checklist_for_Case_Control_Studies2017_0.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoanna Briggs Institute. JBI Critical Appraisal Checklist for Analytical Cross Sectional Studies. 2020 [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://jbi.global/sites/default/files/2020-08/Checklist_for_Analytical_Cross_Sectional_Studies.pdf\u003c/span\u003e\u003cspan address=\"https://jbi.global/sites/default/files/2020-08/Checklist_for_Analytical_Cross_Sectional_Studies.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMunn Z, et al. Chap. 5: \u003cem\u003eSystematic reviews of prevalence and incidence\u003c/em\u003e. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. Editors: JBI; 2020.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTonia T et al. Tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH). BMJ Ment Health, 2023. 26(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePega F, et al. RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environ Int. 2020;135:105039.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMunn Z, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evid Synth. 2020;18(10):2127\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRavensbergen L, El-Geneidy A. Toward Evidence-Based Urban Planning. J Am Plann Association. 2023;89(3):389\u0026ndash;98.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCampbell JM, et al. Diagnostic test accuracy: methods for systematic review and meta-analysis. Int J Evid Based Healthc. 2015;13(3):154\u0026ndash;62.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWhiting PF, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529\u0026ndash;36.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSchneider B, et al. nrba: Methods for Conducting Nonresponse Bias Analysis; in. Westat, Inc. [Accessed; Available from; 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLohr SL, Riddles MK, Morganstein D. Tests for evaluating nonresponse bias in surveys. Surv Methodol. 2016;42(2):195\u0026ndash;218.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaine D. The episensr package: Basic sensitivity analysis of epidemiological results; in. 2025. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://dhaine.codeberg.page/episensr/\u003c/span\u003e\u003cspan address=\"https://dhaine.codeberg.page/episensr/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMathur MB et al. EValue: Sensitivity Analyses for Unmeasured Confounding and Other Biases in Observational Studies and Meta-Analyses; in. 2021. [Accessed Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cran.r-project.org/web/packages/EValue/index.html\u003c/span\u003e\u003cspan address=\"https://cran.r-project.org/web/packages/EValue/index.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrendel P, Torres A, Arah OA. Simultaneous adjustment of uncontrolled confounding, selection bias and misclassification in multiple-bias modelling. Int J Epidemiol. 2023;52(4):1220\u0026ndash;30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrendel Pmultibias. Multiple Bias Analysis in Causal Inference; in. 2025. [Accessed 25 March 2026]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://cran.r-project.org/web/packages/multibias/index.html\u003c/span\u003e\u003cspan address=\"https://cran.r-project.org/web/packages/multibias/index.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZetterstrom S, Waernbaum I. Selection bias and multiple inclusion criteria in observational studies. Epidemiol Methods, 2022. 11(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZetterstrom S, Sj\u0026ouml;lander A, Waernbaum I. Investigations of sharp bounds for causal effects under selection bias. Stat Methods Med Res. 2025;34(12):2270\u0026ndash;83.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNohr EA, Liew Z. How to investigate and adjust for selection bias in cohort studies. Acta Obstet Gynecol Scand. 2018;97(4):407\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Causal inference, Quantitative bias analysis, Review, Risk-of-bias tool, Selection bias, Sensitivity analysis, Software","lastPublishedDoi":"10.21203/rs.3.rs-9283156/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9283156/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eFailure to appropriately account for selection bias may lead to erroneous conclusions. Risk-of-bias tools are recommended (in systematic reviews of medical studies) to identify potential selection bias, and quantitative bias analyses (QBA) are recommended to quantitatively assess the sensitivity of a (meta-analysis or individual) study\u0026rsquo;s conclusions to plausible assumptions about the selection mechanism. While risk-of-bias tools are widely available, their effective use is complicated by the large number of available instruments with varying granularity and a high reliance on investigator expertise. QBAs are not routinely implemented, due to a lack of knowledge about accessible methods and software.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eWe conducted a scoping review of software implementations of risk-of-bias tools and QBA methods, for sample selection bias, published from January 2004 through August 2025. Inclusion criteria were validated risk-of-bias tools and software not requiring adaptation (i.e., code changes) before application, still available in 2025, and accompanied by documentation. Key properties of each risk-of-bias and software tool were identified.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eWe identified 24 risk-of-bias tools. All consisted of one to five signalling questions relating to selection bias and covered a wide range of study designs (e.g., randomised controlled trials, non-randomised and diagnostic accuracy studies). Question granularity varied and fewer than half of tools included a dedicated selection bias domain, with only three providing structured guidance for synthesizing the evidence from these questions into a final risk judgment on selection bias. For QBA, we identified nine software programs (one web-based, five R packages and three Stata commands). Six programs were only applicable when estimating the effect of a binary exposure on a binary outcome. Three programs implemented a probabilistic QBA, one a multidimensional QBA, and one a tipping point QBA. The remaining programs required the user to supply their own code to fully implement a QBA.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eWhile many risk-of-bias tools are available, the degree of guidance provided varies drastically, and the overall assessment of selection bias depends heavily on the investigator's interpretation. Greater provision of QBA software to selection bias, along with detailed QBA guidelines, would facilitate the wider uptake of QBA among future studies.\u003c/p\u003e","manuscriptTitle":"Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-02 08:14:52","doi":"10.21203/rs.3.rs-9283156/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-05-15T14:46:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"39067948853943753172073837670569345959","date":"2026-05-03T22:08:34+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-21T13:07:10+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-07T10:06:39+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-06T02:47:33+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-06T02:47:02+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Research Methodology","date":"2026-03-31T17:26:44+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-research-methodology","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bmrm","sideBox":"Learn more about [BMC Medical Research Methodology](http://bmcmedresmethodol.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/bmrm/default.aspx","title":"BMC Medical Research Methodology","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"490db42c-ec35-464b-9151-93047878a43d","owner":[],"postedDate":"April 2nd, 2026","published":true,"recentEditorialEvents":[{"type":"editorInvitedReview","content":"","date":"2026-05-15T14:46:25+00:00","index":58,"fulltext":""},{"type":"reviewerAgreed","content":"39067948853943753172073837670569345959","date":"2026-05-03T22:08:34+00:00","index":42,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-21T13:23:04+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-02 08:14:52","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9283156","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9283156","identity":"rs-9283156","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}