The limits of inductive reasoning for clinical evidence appraisal – a simulation study (Protocol)

Abstract Objectives: To establish the possible likelihood of a body of evidence, inductively judged to be of ’low bias risk’/‘high-quality’ according to a limited set of appraisal criteria, of actually being error free. Methods: We will generate 45 simulation trials and randomly assign to each 0-5 errors out of a total of 65 error domains. The trials will then be appraised for errors with a simulated appraisal tool comprising of five pre-specified error domains. Trial appraisal will yield either true positive, true negative, false negative and false positive results. From these values the negative likelihood ratio (-LR) with 95% Confidence interval (CI) will be computed. -LR computation will be repeated 25 times, each with newly generated random values for all 45 trials. The results of all 25 runs will be statistically pooled. The pooled –LR result with 95% CI will be interpreted as how likely a ’low bias risk’/‘high-quality’ rated body of evidence is to be actually error-free. Ethics and dissemination: Ethical approval is not required for simulation-based studies. The results will be disseminated as a prior preprint version and subsequent peer-reviewed publication.