Judgments of risk of bias associated with random sequence generation in trials included in Cochrane systematic reviews are frequently erroneous

Background Purpose: of this study was to analyze adequacy of judgments about risk of bias (RoB) for random sequence generation in Cochrane systematic reviews (CSRs) of randomized controlled trials (RCTs). Methods Information was extracted from RoB tables of CSRs using automated data scraping. We categorized all comments provided as supports for judgments for RoB related to randomization. We analyzed number and type of various supporting comments and assessed adequacy of RoB judgment for randomization in line with recommendations from the Cochrane Handbook. Results We analyzed 10527 RCTs that were included in 729 CSRs. For 5682 RCTs randomization was not described; for the others it was indicated randomization was done using computer/software/internet (N=2886), random number table (N=888), mechanic method (N=366), or it was incomplete/inappropriate (N=303). Overall, 1194/10125 trials (12%) had erroneous RoB judgment about randomization. The highest proportion of errors was found for trials with high RoB (28%), followed by those with low (19%), or unclear (3%). Therefore, one in eight judgments for the analyzed domain in CSRs was erroneous, and one in three if the judgment was “high risk”. Conclusion Cochrane systematic reviews cannot be necessarily trusted when it comes to judgments for risk of bias related to randomized sequence generation.