How comprehensive is the Cochrane Central Register of Controlled Trials for identifying clinical trial registration?: the protocol of a diagnostic study

37 Objectives. To evaluate whether or not systematic reviewers can use Cochrane 38 Central Register of Controlled Trials (CENTRAL) to identify ongoing and 39 unpublished studies instead of searching International Clinical Trials Registry 40 Platform (ICTRP) and ClinicalTrials.gov (CT.gov). 41 Methods. This will be a diagnostic accuracy test study. We will collect a 42 consecutive sample of ongoing or unpublished studies on Cochrane Database of 43 Systematic Reviews (CDSRs) during the last six months. We will use all of the 44 records as our reference standard and evaluated whether they are part of the 45 CENTRAL search presented in the CDSRs. 46 The index test is the CENTRAL search using the search terms in the CDSRs, 47 and the reference standard is the list of ongoing or unpublished studies 48 registered on the ICTRP or CT.gov in the CDSRs. 49 We will assess the sensitivity and number needed to read. 50 Ethics & Dissemination. This study does not require ethics approval. We 51 registered this study protocol. We will publish the findings in a peer-reviewed 52 journal and may present them at conferences. 53 Discussion. This study may lessen the burden of systematic reviewers if this 54 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 4 study clarifies CENTRAL search can be used for screening in records about 55 ongoing or unpublished studies. 56 Registration: UMIN-CTR: UMIN000038981 57 58 Author name abbreviations: Masahiro Banno (MB), Yasushi Tsujimoto (YT), Yuki 59 Kataoka (YK) 60 61

62 Cochrane Central Register of Controlled Trials, clinical trial registration, 63 sensitivity, diagnostic test accuracy, research on research 64 65

66 Searching for ongoing or unpublished studies when conducting systematic 67 reviews is important to address publication bias [1]. Cochrane handbook and 68 previous studies require systematic reviewers to search both International 69 Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov (CT.gov) to 70 identify ongoing or unpublished studies [1-4]. 71 However, searching ICTRP directly is problematic because number of records 72 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 5 downloadable per one time is limited and because ICTRP do not return search 73

with complex search strategies [3, 5]. 74 We can recently search records about ongoing or unpublished randomized 75 controlled trials and quasi-randomized controlled trials on the Cochrane Central 76 Register of Controlled Trials(CENTRAL) [6]. All the records on ICTRP and 77 CT.gov before April 2019 have been included in CENTRAL. Newly identified 78 CT.gov and ICTRP records are added to CENTRAL on a monthly basis [7]. 79 Whether systematic reviewers can search CENTRAL instead of ICTRP and 80 CT.gov to screening records about ongoing or unpublished studies is unknown. 81 If CENTRAL search can replace the search of ICTRP or CT.gov, we expect to 82 identify all ongoing and unpublished studies on ICTRP or CT.gov included in 83 systematic reviews by CENTRAL search alone. The objective of this study is to 84 evaluate the sensitivity and number needed to read (NNR) of CENTRAL search 85 to identify ongoing or unpublished studies for systematic review production. 86 87

88 This publication is the full study protocol. This protocol has been registered in 89 the University Hospital Medical Information Network Clinical Trials Registry 90 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 6 (UMIN-CTR). The registration number is UMIN000038981. 91 92 Study design 93 This study is a diagnostic test accuracy study. We will collect a consecutive 94 sample of ongoing or unpublished records on Cochrane Database of Systematic 95 Reviews (CDSRs). We will use the records as our reference standard and 96 evaluate whether they are part of the CENTRAL search. Figure 1 and Figure 2 97 summarize the concept and design of this study. We define CDSR records as 98 ongoing or unpublished studies including each CDSR. We define CENTRAL 99 records as records derived from CENTRAL search. We define CDSR/CENTRAL 100 records as ongoing or unpublished studies including each CDSR and records 101 derived from CENTRAL search. 102 103 Eligible criteria 104 We will include all records about ongoing or unpublished studies registered on 105 the ICTRP or CT.gov in the latest interventional CDSRs. We will include CDSRs 106 during the last six months. We will exclude CDSRs that used Cochrane Review 107 Group Specialized Registers or CDSRs that did not use CENTRAL, ICTRP, or 108 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 7 CT.gov. 109 To evaluate whether the records of ongoing and unpublished studies in CDSRs 110 can be identified by CENTRAL search alone, we will also include all records of 111 the ICTRP or CT.gov that are identified by the search strategy of CENTRAL 112 presented in CDSRs. 113 For eligible records, we will extract the following characteristics; trial identifying 114 number, principal investigator, and year of registration. 115 116 Index test and reference standard 117 The index test will be CENTRAL search. We will manually search CENTRAL with 118 the search strategy presented in each CDSR, in limited publication date, which 119 corresponds to the search date in each CDSR. 120 The reference standard is the list of ongoing or unpublished studies registered 121 on the ICTRP or CT.gov in the CDSRs, and two other authors will confirm the 122 records. We will tackle disagreements by discussion between the authors. We 123 will choose CDSRs as the data source because CDSRs are performed by 124 rigorous methods following Cochrane Handbook and expected to be available 125 sufficient search strategy to perform a comprehensive CENTRAL search [1]. 126 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 8 127 Sample size 128 We do not calculate suitable sample size because this study is an explanatory 129 study. We will focus on CDSRs during the last six months to obtain a certain 130 number of records. 131 132 Data analysis 133 Our primary outcome is a sensitivity, with its 95 % confidence interval, of 134 CENTRAL search to identify all ongoing and unpublished records. We will 135 calculate a sensitivity, dividing the number of CDSR/CENTRAL records by the 136 number of CDSR records [1]. We will calculate 95 % CI in Wilson score interval 137 with continuity correction by a calculator [8]. We will use Wilson score interval 138 with continuity correction because this method is the accurate method than 139 conventional method [9]. 140 The secondary outcome is a NNR. NNR is a measure of how many records in a 141 database have to be read to identify one of adequate clinical quality and 142 relevance [10]. We will calculate a NNR, dividing the number of CENTRAL 143 records by the number of CDSR/CENTRAL records [1]. We will also report the 144 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 9 numbers of records, which are included CDSRs but are not identified by 145 CENTRAL search. 146 If the sensitivity was not 100%, we will conduct pre-specified subgroup analysis 147 about the primary outcome as follows: 1) type of intervention on CDSRs 148 (pharmacological intervention or non-pharmacological intervention), 2) version 149 of CDSRs (the first version or updated version). 150 All statistical analyses will be executed in Stata V.15.1 (StataCorp LLC, College 151 Station, Texas, United States of America) [11]. 152 153 Ethics 154 Ethics approval will not be essential because this study conduct research on 155 research. 156 157

158 This is the first study to investigate the appropriateness whether CENTRAL 159 search instead of ICTRP or CT.gov is sufficient to identify ongoing or 160 unpublished clinical trial registration. Therefore, this study will potentially be able 161 to lessen the burden of systematic reviewers if searching ICTRP or CT.gov for 162 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 10 identifying ongoing or unpublished studies is not mandatory based on the results 163 of this study. 164 This study had some expected limitations. First, not all records which undergo 165 CENTRAL search may be assessed by hand search on CDSRs, the reference 166 standard. The potential reason is that the time lags about records from 167 registration on ICTRP or CT.gov to adding to CENTRAL, which may prevent 168 searching CENTRAL from detecting comprehensive data about ongoing or 169 unpublished studies. We may underestimate sensitivity if this limitation arises 170 (for example, specify number of CDSR records is A, number of CDSR/CENTRAL 171 records is B, number of records added after time lag is X. A is larger than B. 172 Superficial sensitivity=B/A, true sensitivity=(B+X)/(A+X). Superficial sensitivity is 173 smaller than true sensitivity because A>B). 174 In conclusion, this study will perform an investigation about records about 175 ongoing or unpublished clinical trials. The expected results will clarify whether 176 systematic reviewers can search only CENTRAL to identify ongoing or 177 unpublished studies. 178 179

180 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 11 We would like to thank the Cochrane Library for managing the Cochrane Central 181 Register of Controlled Trials (CENTRAL). 182 183 CONTRIBUTORS 184 MB, YT and YK contributed to the conception and design of the research. MB is 185 fully responsible for writing the protocol. All authors gave final approval of the 186 protocol before submission. After the publication of the protocol, we plan for the 187 following contributions by each author: MB will screen CDSR records and extract 188 data. YT and YK will validate the records and data. MB will conduct the data 189 analysis. MB, YT and YK will write the manuscript. 190 191 FUNDING 192 This protocol was supported by no funder. 193 194 COMPETING INTERESTS 195 All authors have no competing interests. 196 197 Provenance and peer review 198 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 12 Not peer reviewed. 199 200 Patient consent for publication 201 Not required. 202 203 Data Availability Statement 204 We have no additional data. 205 206 Figure and table legends 207 Figure 1: Diagram about Cochrane Database of Systematic Reviews (CDSRs) 208 Abbreviations: CDSRs, Cochrane Database of Systematic Reviews; CENTRAL, 209 Cochrane Central Register of Controlled Trials. 210 Figure 2: Diagram about registration records 211 Abbreviations: CDSRs, Cochrane Database of Systematic Reviews; CENTRAL, 212 Cochrane Central Register of Controlled Trials; CT.gov, ClinicalTrials.gov; ICTRP, 213 International Clinical Trials Registry Platform. 214 215

216 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 13 217 [1] Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane 218 Handbook for Systematic Reviews of Interventions version 6.0 (updated July 219 2019). Available from www.training.cochrane.org/handbook .; 2019 [accessed 220 December 7 2019]. 221 [2] Banno M, Tsujimoto Y, Kataoka Y. Studies registered in non-ClinicalTrials.gov 222 accounted for an increasing proportion of protocol registrations in medical 223 research. J Clin Epidemiol. 2019;116:106-13. 224 [3] Glanville JM, Duffy S, McCool R, Varley D. Searching ClinicalTrials.gov and 225 the International Clinical Trials Registry Platform to inform systematic reviews: 226 what are the optimal search approaches? J Med Libr Assoc. 2014;102:177-83. 227 [4] Knelangen M, Hausner E, Metzendorf MI, Sturtz S, Waffenschmidt S. Trial 228 registry searches for randomized controlled trials of new drugs required 229 registry-specific adaptation to achieve adequate sensitivity. J Clin Epidemiol. 230 2018;94:69-75. 231 [5] World Health Organization. International Clinical Trials Registry 232 Platform (ICTRP): Downloading records from the ICTRP database. Available at 233 https://www.who.int/ictrp/search/download/en/; 2019 [accessed December 7 234 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint 14 2019]. 235 [6] Cochrane Library. How CENTRAL is created. Available at 236 https://www.cochranelibrary.com/central/central-creation; 2019 [accessed 237 December 7 2019]. 238 [7] Cochrane Library. The Cochrane Central Register of Controlled Trials. 239 Available at 240 https://www.cochranelibrary.com/central; 2019 [accessed December 9 2019]. 241 [8] Statskingdom. Statistics Kingdom: Proportion confidence interval calculator. 242 http://www.statskingdom.com/41_proportion_confidence_interval.html; 243 [accessed December 15 2019]. 244 [9] Wallis S. Binomial Confidence Intervals and Contingency Tests: Mathematical 245 Fundamentals and the Evaluation of Alternative Methods. Journal of Quantitative 246 Linguistics. 2013;20:178–208. 247 [10] Toth B, Gray JA, Brice A. The number needed to read-a new measure of 248 journal value. Health Info Libr J. 2005;22:81-2. 249 [11] StataCorp. Stata Statistical Software: Release 15. College Station, TX: 250 StataCorp LLC; 2017. 251 252 . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint Figure 1: Diagram about Cochrane Database of Systematic Reviews (CDSRs) Excluded total (n= ): [used Cochrane Review Group Specialized Registers (n= )] [did not use CENTRAL (n= )] Potentially eligible CDSRs (n= ) during the latest 6 months Included CDSRs (n= ) . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint Figure 2: Diagram about registration records Index test positive (n= ) Index test negative (n= ): [not identified by index test (n= )]

standard positive (n= )

standard negative (n= ) [included in CDSRs but not identified by index test (n= )] Index test: CENTRAL search using search strategies (n= ) for CENTRAL in included CDSRs All records in CENTRAL

standard: eligible records about ongoing or unpublished studies registered on the ICTRP or CT.gov (n= ) in included CDSRs (n= ) . CC-BY-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted January 2, 2020. ; https://doi.org/10.1101/2019.12.26.19014274doi: medRxiv preprint