Transfer of Statistical Innovations of the 1990s-2000s in Oncology to the Biomedical Literature

Introduction Innovations in the fields of clinical studies require time to generate and disseminate new knowledge. We aimed to specifically explore lag times between the introduction and widespread use of innovative statistical methods in oncology using the competing risks and phase I model-based clinical trials settings as examples.

First, we defined a set of closed articles for each setting based on two princeps papers (Gray, Annals of Statistics 1998 for the competing risks setting and O’Quigley et al., Biometrics 1990 for the phase I setting). Secondly, we retrieved from the web of science all citations of the papers included in these sets. Each journal was classified as applied, semi-applied or methodological.

A total of 6,727 citations for the competing risks setting and 2,639 citations for the phase I setting were found. Time to reach 25 citations was 6.2 years for the Gray’s paper and 4.5 years for the Fine and Gray paper, while it ranged from 3.4 years up to at least 20.1 years and not reached for 6 papers from the competing risks setting. The vast majority (91%) of the citing papers for the competing risks setting originated from applied journals. In contrast, less than half (44%) of the citing papers for the phase I setting were published in applied journals.

Statistical innovations in the competing risks setting have been widely diffused in the medical literature unlike the model-based designs for phase I trials, which are still seldom used 30 years after publication. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 3

Translational medicine, aiming to expedite the discovery of new diagnostic tools and treatments by using a multi-disciplinary and collaborative approach, has grown interests in the last decade. Moreover, the commonly accepted delay of 17 years for research evidence to reach clinical practice recently appears to decrease [1] as exemplified by the mapping networks of publications and cross-references. [2] However, most translation gaps between knowledge and clinical application that have been investigated, notably in oncology, concerned translation of biological drivers into therapeutic benefits for patients. [3] For the advancement of clinical research and eventually of patient care, the translation of innovative statistical methods into practice is also of crucial importance, though the average time elapsed for biostatical innovation to reach medical use is less known. In 1994, Altman showed that, while the key survival paper of Kaplan and Meier achieved only six citations in medical journals in the first 10 years after publication, evidence of decreasing lag times between the introduction and widespread use of innovative statistical methods was expected. [4] More than 2 decades later, we wondered whether the reported above decreasing dichotomy between basic or preclinical and clinical research [2] could be also true in the transfer of innovative statistical methods, confirming the Altman’s hypothesis. [4] We first focused on competing risks methods, the key innovative survival methods of the last decades, [5] common in cancer. Indeed, competing risks exist whenever the probability of the main event (e.g., cancer) is prevented (e.g., by death) or altered (e.g., by transplantation) to occur. In this setting, standard survival methods that ignore the informative censoring of these competing events may result in biased effects of prognostic factors or treatments, [6] illustrating the need for bridging the gap of innovation into practice. Two innovative papers that first proposed a statistical test [7] and a regression model [8] for such data, were selected as the first innovative set. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 4 With the aim of focusing on statistical innovations of potential large expected therapeutic benefits for patients, we secondly considered the setting of phase I clinical trials, which is also of great interest in the oncology area though with a limited diffusion as compared to phase III. [9] We chose as the most influential statistical pioneering approach, the continual reassessment method (CRM) that is, a model-based approach for dose-finding studies where only empiric 3+3 designs (formerly Fibonacci) were long available. We thus included in the second innovative set the original paper of the CRM [10] together with its close proposed modifications (Table 2). The primary objective of this paper was to assess whether Altman’s prediction was fulfilled in the field of oncology using the competing risks and the phase I clinical trials as examples. Secondary objectives were to study the relative importance of the applied and statistical literature in papers citing statistical innovations and to analyze which medical areas the methods are mainly used.

Selection of articles Based on our knowledge, we a priori included the paper by Gray[7] and that by Fine and Gray [8] in the innovative set for the competing risks setting. For the phase I setting, beside the paper by O’Quigley, [10] because numerous modifications of the initial CRM very similar to the initial paper were published, we jointly considered the set of those articles. Thus, we retrieved all papers citing O’Quigley’s paper [10] in the Web of Science on May 9, 2018. We then constructed a co-citations network, where co-citations were defined as links between two papers, both of which cited by the same paper. The relatedness of papers was based on the number of times they have been cited together. This process was used to create the innovative set for the phase I trials setting. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 5 Citation data For both settings, we retrieved all citations of the innovative set from the Web of Science on July, 23 2018. Citation data were imported into R software (https://www.R-project.org/) using the Bibliometrix package. [12] The information retrieved included the title, abstract, date of publication, journal name and category according to the classification by Incites Journal Citation (https://jcr.incites.thomsonreuters.com). We excluded animal studies by searching animal names (cat, dog, mice, mouse, rat, monkey, primate, macaque) in the title. Each citation article was also segregated to one of the three following categories according to the journal of publication: 1) applied journals, 2) semi-applied journals, and 3) methodological journals (Table 3). For citations of the phase I setting, one investigator (A V) manually reviewed the papers with “phase I” in title to check if they were actual dose-finding phase I clinical trials. Statistical analysis Analysis was performed for each setting, separately. Summary statistics were reported, either mean (standard deviation) or median [interquartile range]. We plotted the cumulative number and proportions of “applied”, “semi-applied”, and “methodological” fractions of citations 5 over time as a percentage of the citations classified in the three categories of journals defined above. We defined translational gap as the time between the publication of the biostatistical paper and the time it reached 25 citations in applied literature, as defined by Altman; [4] estimated cumulative incidence of translational gap was estimated by the Kaplan Meier approach, due to the administrative censoring of the data on 2018. To obtain further insight into the medical literature citing these innovative statistical tools, we constructed a citation network using the journal as the statistical unit (rather than the paper). Use of this network, together with clustering techniques, allowed us to analyze which medical areas All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 6 (using the journal as a proxy) cited the most statistical innovations. V osViewer was used to create maps and clustering analysis. [14] Ethical Statement All methods were carried out in accordance with relevant guidelines and regulations. No informed consent or ethics approval was necessary because this study is based on publicly available data and involved no individual patient data collection or analysis.

The innovative sets of papers used for both settings are reported in Table 2. They included 2 articles for the competing risks set, and 28 for the phase I set. Competing risks set A total of 6,727 citations (5,863 unique articles), including 2,431 for Gray’s article and 4,296 for the Fine and Gray paper, were found (Table 1). Time to translational gap was 6.2 years for the Gray’s paper and 4.5 years for the Fine and Gray paper. The vast majority (91%) of the citations originated from applied journals, with a sharp and continuous increase in citations over time (Figure 1); in contrast, the citations from methodological and semi-applied journals represented only small percentages (6% and 2%, respectively) of the citing papers over the entire period. This was confirmed by the representation of the network of these journals (Figure 2). Beside a methodological cluster, two major clusters confirmed the two main areas of application of competing risks, namely oncology (e.g., Journal of Clinical Oncology, European Journal of Cancer and Hematology) and hematology (e.g., Blood, Haematologica, and Bone Marrow Transplantation). Other clusters represent other areas of application, such as cardiovascular diseases. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 7 Phase I clinical trials set After excluding books and animal studies, a total of 2,639 citations (1,114 unique papers after removing duplicates) for the phase I innovative set were found. The three most cited papers were published by O’Quigley in 1999 (676 citations), Goodman in 1995 (258 citations) and Babb in 1998 (235 citations). Contrarily to the competing risks setting, the translational gap differed across papers, from 3.4 years up to at least 20.1 years since not reached for 6 papers. Overall, it was reached by 10.7% of the set articles at year-5 and by 47.9% at year-10. Less than one half (44%) of citing articles were published in applied journals, more than one third (36%) in statistics journals, and a fifth (20%) in semi-applied journals, with roughly similar rate of citations over time (Figure 1). When we restricted the citations to those articles with “phase I” in the title, only 415 (37%) articles were selected; of these, only 110 were found to be phase I clinical trials after manual reviewing the titles. Network of the journals of papers citing one of the phase I set is displayed in Figure 2. The right part of the graph represents the methodological cluster containing, for example, Statistics in Medicine and Biometrics. The left part of the graph represents the large group concerning applications or reviews of the CRM in the most important clinical cancer journals (Journal of Clinical Oncology, Clinical Cancer Research, Annals of Oncology, British Journal of Cancer, and Journal of National Cancer Institute); in contrast, non-cancer journals had published only a few studies.

This study aimed to check whether the time lags in the statistical translation process was actually shortened, focusing on two areas with a large potential for clinical research improvements and widely encountered in the oncology literature, namely, the survival methods for competing risks All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 8 data and the new designs for phase I clinical trials. The answer was bifid, with a short translational gap, defined by the time to reach 25 citations, [4] of about 5 years for the former, but delayed above 10 years for one half of the later. This could be expected given the two statistical innovative sets are used at different stages of clinical studies, with different levels of complexity. Indeed, the first set of competing risks articles deals with method of data analysis, that can easily be performed using modern statistical software without the need for major expertise - although whether such studies are used, performed and interpreted correctly can be another matter. [15] Therefore, such a lag time of about 5 years for using an innovative data analytic method is in agreement with the time to publication after completion of data collection and analysis recently estimated at about 3 years in six journals with high impact factors. [16] In contrast, the second set of innovative methods concern a change in trial design and logistics; thus, beside the time of analysis and publication, the transfer of innovative clinical trial design into the medical literature is obviously impacted by the additional constraints of patient enrollment time, and the follow-up period for the end point. It moreover requires statisticians to engage since the planning phase, [17] up to the analysis of the data, contrary to traditional methods such as the ‘3+3’ design, which can be used without involving statisticians and computer programs and remains the most common choice among clinicians for phase I dose- escalation oncology trials. [18] Moreover, difference in complexity of both settings was illustrated in terms of citation patterns. Although the competing risks methodology was widely diffused over the medical (in particular, oncology) community, methodology relating to innovative designs for cancer Phase I trials failed to translate easily into practice, consistent with the results of a previous study that showed a very slow transfer of phase I design improvements into clinical practice. [19] Nevertheless, the two settings share large implications for clinical research, and researchers trying to apply the statistical innovative methods should not be delayed in using the new All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 9 knowledge. This is notably true in the setting of phase I trials in oncology, where improved selection of patients due to improvements in translational medicine should translate into faster and more precise dose determination. [20] Diffusing more widely into the applied literature and the medical community could be achieved in several ways. First, communication between biostatisticians and clinical colleagues should be improved, although many reviews have been published on these methods in the medical literature, as illustrated by the non-negligible fraction of semi-applied and applied papers that are different from the original clinical trials used in our study. This could be driven through key gateway journals, as suggested by the clusters of citation journals where Journal of Clinical Oncology and Statistics in Medicine appear major players (Figure 2). Concerning competing risks analysis, for which adequate modern methods have been integrated into all modern statistical software, a key step may be to educate clinicians to recognize the settings in which competing risks are of concern and to persuade them of the importance of using those methods. For the design of phase I studies, improvements in providing concrete guidance for designing such trials and facilitating their implementation in practice is still mandatory for bridging the gap between statistical innovation and practical implementation. Model-based designs are cited in the US FDA guidance for industries that are related to adaptive clinical trials as “less understood models,” and the FDA highlights some of their disadvantages. [21] Our study has some limitations. First, we used citations as a measure of the transfer of knowledge between researchers and physicians. Such citation counts, although not a direct measure of the intrinsic significance of a research idea, provide a measure of statistical technology transfer and of its impact. [4] However, in addition to papers that focus on ranking journals and impact factors [22] many others have considered the citations only qualitatively. [23] Second, we segregated articles into three categories of applied, semi-applied and methodological journals, as previously reported. [5] However, this distinction is somewhat simplistic because the applied All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 10 fractions of articles published in statistical journals may increase over time; this is reportedly highest for statistical medicine. [5] Third, we focused on two main and distinct topics of biostatistics, and the results might differ for other statistical innovations, such as dynamic prediction modelling, the joint modelling of longitudinal data, and multiple imputation techniques for handling missing data.

In summary, statistical innovations for the competing risks setting have been widely diffused in the medical literature, especially in oncology and hematology, fulfilling Altman’s prediction about decreasing lag times, unlike the model-based designs for phase I trials, which are still seldom used 30 years after their first publication. However, for both statistical methods, a translational gap remains that needs to be filled before the oncology community can benefit fully from these modern methods.

[1] Green LW, Ottoson JM, García C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health 2009;30:151–74. doi:10.1146/annurev.publhealth.031308.100049. [2] Gal D, Thijs B, Glänzel W, Sipido KR. A Changing Landscape in Cardiovascular Research Publication Output. J Am Coll Cardiol 2018;71:1584–9. doi:10.1016/j.jacc.2018.01.073. [3] Dalasanur Nagaprashantha L, Adhikari R, Singhal J, Chikara S, Awasthi S, Horne D, et al. Translational opportunities for broad-spectrum natural phytochemicals and targeted agent combinations in breast cancer: Translational strategies for breast cancer prevention and therapy. Int J Cancer 2018;142:658–70. doi:10.1002/ijc.31085. [4] Altman DG, Goodman SN. Transfer of Technology From Statistical Journals to the Biomedical Literature: Past Trends and Future Predictions. JAMA 1994;272:129–32. doi:10.1001/jama.1994.03520020055015. [5] Schell MJ. Identifying Key Statistical Papers From 1985 to 2002 Using Citation Data for Applied Biostatisticians. Am Stat 2010;64:310–7. doi:10.1198/tast.2010.08250. [6] Lesko CR, Lau B. Bias Due to Confounders for the Exposure–Competing Risk Relationship. Epidemiology 2017;28:20. doi:10.1097/EDE.0000000000000565. [7] Gray RJ. A Class of K-Sample Tests for Comparing the Cumulative Incidence of a Competing Risk. Ann Stat 1988;16:1141–54. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 11 [8] Fine JP , Gray RJ. A Proportional Hazards Model for the Subdistribution of a Competing Risk. J Am Stat Assoc 1999;94:496–509. doi:10.1080/01621459.1999.10474144. [9] Decullier E, Chan A-W, Chapuis F. Inadequate Dissemination of Phase I Trials: A Retrospective Cohort Study. PLOS Med 2009;6:e1000034. doi:10.1371/journal.pmed.1000034. [10] O’Quigley J, Pepe M, Fisher L. Continual Reassessment Method: A Practical Design for Phase 1 Clinical Trials in Cancer. Biometrics 1990;46:33–48. doi:10.2307/2531628. [11] R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria; 2016. [12] Aria M, Cuccurullo C. bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetr 2017;11:959–75. [13] Clavirate Analytics. Journal Citations Report 2017. https://jcr.incites.thomsonreuters.com/. [14] van Eck NJ, Waltman L. Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics 2017;111:1053–70. doi:10.1007/s11192-017-2300-7. [15] Austin PC, Fine JP. Accounting for competing risks in randomized controlled trials: a review and recommendations for improvement. Stat Med 2017;36:1203–9. doi:10.1002/sim.7215. [16] Welsh J, Lu Y , Dhruva SS, Bikdeli B, Desai NR, Benchetrit L, et al. Age of Data at the Time of Publication of Contemporary Clinical Trials. JAMA Netw Open 2018;1:e181065. doi:10.1001/jamanetworkopen.2018.1065. [17] Thorlund K, Haggstrom J, Park JJ, Mills EJ. Key design considerations for adaptive clinical trials: a primer for clinicians. BMJ 2018;360:k698. doi:10.1136/bmj.k698. [18] Love SB, Brown S, Weir CJ, Harbron C, Yap C, Gaschler-Markefski B, et al. Embracing model-based designs for dose-finding trials. Br J Cancer 2017;117:332–9. doi:10.1038/bjc.2017.186. [19] Rogatko A, Schoeneck D, Jonas W, Tighiouart M, Khuri FR, Porter A. Translation of innovative designs into phase I trials. J Clin Oncol Off J Am Soc Clin Oncol 2007;25:4982–6. doi:10.1200/JCO.2007.12.1012. [20] Wong KM, Capasso A, Eckhardt SG. The changing landscape of phase I trials in oncology. Nat Rev Clin Oncol 2016;13:106–17. doi:10.1038/nrclinonc.2015.194. [21] US Food and Drug Administration. Guidance for Industry. Adaptive Design Clinical Trials for Drugs and Biologics. 2010. [22] Varin C, Cattelan M, Firth D. Statistical modelling of citation exchange between statistics journals. J R Stat Soc Ser A Stat Soc 2016;179:1–63. doi:10.1111/rssa.12124. [23] Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: the “Statistical Analyses and Methods in the Published Literature” or the SAMPL Guidelines. Int J Nurs Stud 2015;52:5–9. doi:10.1016/j.ijnurstu.2014.09.006. [24] Nieminen P, Carpenter J, Rucker G, Schumacher M. The relationship between quality of research and citation frequency. BMC Med Res Methodol 2006;6:42. doi:10.1186/1471-2288- 6-42. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 12 Competing interests: The author(s) declare no competing interests. Funding: There was no specific funding for this study. Authors’ contributions: SC had the original idea. SC and A V designed the study. A V conducted the literature search, extracted and analyzed data. A V , VL, and SC participated in the interpretation of the data. A V drafted the manuscript. A V , VL, and SC authors reviewed the manuscript and approved the final version. Availability of data and material: Data used in this study are publicly available data. The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

We wish to thank Dr Lucie Biard for helpful comments on the manuscript. We acknowledge the paid contribution of American Journal Experts (Research Square, Durham, NC, USA) for copyediting. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 13 Table 1: Number of citations retrieved from Web of Science on July 23, 2018 Setting Competing Risks Phase I trials No. of articles in the innovative set a 2 28 Number of citations Mean per year 232 97 Per article: median [IQR] 3,364 [2,897-3,830] 41.5 [28-115.75] Total 6,727 2,639 Type of journals,b n (%) Applied 6,098 (91) 852 (32) Semi-Applied 158 (2) 544 (21) Methodological 471 (7) 1,243 (47) Time to bridge translation gap Cumulative incidence at 5 years 50% 10.7% Cumulative incidence at 10 years 100% 47.9% No. of unique citing articles 5,863 1,114 Type of journals,a n (%) Applied 5,369 (92) 493 (44) Semi-Applied 134 (2) 218 (20) Methodological 360 (6) 403 (36) Mean number of citations per year 232 36 a. The list of journals of each innovative set is presented in Table 2 b. The list of journals of each category is presented in Table 3. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 14 Table 2. List of selected articles for each innovative set. First Author Journal Pub. Year No. of citations Time to 25 citations (years) Competing Risks Setting Fine Journal of The American Statistical Association 1999 4,296 4.5 Gray The Annals of Statistics 1988 2,431 6.2 Phase I setting O’Quigley Biometrics 1990 676 5.7 Goodman Statistics In Medicine 1995 258 5.0 Babb J Statistics In Medicine 1998 235 5.3 Cheung Biometrics 2000 181 3.4 O’Quigley Biometrics 1996 170 7.1 Korn Statistics In Medicine 1994 144 5.5 Ratain Journal of The National Cancer Institute 1993 136 4.6 Braun Controlled Clinical Trials 2002 109 5.6 Moller Statistics In Medicine 1995 85 8.8 Piantadosi Cancer Chemotherapy And Pharmacology 1998 84 10.4 Chevret Statistics In Medicine 1993 68 10.3 Whitehead Journal of Biopharmaceutical Statistics 1998 58 9.8 Mick Journal of The National Cancer Institute 1993 46 8.4 Tighiouart Statistics In Medicine 2005 42 9.5 O’Quigley Biometrics 1992 41 9.5 Reiner Computational Statistics & Data Analysis 1999 38 13.2 Ishizuka Statistics In Medicine 2001 37 10.9 O'Quigley Biostatistics 2002 36 11.8 O'Quigley Biometrics 2003 35 11.8 O'Quigley Journal of Biopharmaceutical Statistics 1999 32 16.6 Babb Statistics In Medicine 2001 29 15.6 Zacks Statistics & Probability Letters 1998 25 20.1 Rogatko Clinical Cancer Research 2005 19 Not reached O'Quigley Journal of Statistical Planning And Inference 2006 17 Not reached Doussau Statistics In Medicine 2013 14 Not reached Legedza Controlled Clinical Trials 2000 10 Not reached Shu Statistics In Medicine 2008 7 Not reached Mahmood Journal of Clinical Pharmacology 2001 7 Not reached All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 15 Table 3. List of selected journals for each class. Applied Semi-Applied Methodological All other journals 1. Trials 2. Contemp Clin Trials 3. Clin Trials 4. Int J Clin Pharm Th 5. Expert Rev Clin Phar 6. Invest New Drug 7. Neurotherapeutics 8. Ther Innov Regul Sci 9. Cancer Chemoth Pharm 10. Expert Opin Drug Met 11. Fund Clin Pharmacol 12. Clin Pharmacol Ther 13. Control Clin Trials 14. Phytother Res 15. Cancer Drug Design And Discovery 16. Drug Inf J 17. Pharm Med 18. J Food Drug Anal 19. Toxicon 20. Bmc Med Res Methodol 21. Oncology Clinical Trials 22. Brit J Clin Pharmaco 23. Expert Opin Drug Dis 24. Pers Med 25. Drug Discovery And Development - Present And Future 26. Cancer Drug Discov D 27. J Pharmacokinet Phar 28. Clin Pharmacokinet 29. Aaps J 30. Icsa Book Ser Stat 31. Drug Discovery And Development - From Molecules To Medicine 32. Re-Engineering Clinical Trials: Best Practices For S treamlining Drug Development 33. Aliment Pharm Ther 34. Pharm Res-Dordr 35. Essential Cns Drug Development 36. Am J Epidemiol 37. Therapie 38. Drug Develop Res 39. Anticancer Therapeutics 40. Anti-Cancer Drug 41. E Schering Res Fdn W 42. J Clin Pharmacol 43. J Clin Pharm Ther 1. Stat Med 2. Stat Pap 3. Stat Methods Med Res 4. J Biopharm Stat 5. Stat Interface 6. Pharm Stat 7. Biostatistics 8. Comput Meth Prog Bio 9. Commun Stat Appl Met 10. J Am Stat Assoc 11. Biometrical J 12. Ann Appl Stat 13. Biometrics 14. J Stat Softw 15. Comput Stat Data An 16. Appl Bioinf Biostat 17. Adaptive And Flexible Clinical Trials 18. J Appl Stat 19. Stat Sci 20. Fundamentals Of Clinical Trials, Fourth Edition 21. Stat Biopharm Res 22. J Roy Stat Soc C-App 23. Handb Stat 24. Wiley Ser Probab St 25. J Stat Plan Infer 26. Biometrika 27. Stat Model 28. Sequential Anal 29. Stat Sinica 30. J R Stat Soc B 31. J R Stat Soc C-Appl 32. Stat Biosci 33. Environmetrics 34. Aust Nz J Stat 35. Commun Stat-Theor M 36. Contr Stat 37. J Chem Inf Model 38. Stat Pract 39. Commun Stat-Simul C 40. Ch Crc Biostat Ser 41. Stat Biol Health 42. Can J Stat 43. J Stat Comput Sim 44. Bayesian Anal 45. J Off Stat 46. Stat Probabil Lett 47. Stata J 48. Cr Math 49. Springer P Math Stat 50. J Multivariate Anal All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint Page 16 44. Drug Discov Today 45. Fundam Clin Pharm 46. Am J Ther 47. Indian J Pharmacol 48. Biopharm Drug Dispos 49. Curr Opin Drug Disc 50. Curr Drug Metab 51. Eur Neuropsychopharm 52. Curr Pharm Design 53. Trends Pharmacol Sci 54. Ther Drug Monit 55. Evaluation Of Biomarkers And Surrogate Endpoints In Chronic Disease 56. Antimicrob Agents Ch 57. Eur J Clin Pharmacol 58. Int J Neuropsychoph 59. Am J Health-Syst Ph 60. Pharmacogenomics J 61. Health Technol Asses 62. J Clin Lipidol 63. J Clin Epidemiol 64. Eur J Epidemiol 65. Pharmacoepidem Dr S 66. J Chemotherapy 67. Pharmacotherapy 68. Soc Sci Med 69. Prog Neuro-Psychoph 70. Va l ue H e a l th 71. Drug Aging 72. Qual Life Res 73. Antivir Ther 74. Rev Saude Publ 75. Toxicol Appl Pharm 76. J Infect Chemother 77. In t J Clin Pract 78. Drug Safety 79. J Epidemiol 80. Bmc Health Serv Res 81. J Antimicrob Chemoth 82. Hiv Clin Trials 83. Pharmacogenomics 84. Rev Epidemiol Sante 51. Stat Ind Technol 52. J Roy Stat Soc A Sta 53. Applied Optimal Designs 54. Med Decis Making 55. Int Stat Rev 56. J Theor Biol 57. Ann I Stat Math 58. Lifetime Data Anal 59. Stat Neerl 60. Pak J Stat Oper Res 61. Bshm Bull 62. Ieee Access 63. R J 64. Comput Biol Med 65. Jmir Med Inf 66. Oxford B Econ Stat 67. Korean J Appl Stat 68. Statistics-Abingdon 69. Internet Res 70. Scand J Stat 71. Expert Syst Appl 72. Jmir Mhealth Uhealth 73. J Eval Clin Pract 74. Ch Crc Handb Mod Sta 75. Annu Rev Stat Appl 76. Electron J Stat 77. C omput-Aided Civ Inf 78. Revstat-Stat J 79. J Biomed Inform 80. Genet Epidemiol 81. Comput Oper Res 82. Theor Biol Med Model 83. Stat Method Appl-Ger 84. Computation Stat 85. J Nonparametr Stat 86. Am Stat 87. J Roy Stat Soc B 88. Bioinformatics 89. Front Artif Intel Ap 90. Int J Biostat 91. Proc Wrld Acad Sci E 92. Stud Class Data Anal 93. Res Synth Methods 94. Bmc Med Inform Decis 95. Brief Bioinform 96. 2015 2nd International Conference On Information Science And Security (Iciss) All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint A B Figure 1. Cumulative number of citations for the competing risks setting (Panel A) and the phase I trials setting (Panel B) retrieved from the Web of Science and classified in each category of journals. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint A B Figure 2. Citation network of journals for the two innovative sets (Competing risks- Fig 2A, Phase I trials- Fig 2B). For clarity, only journals with at least 5 citations are represented. All rights reserved. No reuse allowed without permission. certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which was notthis version posted June 25, 2019. ; https://doi.org/10.1101/19000638doi: medRxiv preprint