Parametric analysis and normality assumption in phase 3 trials with small sample sizes

Parametric analysis and normality assumption in phase 3 trials with small sample sizes | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Parametric analysis and normality assumption in phase 3 trials with small sample sizes Seong Kyung Kim, Soeun Kim, Jisu Park, Eojin Han, Hayeon Kim, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4447330/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background In studies with small sample sizes, the use of nonparametric methods is generally recommended for statistical analysis. However, various studies continue to employ parametric analysis without verifying the assumption of a normal distribution. Objectives To assess the current utilization of parametric and nonparametric methods for primary outcomes, as well as the reporting of normality assumptions, in phase 3 clinical trials with small sample sizes. Methods All phase 3 trials registered on ClinicalTrials.gov until September 12, 2023, were collected. After undergoing a two-step selection process, only publications with a sample size per group of less than 30, involving two or more groups, and specifying the statistical methods used to compare the means or medians of primary outcomes between groups were selected. Statistical methods were categorized as nonparametric, parametric, and either parametric or nonparametric. The reporting of normality assumptions was also evaluated. Results A total of 317 studies were assessed in this study. Among these studies, 164 (51.7%) studies conducted parametric analysis, and 111 (35.0%) studies employed nonparametric analysis; however, 42 (13.2%) studies conducted parametric or nonparametric analysis without specifying which method was used. In addition, 63.1% of the total studies did not report normality assumptions. Specifically, within the subset of studies with parametric analysis, 70.1% of studies did not report normality assumptions. Conclusions This research demonstrated that most studies with small sample sizes employed parametric analysis without reporting normality assumptions. The findings emphasize the need for increased awareness of and compliance with statistical principles in the analysis of phase 3 clinical trials with limited sample sizes. Clinical Trial Phase III Sample Size Statistics Nonparametric Figures Figure 1 Figure 2 Figure 3 Figure 4 Background In clinical studies, a comparison of mean or median values between groups is common. Researchers have the option to choose between parametric and nonparametric methods for such a comparison. Nonparametric tests are distribution-free or rank-based methods, whereas parametric analysis makes the following assumptions: 1) data in each group are normally distributed (normality), 2) data in each group have approximately equal variances (homogeneity of variance), and 3) data in each group are independent of the data in every other group (independence) [ 1 , 2 ]. The distribution of sample means approaches a normal distribution as dictated by the central limit theorem, irrespective of the population distribution, provided that the sample size is sufficiently large [ 3 ]. However, the sample size in clinical trials is often not sufficiently large to ensure a normal distribution [ 4 ]. When the sample size is less than 7–10, nonparametric methods are recommended [ 5 – 8 ]. When the sample size ranges from 10 to 30, a normality test is advised [ 9 ]. Based on the results, one can choose between parametric or nonparametric analysis. Common examples of normality tests include the Shapiro-Wilk test, Kolmogorov-Smirnov test, Anderson-Darling test, and D’Agostino’s K-squared test [ 10 ]. The normality assumption must be checked before using either parametric or nonparametric methods because the validity of the results depends on it [ 2 , 11 ]. The use of parametric methods for data that require nonparametric methods can substantially influence the rates of type I error (false positive) and type II error (false negative) and subsequently reduce statistical power [ 12 ]. On the other hand, compared with parametric analysis, nonparametric analysis tends to be more robust and powerful [ 4 , 13 ]. For instance, the Wilcoxon rank-sum test is more powerful than Student’s t-test for small sample sizes when dealing with asymmetric distributions [ 4 ]. In addition, when the distributions are mixtures of normal, exponential, and double exponential, the Kruskal-Wallis test is substantially more powerful than analysis of variance (ANOVA) [ 4 ]. Statistical errors are common in scientific literature, with approximately 50% of publications containing at least one error [ 11 ]. As clinical studies are vital in evidence-based medicine, erroneous statistical analyses may lead to inaccurate assessments of the quality of healthcare. To ensure the validity of results and to enhance statistical power, parametric methods require validation of the underlying assumptions of normality. However, several studies have performed parametric analysis without verifying the normality assumption [ 14 , 15 ]. To date, no research has determined whether assumptions are adequately addressed with a focus on clinical studies with small sample sizes. This study aimed to assess the current utilization of parametric and nonparametric methods for primary outcomes, as well as the reporting of normality assumptions, in phase 3 clinical trials with small sample sizes. Methods Study selection All phase 3 trials registered on ClinicalTrials.gov until September 12, 2023, were collected. The initial selection of clinical trials was based on the information provided on the website. Studies were included if they 1) had been completed (defined as having concluded normally with participants no longer under examination or treatment) before the search date, 2) had two or more intervention types (file downloaded from the website contains information only on interventions instead of groups), 3) had a sample size of less than 30 for each intervention (calculated by dividing the total enrollment by the number of intervention types), and 4) had publication data available on the search date. Publication data included links to publications provided by the data provider (sponsors or investigators) or identified by the ClinicalTrials.gov identifier (NCT number) in Medline. Subsequently, the selection process was conducted by accessing the full text using the publication links provided by ClinicalTrials.gov. Studies were included if they 1) had two or more groups and 2) had a sample size of less than 30 in each group. The accurate number of groups and sample size per group can be confirmed through the full text. Studies were excluded if they 1) were not relevant, where the publication data only included references that the data provider consulted for the study design, 2) were only publications of study protocols, without presenting the study results, 3) were publications with inaccessible full text or were non-English publications, 4) were not phase 3 trials, 5) were single-arm trials, 6) had a sample size of 30 or more in each group, 7) did not specify statistical methods, and 8) did not compare the means or medians of primary outcomes between groups. Assessment of statistical methods Statistical methods for primary outcomes and the consideration of normality assumptions were assessed. The methods were classified into three categories: 1) parametric; 2) nonparametric; and 3) either parametric or nonparametric. The third category referred to cases where both parametric and nonparametric methods were mentioned in the methodology of the study, making it difficult to determine which method was actually used. Parametric methods and their nonparametric counterparts included in the studies are shown in Table 1 . Table 1 Parametric methods and their nonparametric counterparts in the studies Parametric Nonparametric Student’s t-test Wilcoxon rank-sum test, Mann-Whitney U test Paired t-test Wilcoxon signed rank test ANOVA Kruskal-Wallis test, Friedman test Statistical analysis Descriptive statistics were used to summarize the characteristics of the included studies. Continuous variables are presented as the mean ± standard deviation or median (interquartile range [IQR]) based on the results of normality analysis, whereas for categorical variables, absolute and relative frequencies are presented. Normality was assessed using the Shapiro-Wilk test and visually compared with Q-Q plots. The proportions of studies using the statistical methods for primary outcomes and reporting normality assumptions were calculated. Subgroup analysis was performed according to whether the sample size per group was less than 10 or 10–30. Additionally, subgroup analysis was conducted for studies where primary outcomes were ordinal data, such as pain scales. All analyses were performed using Excel and R version 4.3.2. Results Characteristics of the included studies Figure 1 shows the study selection flowchart. Of 43,527 phase 3 trials, 1,048 studies were initially selected based on website information. Following full-text reviews, 317 studies and their publications were finally included in the analysis. Table 2 shows the characteristics of the included studies. The median sample size per group was 19 (IQR, 13–23), with the majority falling within the range of 10–30 (n = 289, 91.2%). Most studies employed a parallel study design (n = 281, 88.6%). The primary focus of the studies included endocrine and metabolic diseases (n = 57, 18.0%), circulatory diseases (n = 38, 12.0%), and mental disorders (n = 33, 10.4%). The list of the included studies is presented in Additional file 1. Table 2 Characteristics of the included studies Characteristics Total Total number of studies 317 Sample size per group, median (IQR) 19 (13–23) < 10, n (%) 28 (8.8) 10–30, n (%) 289 (91.2) Study design, n (%) Parallel 281 (88.6) Crossover 36 (11.4) Target conditions, n (%) Endocrine, nutritional or metabolic diseases 57 (18.0) Circulatory diseases 38 (12.0) Mental disorders 33 (10.4) Musculoskeletal or connective tissue diseases 30 (9.5) Neurological diseases 26 (8.2) Digestive diseases 24 (7.6) Respiratory diseases 21 (6.6) Neoplasm 17 (5.4) Infectious diseases 16 (5.0) Genitourinary diseases 13 (4.1) Diseases of the blood and blood-forming organs 9 (2.8) Skin diseases 6 (1.9) Injury, poisoning and other consequences of external causes 5 (1.6) Immune system diseases 3 (0.9) Diseases of the visual system 2 (0.6) Others 17 (5.4) IQR, interquartile range Distribution of statistical methods Figure 2 shows the distribution of statistical methods used in the studies and reporting of normality assumptions. Overall, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 164 (51.7%), 111 (35.0%), and 42 (13.2%) studies, respectively. Among 317 studies, 63.1% of them did not report normality assumptions. Among studies with parametric analysis, 70.1% (n = 115) of them did not report normality assumptions, constituting 36.3% of the total studies included. Among studies with nonparametric analysis, 67.6% (n = 75, 23.7% of the total) of them did not mention normality assumptions, and among those employing either parametric or nonparametric analysis, 23.8% (n = 10, 3.1% of the total) of them did not mention normality assumptions. A total of 13 publications performed log transformations on primary outcomes. Figure 3 shows the results of subgroup analysis according to the sample size per group. Among studies with a sample size of less than 10, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 18 (64.3%), 8 (28.6%), and 2 (7.1%) studies, respectively. Within the subset of studies with parametric analysis, 83.3% of studies (n = 15, 53.6% of the total) did not report normality assumptions. Among studies with a sample size of 10–30, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 146 (50.5%), 100 (34.6%), and 40 (13.7%) studies, respectively. Among these studies, 61.2% of them did not mention normality assumptions. Specifically, among studies with parametric analysis, this percentage was 68.5%, and among those with nonparametric analysis, it was 66.0%. Of 70 studies that analyzed ordinal data as the primary outcomes, 16 of them focused on pain levels. Figure 4 shows the distribution of the statistical methods used and reporting of normality assumptions in studies that analyzed ordinal data. Parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 42 (60.0%), 23 (32.9%), and 5 (7.1%) studies, respectively. Within the subset of studies with parametric analysis, 83.3% of studies (n = 35, 50% of the total) did not report normality assumptions. Discussion This study assessed statistical methods and normality assumptions in phase 3 clinical trials with small sample sizes and identified three points of concern. First, most studies did not report normality assumptions. In particular, studies employing parametric methods failed to consider normality assumptions. Second, around 10% of the studies did not specify whether they used parametric or nonparametric methods. Third, the majority of the studies that analyzed ordinal data as primary outcomes used parametric methods instead of nonparametric methods, and most of them did not mention normality assumptions. In this study, we observed that 70.1% of studies employing parametric methods did not mention normality assumptions. Parametric tests are predicated on the assumption that the data within each group adhere to a normal distribution. According to the central limit theorem, parametric tests are valid when the sample size is 30 or more [ 11 ]. However, when the sample size ranges from 10 to 30, normality in the population is uncertain, thus necessitating a normality test [ 9 ]. In addition, if the sample size is less than 10, nonparametric methods are preferred [ 5 – 8 ]. In our study, we noted that 64.3% of studies with a sample size per group of less than 10 used parametric tests. Furthermore, among studies with a sample size per group of 10–30, 61.2% of them did not report normality assumptions. We also found that 13.2% of studies used either parametric or nonparametric methods but did not specify which statistical method was employed. Among these studies, 23.8% of them did not mention normality assumptions. The selection of methods during the statistical analysis of clinical trial results has the potential to influence outcomes [ 15 ]. Therefore, the methods section should clearly state which statistical methods were used, as suggested by CONSORT, the reporting guidelines for randomized trials [ 16 ]. In addition, if parametric tests were used, whether normality assumptions were met should be reported explicitly or justifications should be provided (e.g., referring to the central limit theorem) [ 17 ]. This practice will reduce the risk of incorrect analysis or data-driven biased trial results [ 15 ]. We observed that 60.0% of studies that analyzed ordinal data as primary outcomes used parametric methods instead of nonparametric methods. Within this subset, 83.3% of studies did not mention assumptions. The pitfalls of using parametric tests to analyze ordinal data are well known, especially in pain research using a visual analog scale [ 18 ]. Parametric tests assume that the scale of the data is a ratio or an interval [ 17 ]. However, ordinal data lack the quantitative properties assumed by parametric tests and cannot be interpreted quantitatively; differences between them cannot be compared as numerical data [ 19 ]. Treating ordinal data as if they were metric data can lead to systematic errors, including type I errors and type II errors [ 20 ]. Moreover, this approach can result in the systematic inversions of effects, indicating the opposite ordering of means compared with the true ordering [ 20 ]. Therefore, nonparametric methods are more appropriate for the analysis of ordinal data [ 5 , 8 ]. Additionally, in 13 studies, data transformation was applied to use parametric methods. Transformations, such as the square root or logarithm, are widely applied in clinical studies to deal with skewed data. In many cases, transformations are incorrectly thought to reduce data variability and normalize distributions. Moreover, their use presents various challenges, including failure to restore normality, neglect of outliers, potential reduction in statistical power, and production of estimates often irrelevant to the original data [ 21 , 22 ]. Non-reproducible single occurrences lack scientific significance, underscoring the crucial role of reproducibility in establishing scientific knowledge or estimates. Many trials have failed to provide sufficient information on the statistical methods and assessment of assumptions. This lack of transparency can compromise the reliability of true values. To prevent this issue and improve clarity in reporting clinical trials, it is essential to thoroughly describe every step of the process, including the tests conducted to assess assumptions for the selected statistical methods [ 15 ]. Our findings highlight the need for efforts to improve the awareness and understanding of statistical assumptions to enhance the integrity and reproducibility of research. This study has several limitations. First, this study focused on evaluating the appropriateness of statistical analyses for comparing mean or median values. The appropriateness of other statistical methods used in clinical studies lies beyond the scope of this study. Second, if the population follows a normal distribution, parametric analyses can be performed with a small sample size. However, since it was not feasible to confirm whether the primary outcomes in the population followed a normal distribution, we did not assume a normal distribution in the population. Lastly, this study is limited to trials registered on ClinicalTrials.gov. However, our study was designed to point out statistical issues rather than to confirm the exact frequency. Conclusion We found that most studies with small sample sizes employed parametric analysis without reporting normality assumptions. The findings emphasize the need for increased awareness of and compliance with statistical principles in the analysis of phase 3 clinical trials with limited sample sizes. Abbreviations ANOVA: analysis of variance IQR: interquartile range Declarations Ethics approval and consent to participate : Not applicable. Consent for publication : Not applicable. Availability of data and materials : All data generated or analysed during the current study are included in this published article and its supplementary information files. Competing interests : The authors declare that they have no competing interests. Funding : This study was supported by the Clinical Pharmacy Research Network funded by the Korean College of Clinical Pharmacy (no grant number applicable). Authors' contributions : All authors contributed to the study concept and design. SKK, SK, JP, EH, and HK contributed to preparation and analysis of the data. SKK drafted the manuscript. SKK, HK, MGK, and KK contributed to the interpretation of the data. YGC verified the accuracy of the interpretation. YGC, MGK, and KK critically reviewed the manuscript for publication. Acknowledgements : Not applicable. Supplemental Information: Additional file 1. An Excel sheet that contains the list of included studies (NCT number and article title). References McCrum-Gardner E. Which is the correct statistical test to use? British Journal of Oral and Maxillofacial Surgery. 2008;46(1):38-41. Petrie A, Sabin C. Medical statistics at a glance. 4th ed. Hoboken (NJ): Wiley Blackwell; 2020. Kwak SG, Kim JH. Central limit theorem: the cornerstone of modern statistics. Korean journal of anesthesiology. 2017;70(2):144. Kitchen CM. Nonparametric vs parametric tests of location in biomedical research. American journal of ophthalmology. 2009;147(4):571-2. Siegel S. Nonparametric statistics. The American Statistician. 1957;11(3):13-9. Sidney S. Nonparametric statistics for the behavioral sciences. The Journal of Nervous and Mental Disease. 1957;125(3):497. VanVoorhis CW, Morgan BL. Understanding power and rules of thumb for determining sample sizes. Tutorials in quantitative methods for psychology. 2007;3(2):43-50. Tomkins CC, Hall C. An introduction to non-parametric statistics for health scientists. University of Alberta Health Sciences Journal. 2006;3(1):20-6. Orcan F. Parametric or non-parametric: Skewness to test normality for mean comparison. International Journal of Assessment Tools in Education. 2020;7(2):255-65. Razali NM, Wah YB. Power comparisons of shapiro-wilk, kolmogorov-smirnov, lilliefors and anderson-darling tests. Journal of statistical modeling and analytics. 2011;2(1):21-33. Ghasemi A, Zahediasl S. Normality tests for statistical analysis: a guide for non-statisticians. International journal of endocrinology and metabolism. 2012;10(2):486. Dwivedi AK, Mallawaarachchi I, Alvarado LA. Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method. Statistics in medicine. 2017;36(14):2187-205. Gibbons JD, Chakraborti S. Comparisons of the Mann-Whitney, Student’st, and alternate t tests for means of normal distributions. The Journal of Experimental Education. 1991;59(3):258-67. Gerald B, Patson TF. Parametric and nonparametric tests: A brief review. Int J Stat Distrib Appl. 2021;7:78-82. Nielsen EE, Nørskov AK, Lange T, et al . Assessing assumptions for statistical analyses in randomised clinical trials. BMJ evidence-based medicine. 2019;24(5):185-9. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Journal of Pharmacology and pharmacotherapeutics. 2010;1(2):100-7. Vrbin CM. Parametric or nonparametric statistical tests: Considerations when choosing the most appropriate option for your data. Cytopathology. 2022;33(6):663-7. Kim TK. Practical statistics in pain research. The Korean journal of pain. 2017;30(4):243. Calver M, Fletcher D. When ANOVA isn't ideal: Analyzing ordinal data from practical work in biology. The American Biology Teacher. 2020;82(5):289-94. Liddell TM, Kruschke JK. Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology. 2018;79:328-48. Changyong F, Hongyue W, Naiji L, et al . Log-transformation and its implications for data analysis. Shanghai archives of psychiatry. 2014;26(2):105. Erceg-Hurn DM, Mirosevich VM. Modern robust statistical methods: an easy way to maximize the accuracy and power of your research. Am Psychol. 2008;63(7):591-601. Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.xlsx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4447330","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":308264482,"identity":"6304abc0-66c1-4542-9413-7f7f92d75a61","order_by":0,"name":"Seong Kyung Kim","email":"","orcid":"","institution":"Ewha Womans University","correspondingAuthor":false,"prefix":"","firstName":"Seong","middleName":"Kyung","lastName":"Kim","suffix":""},{"id":308264483,"identity":"d12780a1-9945-4861-9ed7-6f2c5f131ace","order_by":1,"name":"Soeun Kim","email":"","orcid":"","institution":"Ewha Womans University","correspondingAuthor":false,"prefix":"","firstName":"Soeun","middleName":"","lastName":"Kim","suffix":""},{"id":308264484,"identity":"bc3be55d-ff2c-4803-b45e-d751488cf125","order_by":2,"name":"Jisu Park","email":"","orcid":"","institution":"Korea University","correspondingAuthor":false,"prefix":"","firstName":"Jisu","middleName":"","lastName":"Park","suffix":""},{"id":308264485,"identity":"d736965c-ecd9-45de-8e9e-b72c8d3f0bf4","order_by":3,"name":"Eojin Han","email":"","orcid":"","institution":"Korea University","correspondingAuthor":false,"prefix":"","firstName":"Eojin","middleName":"","lastName":"Han","suffix":""},{"id":308264486,"identity":"b4c7ad40-e8db-415b-bdc6-9a45f5bd727b","order_by":4,"name":"Hayeon Kim","email":"","orcid":"","institution":"Korea University","correspondingAuthor":false,"prefix":"","firstName":"Hayeon","middleName":"","lastName":"Kim","suffix":""},{"id":308264487,"identity":"20773cbe-0aa1-463e-9a5e-6073ccf1371b","order_by":5,"name":"Young-Geun Choi","email":"","orcid":"","institution":"Sungkyunkwan University","correspondingAuthor":false,"prefix":"","firstName":"Young-Geun","middleName":"","lastName":"Choi","suffix":""},{"id":308264488,"identity":"ea7b938d-e07d-464c-8d07-f16bb1b8ce2e","order_by":6,"name":"Myeong Gyu Kim","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAwUlEQVRIiWNgGAWjYBACCWYQWcHAYADhJxCr5QxJWkAEYxspWiTbuVM3fp1nk7id/QDjhx8MafkEtUgz8267LbstLXFnTwKzZA9DjmUDIS1yIC2S2w7nbjiQwCANDAgDgrZAtMz5n7vh/APm30RpATns5seGA7kbbiSwAW3JIaxFshloC8Ox5PoNNx62WfYYpBHWInH+7LabP2rsjA3OJx++8aMimbAWEGDmAVOMDfDYIQgYfxCpcBSMglEwCkYoAACWrzwv+mfsoAAAAABJRU5ErkJggg==","orcid":"","institution":"Ewha Womans University","correspondingAuthor":true,"prefix":"","firstName":"Myeong","middleName":"Gyu","lastName":"Kim","suffix":""},{"id":308264491,"identity":"6994db12-86d2-4df1-a5be-2a60e3b749e6","order_by":7,"name":"Kyungim Kim","email":"","orcid":"","institution":"Korea University","correspondingAuthor":false,"prefix":"","firstName":"Kyungim","middleName":"","lastName":"Kim","suffix":""}],"badges":[],"createdAt":"2024-05-20 07:26:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4447330/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4447330/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":57652067,"identity":"7e2b3e35-d165-4a95-93b8-bec49cb85245","added_by":"auto","created_at":"2024-06-03 23:24:53","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":356169,"visible":true,"origin":"","legend":"\u003cp\u003eFlow chart of the study selection\u003c/p\u003e","description":"","filename":"Figure1.png","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/53779cef3c2811359bd69bd1.png"},{"id":57651937,"identity":"dafdf5e9-b9a9-48ff-adb4-eaaad5a0b38b","added_by":"auto","created_at":"2024-06-03 23:16:53","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":119620,"visible":true,"origin":"","legend":"\u003cp\u003eStatistical methods and reporting of normality assumptions in studies (n = 317)\u003c/p\u003e","description":"","filename":"Figure2.png","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/71673250f046f81cc2d07f1c.png"},{"id":57651939,"identity":"f8283ef5-8c4b-4f00-bfb8-527871a41e6c","added_by":"auto","created_at":"2024-06-03 23:16:53","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":241915,"visible":true,"origin":"","legend":"\u003cp\u003eStatistical methods and reporting of normality assumptions in studies according to the sample size. (A) Sample size per group \u0026lt; 10 (n = 28). (B) Sample size per group = 10–30 (n = 289)\u003c/p\u003e","description":"","filename":"Figure3.png","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/109f4a90cb8eb075ad8b2503.png"},{"id":57651941,"identity":"0a55bbfa-8a21-4776-9c31-a168da3e39d9","added_by":"auto","created_at":"2024-06-03 23:16:53","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":113876,"visible":true,"origin":"","legend":"\u003cp\u003eStatistical methods and reporting of normality assumptions in studies that analyzed ordinal data (n = 70)\u003c/p\u003e","description":"","filename":"Figure4.png","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/5d280c6124a79db1754be265.png"},{"id":75096500,"identity":"2c31ef32-5697-446f-83c5-3ae76f1f71d8","added_by":"auto","created_at":"2025-01-30 12:09:06","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1341359,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/cf9f7d0c-ca62-4fc7-abbd-7c4a3234a18a.pdf"},{"id":57651940,"identity":"0d061a02-d556-4b98-a29c-f4b42e932609","added_by":"auto","created_at":"2024-06-03 23:16:53","extension":"xlsx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":35233,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-4447330/v1/78de2955c4224a3d5011b979.xlsx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Parametric analysis and normality assumption in phase 3 trials with small sample sizes","fulltext":[{"header":"Background","content":"\u003cp\u003eIn clinical studies, a comparison of mean or median values between groups is common. Researchers have the option to choose between parametric and nonparametric methods for such a comparison. Nonparametric tests are distribution-free or rank-based methods, whereas parametric analysis makes the following assumptions: 1) data in each group are normally distributed (normality), 2) data in each group have approximately equal variances (homogeneity of variance), and 3) data in each group are independent of the data in every other group (independence) [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe distribution of sample means approaches a normal distribution as dictated by the central limit theorem, irrespective of the population distribution, provided that the sample size is sufficiently large [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. However, the sample size in clinical trials is often not sufficiently large to ensure a normal distribution [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. When the sample size is less than 7\u0026ndash;10, nonparametric methods are recommended [\u003cspan additionalcitationids=\"CR6 CR7\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. When the sample size ranges from 10 to 30, a normality test is advised [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Based on the results, one can choose between parametric or nonparametric analysis. Common examples of normality tests include the Shapiro-Wilk test, Kolmogorov-Smirnov test, Anderson-Darling test, and D\u0026rsquo;Agostino\u0026rsquo;s K-squared test [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe normality assumption must be checked before using either parametric or nonparametric methods because the validity of the results depends on it [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. The use of parametric methods for data that require nonparametric methods can substantially influence the rates of type I error (false positive) and type II error (false negative) and subsequently reduce statistical power [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. On the other hand, compared with parametric analysis, nonparametric analysis tends to be more robust and powerful [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. For instance, the Wilcoxon rank-sum test is more powerful than Student\u0026rsquo;s t-test for small sample sizes when dealing with asymmetric distributions [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In addition, when the distributions are mixtures of normal, exponential, and double exponential, the Kruskal-Wallis test is substantially more powerful than analysis of variance (ANOVA) [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eStatistical errors are common in scientific literature, with approximately 50% of publications containing at least one error [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. As clinical studies are vital in evidence-based medicine, erroneous statistical analyses may lead to inaccurate assessments of the quality of healthcare. To ensure the validity of results and to enhance statistical power, parametric methods require validation of the underlying assumptions of normality. However, several studies have performed parametric analysis without verifying the normality assumption [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. To date, no research has determined whether assumptions are adequately addressed with a focus on clinical studies with small sample sizes. This study aimed to assess the current utilization of parametric and nonparametric methods for primary outcomes, as well as the reporting of normality assumptions, in phase 3 clinical trials with small sample sizes.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy selection\u003c/h2\u003e \u003cp\u003eAll phase 3 trials registered on ClinicalTrials.gov until September 12, 2023, were collected. The initial selection of clinical trials was based on the information provided on the website. Studies were included if they 1) had been completed (defined as having concluded normally with participants no longer under examination or treatment) before the search date, 2) had two or more intervention types (file downloaded from the website contains information only on interventions instead of groups), 3) had a sample size of less than 30 for each intervention (calculated by dividing the total enrollment by the number of intervention types), and 4) had publication data available on the search date. Publication data included links to publications provided by the data provider (sponsors or investigators) or identified by the ClinicalTrials.gov identifier (NCT number) in Medline.\u003c/p\u003e \u003cp\u003eSubsequently, the selection process was conducted by accessing the full text using the publication links provided by ClinicalTrials.gov. Studies were included if they 1) had two or more groups and 2) had a sample size of less than 30 in each group. The accurate number of groups and sample size per group can be confirmed through the full text. Studies were excluded if they 1) were not relevant, where the publication data only included references that the data provider consulted for the study design, 2) were only publications of study protocols, without presenting the study results, 3) were publications with inaccessible full text or were non-English publications, 4) were not phase 3 trials, 5) were single-arm trials, 6) had a sample size of 30 or more in each group, 7) did not specify statistical methods, and 8) did not compare the means or medians of primary outcomes between groups.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eAssessment of statistical methods\u003c/h2\u003e \u003cp\u003eStatistical methods for primary outcomes and the consideration of normality assumptions were assessed. The methods were classified into three categories: 1) parametric; 2) nonparametric; and 3) either parametric or nonparametric. The third category referred to cases where both parametric and nonparametric methods were mentioned in the methodology of the study, making it difficult to determine which method was actually used. Parametric methods and their nonparametric counterparts included in the studies are shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eParametric methods and their nonparametric counterparts in the studies\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParametric\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNonparametric\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStudent\u0026rsquo;s t-test\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWilcoxon rank-sum test, Mann-Whitney U test\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePaired t-test\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eWilcoxon signed rank test\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eANOVA\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eKruskal-Wallis test, Friedman test\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eStatistical analysis\u003c/h2\u003e \u003cp\u003eDescriptive statistics were used to summarize the characteristics of the included studies. Continuous variables are presented as the mean\u0026thinsp;\u0026plusmn;\u0026thinsp;standard deviation or median (interquartile range [IQR]) based on the results of normality analysis, whereas for categorical variables, absolute and relative frequencies are presented. Normality was assessed using the Shapiro-Wilk test and visually compared with Q-Q plots.\u003c/p\u003e \u003cp\u003eThe proportions of studies using the statistical methods for primary outcomes and reporting normality assumptions were calculated. Subgroup analysis was performed according to whether the sample size per group was less than 10 or 10\u0026ndash;30. Additionally, subgroup analysis was conducted for studies where primary outcomes were ordinal data, such as pain scales. All analyses were performed using Excel and R version 4.3.2.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eCharacteristics of the included studies\u003c/h2\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the study selection flowchart. Of 43,527 phase 3 trials, 1,048 studies were initially selected based on website information. Following full-text reviews, 317 studies and their publications were finally included in the analysis.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the characteristics of the included studies. The median sample size per group was 19 (IQR, 13\u0026ndash;23), with the majority falling within the range of 10\u0026ndash;30 (n\u0026thinsp;=\u0026thinsp;289, 91.2%). Most studies employed a parallel study design (n\u0026thinsp;=\u0026thinsp;281, 88.6%). The primary focus of the studies included endocrine and metabolic diseases (n\u0026thinsp;=\u0026thinsp;57, 18.0%), circulatory diseases (n\u0026thinsp;=\u0026thinsp;38, 12.0%), and mental disorders (n\u0026thinsp;=\u0026thinsp;33, 10.4%). The list of the included studies is presented in Additional file 1.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eCharacteristics of the included studies\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCharacteristics\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTotal number of studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e317\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSample size per group, median (IQR)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e19 (13\u0026ndash;23)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;10, n (%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e28 (8.8)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e10\u0026ndash;30, n (%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e289 (91.2)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStudy design, n (%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eParallel\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e281 (88.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCrossover\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e36 (11.4)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTarget conditions, n (%)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEndocrine, nutritional or metabolic diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e57 (18.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCirculatory diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e38 (12.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMental disorders\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e33 (10.4)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMusculoskeletal or connective tissue diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e30 (9.5)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNeurological diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e26 (8.2)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDigestive diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e24 (7.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRespiratory diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e21 (6.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNeoplasm\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e17 (5.4)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eInfectious diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e16 (5.0)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGenitourinary diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e13 (4.1)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiseases of the blood and blood-forming organs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e9 (2.8)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSkin diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e6 (1.9)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eInjury, poisoning and other consequences of external causes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e5 (1.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eImmune system diseases\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3 (0.9)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiseases of the visual system\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2 (0.6)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOthers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e17 (5.4)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"2\"\u003eIQR, interquartile range\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eDistribution of statistical methods\u003c/h2\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the distribution of statistical methods used in the studies and reporting of normality assumptions. Overall, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 164 (51.7%), 111 (35.0%), and 42 (13.2%) studies, respectively. Among 317 studies, 63.1% of them did not report normality assumptions. Among studies with parametric analysis, 70.1% (n\u0026thinsp;=\u0026thinsp;115) of them did not report normality assumptions, constituting 36.3% of the total studies included. Among studies with nonparametric analysis, 67.6% (n\u0026thinsp;=\u0026thinsp;75, 23.7% of the total) of them did not mention normality assumptions, and among those employing either parametric or nonparametric analysis, 23.8% (n\u0026thinsp;=\u0026thinsp;10, 3.1% of the total) of them did not mention normality assumptions. A total of 13 publications performed log transformations on primary outcomes.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the results of subgroup analysis according to the sample size per group. Among studies with a sample size of less than 10, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 18 (64.3%), 8 (28.6%), and 2 (7.1%) studies, respectively. Within the subset of studies with parametric analysis, 83.3% of studies (n\u0026thinsp;=\u0026thinsp;15, 53.6% of the total) did not report normality assumptions. Among studies with a sample size of 10\u0026ndash;30, parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 146 (50.5%), 100 (34.6%), and 40 (13.7%) studies, respectively. Among these studies, 61.2% of them did not mention normality assumptions. Specifically, among studies with parametric analysis, this percentage was 68.5%, and among those with nonparametric analysis, it was 66.0%.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eOf 70 studies that analyzed ordinal data as the primary outcomes, 16 of them focused on pain levels. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows the distribution of the statistical methods used and reporting of normality assumptions in studies that analyzed ordinal data. Parametric analysis, nonparametric analysis, and either parametric or nonparametric analysis were used in 42 (60.0%), 23 (32.9%), and 5 (7.1%) studies, respectively. Within the subset of studies with parametric analysis, 83.3% of studies (n\u0026thinsp;=\u0026thinsp;35, 50% of the total) did not report normality assumptions.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study assessed statistical methods and normality assumptions in phase 3 clinical trials with small sample sizes and identified three points of concern. First, most studies did not report normality assumptions. In particular, studies employing parametric methods failed to consider normality assumptions. Second, around 10% of the studies did not specify whether they used parametric or nonparametric methods. Third, the majority of the studies that analyzed ordinal data as primary outcomes used parametric methods instead of nonparametric methods, and most of them did not mention normality assumptions.\u003c/p\u003e \u003cp\u003eIn this study, we observed that 70.1% of studies employing parametric methods did not mention normality assumptions. Parametric tests are predicated on the assumption that the data within each group adhere to a normal distribution. According to the central limit theorem, parametric tests are valid when the sample size is 30 or more [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. However, when the sample size ranges from 10 to 30, normality in the population is uncertain, thus necessitating a normality test [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. In addition, if the sample size is less than 10, nonparametric methods are preferred [\u003cspan additionalcitationids=\"CR6 CR7\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. In our study, we noted that 64.3% of studies with a sample size per group of less than 10 used parametric tests. Furthermore, among studies with a sample size per group of 10\u0026ndash;30, 61.2% of them did not report normality assumptions.\u003c/p\u003e \u003cp\u003eWe also found that 13.2% of studies used either parametric or nonparametric methods but did not specify which statistical method was employed. Among these studies, 23.8% of them did not mention normality assumptions. The selection of methods during the statistical analysis of clinical trial results has the potential to influence outcomes [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. Therefore, the \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003emethods\u003c/span\u003e section should clearly state which statistical methods were used, as suggested by CONSORT, the reporting guidelines for randomized trials [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. In addition, if parametric tests were used, whether normality assumptions were met should be reported explicitly or justifications should be provided (e.g., referring to the central limit theorem) [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. This practice will reduce the risk of incorrect analysis or data-driven biased trial results [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWe observed that 60.0% of studies that analyzed ordinal data as primary outcomes used parametric methods instead of nonparametric methods. Within this subset, 83.3% of studies did not mention assumptions. The pitfalls of using parametric tests to analyze ordinal data are well known, especially in pain research using a visual analog scale [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. Parametric tests assume that the scale of the data is a ratio or an interval [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. However, ordinal data lack the quantitative properties assumed by parametric tests and cannot be interpreted quantitatively; differences between them cannot be compared as numerical data [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. Treating ordinal data as if they were metric data can lead to systematic errors, including type I errors and type II errors [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Moreover, this approach can result in the systematic inversions of effects, indicating the opposite ordering of means compared with the true ordering [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Therefore, nonparametric methods are more appropriate for the analysis of ordinal data [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAdditionally, in 13 studies, data transformation was applied to use parametric methods. Transformations, such as the square root or logarithm, are widely applied in clinical studies to deal with skewed data. In many cases, transformations are incorrectly thought to reduce data variability and normalize distributions. Moreover, their use presents various challenges, including failure to restore normality, neglect of outliers, potential reduction in statistical power, and production of estimates often irrelevant to the original data [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eNon-reproducible single occurrences lack scientific significance, underscoring the crucial role of reproducibility in establishing scientific knowledge or estimates. Many trials have failed to provide sufficient information on the statistical methods and assessment of assumptions. This lack of transparency can compromise the reliability of true values. To prevent this issue and improve clarity in reporting clinical trials, it is essential to thoroughly describe every step of the process, including the tests conducted to assess assumptions for the selected statistical methods [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. Our findings highlight the need for efforts to improve the awareness and understanding of statistical assumptions to enhance the integrity and reproducibility of research.\u003c/p\u003e \u003cp\u003eThis study has several limitations. First, this study focused on evaluating the appropriateness of statistical analyses for comparing mean or median values. The appropriateness of other statistical methods used in clinical studies lies beyond the scope of this study. Second, if the population follows a normal distribution, parametric analyses can be performed with a small sample size. However, since it was not feasible to confirm whether the primary outcomes in the population followed a normal distribution, we did not assume a normal distribution in the population. Lastly, this study is limited to trials registered on ClinicalTrials.gov. However, our study was designed to point out statistical issues rather than to confirm the exact frequency.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eWe found that most studies with small sample sizes employed parametric analysis without reporting normality assumptions. The findings emphasize the need for increased awareness of and compliance with statistical principles in the analysis of phase 3 clinical trials with limited sample sizes.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eANOVA: analysis of variance\u003c/p\u003e\n\u003cp\u003eIQR: interquartile range\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e: Not applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e: Not applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e: All data generated or analysed during the current study are included in this published article and its supplementary information files.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e: The authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e: This study was supported by the Clinical Pharmacy Research Network funded by the Korean College of Clinical Pharmacy (no grant number applicable).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e: All authors contributed to the study concept and design. SKK, SK, JP, EH, and HK contributed to preparation and analysis of the data. SKK drafted the manuscript. SKK, HK, MGK, and KK contributed to the interpretation of the data. YGC verified the accuracy of the interpretation. YGC, MGK, and KK critically reviewed the manuscript for publication.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e: Not applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSupplemental Information:\u003c/strong\u003e Additional file 1. An Excel sheet that contains the list of included studies (NCT number and article title).\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eMcCrum-Gardner E. Which is the correct statistical test to use? British Journal of Oral and Maxillofacial Surgery. 2008;46(1):38-41.\u003c/li\u003e\n\u003cli\u003ePetrie A, Sabin C. Medical statistics at a glance. 4th ed. Hoboken (NJ): Wiley Blackwell; 2020.\u003c/li\u003e\n\u003cli\u003eKwak SG, Kim JH. Central limit theorem: the cornerstone of modern statistics. Korean journal of anesthesiology. 2017;70(2):144.\u003c/li\u003e\n\u003cli\u003eKitchen CM. Nonparametric vs parametric tests of location in biomedical research. American journal of ophthalmology. 2009;147(4):571-2.\u003c/li\u003e\n\u003cli\u003eSiegel S. Nonparametric statistics. The American Statistician. 1957;11(3):13-9.\u003c/li\u003e\n\u003cli\u003eSidney S. Nonparametric statistics for the behavioral sciences. The Journal of Nervous and Mental Disease. 1957;125(3):497.\u003c/li\u003e\n\u003cli\u003eVanVoorhis CW, Morgan BL. Understanding power and rules of thumb for determining sample sizes. Tutorials in quantitative methods for psychology. 2007;3(2):43-50.\u003c/li\u003e\n\u003cli\u003eTomkins CC, Hall C. An introduction to non-parametric statistics for health scientists. University of Alberta Health Sciences Journal. 2006;3(1):20-6.\u003c/li\u003e\n\u003cli\u003eOrcan F. Parametric or non-parametric: Skewness to test normality for mean comparison. International Journal of Assessment Tools in Education. 2020;7(2):255-65.\u003c/li\u003e\n\u003cli\u003eRazali NM, Wah YB. Power comparisons of shapiro-wilk, kolmogorov-smirnov, lilliefors and anderson-darling tests. Journal of statistical modeling and analytics. 2011;2(1):21-33.\u003c/li\u003e\n\u003cli\u003eGhasemi A, Zahediasl S. Normality tests for statistical analysis: a guide for non-statisticians. International journal of endocrinology and metabolism. 2012;10(2):486.\u003c/li\u003e\n\u003cli\u003eDwivedi AK, Mallawaarachchi I, Alvarado LA. Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method. Statistics in medicine. 2017;36(14):2187-205.\u003c/li\u003e\n\u003cli\u003eGibbons JD, Chakraborti S. Comparisons of the Mann-Whitney, Student\u0026rsquo;st, and alternate t tests for means of normal distributions. The Journal of Experimental Education. 1991;59(3):258-67.\u003c/li\u003e\n\u003cli\u003eGerald B, Patson TF. Parametric and nonparametric tests: A brief review. Int J Stat Distrib Appl. 2021;7:78-82.\u003c/li\u003e\n\u003cli\u003eNielsen EE, N\u0026oslash;rskov AK, Lange T, \u003cem\u003eet al\u003c/em\u003e. Assessing assumptions for statistical analyses in randomised clinical trials. BMJ evidence-based medicine. 2019;24(5):185-9.\u003c/li\u003e\n\u003cli\u003eSchulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Journal of Pharmacology and pharmacotherapeutics. 2010;1(2):100-7.\u003c/li\u003e\n\u003cli\u003eVrbin CM. Parametric or nonparametric statistical tests: Considerations when choosing the most appropriate option for your data. Cytopathology. 2022;33(6):663-7.\u003c/li\u003e\n\u003cli\u003eKim TK. Practical statistics in pain research. The Korean journal of pain. 2017;30(4):243.\u003c/li\u003e\n\u003cli\u003eCalver M, Fletcher D. When ANOVA isn\u0026apos;t ideal: Analyzing ordinal data from practical work in biology. The American Biology Teacher. 2020;82(5):289-94.\u003c/li\u003e\n\u003cli\u003eLiddell TM, Kruschke JK. Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology. 2018;79:328-48.\u003c/li\u003e\n\u003cli\u003eChangyong F, Hongyue W, Naiji L, \u003cem\u003eet al\u003c/em\u003e. Log-transformation and its implications for data analysis. Shanghai archives of psychiatry. 2014;26(2):105.\u003c/li\u003e\n\u003cli\u003eErceg-Hurn DM, Mirosevich VM. Modern robust statistical methods: an easy way to maximize the accuracy and power of your research. Am Psychol. 2008;63(7):591-601.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Clinical Trial, Phase III, Sample Size, Statistics, Nonparametric","lastPublishedDoi":"10.21203/rs.3.rs-4447330/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4447330/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eIn studies with small sample sizes, the use of nonparametric methods is generally recommended for statistical analysis. However, various studies continue to employ parametric analysis without verifying the assumption of a normal distribution.\u003c/p\u003e\u003ch2\u003eObjectives\u003c/h2\u003e \u003cp\u003eTo assess the current utilization of parametric and nonparametric methods for primary outcomes, as well as the reporting of normality assumptions, in phase 3 clinical trials with small sample sizes.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eAll phase 3 trials registered on ClinicalTrials.gov until September 12, 2023, were collected. After undergoing a two-step selection process, only publications with a sample size per group of less than 30, involving two or more groups, and specifying the statistical methods used to compare the means or medians of primary outcomes between groups were selected. Statistical methods were categorized as nonparametric, parametric, and either parametric or nonparametric. The reporting of normality assumptions was also evaluated.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eA total of 317 studies were assessed in this study. Among these studies, 164 (51.7%) studies conducted parametric analysis, and 111 (35.0%) studies employed nonparametric analysis; however, 42 (13.2%) studies conducted parametric or nonparametric analysis without specifying which method was used. In addition, 63.1% of the total studies did not report normality assumptions. Specifically, within the subset of studies with parametric analysis, 70.1% of studies did not report normality assumptions.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThis research demonstrated that most studies with small sample sizes employed parametric analysis without reporting normality assumptions. The findings emphasize the need for increased awareness of and compliance with statistical principles in the analysis of phase 3 clinical trials with limited sample sizes.\u003c/p\u003e","manuscriptTitle":"Parametric analysis and normality assumption in phase 3 trials with small sample sizes","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-03 23:16:48","doi":"10.21203/rs.3.rs-4447330/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4cf28a66-94bb-4c65-b738-ea69e98ae7bb","owner":[],"postedDate":"June 3rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-01-30T12:08:34+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-03 23:16:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4447330","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4447330","identity":"rs-4447330","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}