Validity of ANOVA under Non-normality & Heterogeneity

Abstract Background Performance of the F-test critically depends on the distributional assumptions and the choice of the robustness measures. The robustness of F-test to non-normality has been studied and validated in literature however, the assumption of homogeneity has not been given the due attention in literature. Regarding robustness measures, F-test is not evaluated against both the liberal and conservative criterion simultaneously. This study provides a systematic examination of F-test robustness to violations of normality and homogeneity in terms of Type I error, considering a wide variety of distributions commonly found in psychology, social and medical sciences. Method This study conducted Monte Carlo simulations to compute the Type-I error rates of F-test under non-normality and heterogeneity assumption. To assess the Type-I error rates, 100,000 samples were generated with 1900 scenarios to ensure reliable results. The manipulated parameters include shape and scale parameters of distributions, number of groups, equal and unequal sample sizes, total and average sample size, inequality in the sample size and variance ratio. Results The findings of this study show that the robustness of the F-test in terms of Type-I error rates depend on the choice of robustness measure, variance ratio, sample size, and equality of samples. The F-test is robust under a threshold value of variance ratio when evaluated against liberal criteria and non-robust against conservative criterion. MSC Classification: 91-10, 62J10, 65C20