Examining the Feasibility and Functionality of Common Analysis Methods for Longitudinal Data with Binary Outcomes

Background: Previous literature often indicates that binary longitudinal data, such as responder analysis with patient-reported outcomes to evaluate treatment risks and benefits in cancer randomized controlled trials, need complex statistical approaches for a thorough evaluation of the treatment effect and consequently elaborated clinical design and power calculation for prospective trials. This simulated analysis aims to evaluate six common statistical techniques used for binary longitudinal data to explore the feasibility and overall functionality of these methods in comparison to more complex modeling techniques. Methods We simulated data to represent a typical clinical trial design, with dichotomous outcome, and treatment vs control group at three waves T1, T2, and T3. We evaluated five sample size conditions ranging from small to large with 1,000 replicates each. We then evaluated results from six statistical techniques (Generalized linear model [GLM] with generalized estimating equations [GEE], Generalized linear mixed model [GLMM], Logistic regression, Cochran-Mantel-Haenszel [CMH], Chi-square, and Fisher’s Exact test) and their ability in detecting difference between treatment groups under different conditions. Results While less elaborated models always achieved convergence, the convergence rate of GLMM with unstructured covariance matrix ranged from 7–43%. The proportion of detected significant differences between control and treatment expectedly increased with the increasing of the sample size but remained somewhat similar across modeled outcome and sample size scenarios. Conclusions Overall results indicated that, with careful consideration, straightforward modelling approaches which require less assumptions in design and power calculation are sufficient to ensure meaningful, adequately powered, evaluation of treatment effect.