Improved beta-binomial estimation for reliability of healthcare quality measures

Background The popular beta-binomial approach to estimate the reliability of healthcare quality measures (Adams et al. 2010 New England Journal of Medicine ) may yield grossly over-estimated reliabilities for providers with event rates equal to 0% or 100%. Objective Improve the beta-binomial approach to yield more reasonable reliability estimates for providers with event rates equal to 0% or 100%. Method We revise the beta-binomial approach by substituting Bayesian estimates with various priors for the crude event rates. We evaluate the new reliability estimates using Monte Carlo studies and two real-world measure examples. Results and conclusion The revised beta-binomial approach based on Jeffreys non-informative prior yields more reasonable reliability estimates for providers with event rates equal to 0% or 100% and statistically outperforms the original beta-binomial approach regarding bias and standard errors.