Lessons for Covid-19 Vaccination from Eight Federal Government Direct Communication Evaluations

We discuss eight randomized evaluations intended to increase vaccination uptake conducted by the US General Services Administration’s Office of Evaluation Sciences (OES). These evaluations had a median sample size of 55,000, deployed a variety of behaviorally-informed direct communications, and used administrative data to measure vaccination uptake. The confidence interval from an internal meta-analysis shows changes in vaccination rates ranging from -0.004 to 0.394 percentage points. Two studies yielded statistically significant increases, of 0.59 and 0.16 percentage points. The other six were not statistically significant, although the studies were powered to detect effect sizes in line with published research. This work highlights the likely effects of government communications and demonstrates the value of conducting rapid evaluations to support COVID-19 vaccination efforts.