Bias-adjusted predictions of county-level vaccination coverage from the COVID-19 Trends and Impact Survey

Word Count: 185 19 Manuscript Word Count: 3579 20 Number of Figures/Tables: 7 21 22

COVID-19, vaccination, population health measurement, surveys 23 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice. 2

24 25 The potential for bias in non-representative, large-scale, low-cost survey data can limit their 26 utility for population health measurement and public health decision-making. We developed a 27 multi-step regression framework to bias-adjust vaccination coverage predictions from the large-28 scale US COVID-19 Trends and Impact Survey that included post-stratification to the American 29 Community Survey and secondary normalization to an unbiased reference indicator. As a case 30 study, we applied this framework to generate county-level predictions of long-run vaccination 31 coverage among children ages 5 to 11 years. Our vaccination coverage predictions suggest a 32 low ceiling on long-term national coverage (46%), detect substantial geographic heterogeneity 33 (ranging from 11% to 91% across counties in the US), and highlight widespread disparities in the 34 pace of scale-up in the three months following Emergency Use Authorization of COVID-19 35 vaccination for 5 to 11 year-olds. Generally, our analysis demonstrates an approach to leverage 36 differing strengths of multiple sources of information to produce estimates on the time-scale 37 and geographic-scale necessary for proactive decision-making. The utility of large-scale, low-38 cost survey data for improving population health measurement is amplified when these data 39 are combined with other representative sources of data. 40 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 3

41 42 The COVID-19 pandemic highlighted the importance of local and timely indicators to inform 43 public health decision-making, but such indicators have remained elusive in areas critical to 44 pandemic response. For example, indicators of people’s vaccination intentions could ideally be 45 used to predict subsequent vaccine uptake and to drive targeted efforts to reduce hesitancy 46 and thereby increase achieved coverage. However, representative survey data are too costly to 47 collect repeatedly with samples large enough for county-level estimation in the United States, 48 while unrepresentative large-scale survey data have been shown to yield biased estimates with 49 misleadingly small margins of error.1–3 Although programmatic data offer retrospective 50 reporting of coverage at county-level, these data become available too late to enable 51 prospective planning and decision-making, and many important indicators do not have routine 52 reporting systems.1,4 53 54 Combining data sources with different advantages and limitations can help to balance tradeoffs 55 between time, cost, and representativeness of data collection. Studies in other areas of health 56 have combined multiple data sources for retrospective bias correction and small area 57 estimation.5–8 The COVID-19 pandemic catalyzed a new era of massive real-time data collection 58 for public health, exemplified by the US COVID-19 Trends and Impact Survey, which has been 59 running daily since April 2020.2 The US survey has an average of 40,000 responses daily. Its size 60 allows for timely small-area estimation of many policy-relevant leading indicators, but its utility 61 has been questioned due to bias in estimates of vaccination coverage compared to 62 representative reporting data.3 Approaches to gain actionable insights from these large-scale 63 survey data are relevant to current COVID-19 pandemic response, and to general population 64 health measurement, for which similar large-scale, low-cost surveys could be deployed in the 65 future. 66 67 Although COVID-19 vaccination has been central to the public health response to the pandemic, 68 coverage has plateaued well below 100%, with wide variation across communities. COVID-19 69 vaccination intentions have been an important indicator derived from survey data over the 70 course of the pandemic.5,9–13 Vaccination intentions can be used to anticipate eventual 71 vaccination coverage for different groups, which can then be used to direct resources and 72 targeted interventions, design policies, deploy additional risk reduction tools, and monitor both 73 the pace and equity of scale-up. In the United States, children ages 5 to 11 years became the 74 most recently eligible group for COVID-19 vaccination when Emergency Use Authorization was 75 extended at the end of October 2021.14 In this study, we present a framework to bias-adjust 76 estimates of vaccine intentions from the large-scale COVID-19 Trends and Impact Survey and 77 predict future county-level vaccination coverage plateaus, using vaccination among children 78 ages 5 to 11 years as an illustrative case study. 79 80

81 82 We developed a multi-step regression framework (Figure 1) to predict vaccination coverage 83 plateaus among children ages 5 to 11 years. First, we estimated county-level parental hesitancy 84 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 4 toward vaccinating their children using a mixed effects logistic regression model fit to survey 85 data. Next, we estimated the relationship between county-level hesitancy and observed 86 vaccination coverage, for youth ages 12 to 17 years, who became eligible for COVID-19 87 vaccination earlier than children ages 5 to 11 and therefore provide a reference group, using a 88 second logistic regression model. Finally, we combined the results from the two regression 89 models to predict county-level vaccination coverage for 5- to 11-year-olds. 90 91 Data Sources 92 We combined individual-level survey responses from Wave 11 of the COVID-19 Trends and 93 Impact Survey (CTIS) collected during the period from July 1, 2021, through October 31, 2021, 94 with data from Wave 12, collected during the period from December 19, 2021, through 95 February 14, 2022. The survey is managed and implemented by the Delphi Group at Carnegie 96 Mellon University. Participants are recruited through Facebook and the sampling frame is the 97 Facebook Active User base. Additional information on the COVID-19 Trends and Impact Survey 98 has been previously published.2 The full questionnaire for Waves 11 and 12 is available online.15 99 100 In addition to CTIS, we used individual-level sociodemographic data (age, documented sex, 101 education, race/ethnicity, and household structure) from the 2015-2019 American Community 102 Survey for post-stratification.16 Individual-level data from the American Community Survey are 103 available at the public-use microdata area level. We mapped public-use microdata areas to 104 counties using the Missouri Census Data Center’s Geographic Correspondence Engine.17 When 105 a single county contained multiple public-use microdata areas, we aggregated public-use 106 microdata areas to the county-level. When a single public-use microdata area spanned multiple 107 counties, we assumed the same distribution of sociodemographic characteristics for each 108 county. 109 110 Finally, we used complete vaccination coverage data for ages 12 to 17 years reported at the 111 county-level by the Centers for Disease Control and Prevention for the second-stage regression, 112 and data from the same source over the first three months after eligibility for ages 5 to 11 years 113 for performance evaluation of coverage predictions.18 114 115 Estimating County-Level Hesitancy 116 To estimate county-level parental hesitancy, we fit a mixed-effects logistic regression to survey 117 data on attitudes of parents/guardians towards vaccinating their children. We classified “No, 118 definitely not” and “No, probably not” as hesitant responses to the question “Will you choose 119 to get a COVID-19 vaccine for your child or children when they are eligible?”. Responses of “Yes, 120 definitely” and “Yes, probably” were considered not hesitant. Consistent with previous 121 analyses, we used the imprecise but available construct of “reported hesitancy” and focused on 122 it principally as an intermediate indicator that would be subsequently mapped to long-run 123 vaccination coverage. 124 125 The CTIS survey questionnaire evolved as new information became available over the course of 126 the pandemic. Importantly, while Wave 11 asked parents about vaccine hesitancy, it did not ask 127 for the ages of their children. Since Wave 12 elicited the age of the parent’s oldest child, we 128 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 5 used it to examine differences in parental hesitancy for those whose oldest child was between 129 the ages of 12 and 17 versus ages 5 to 11. 130 131 The first-stage logistic regression modeled the probability of parental hesitancy as a function of 132 fixed effects for documented gender (male, female), age group (18-24, 25-34, 35-44, 45-54, 55-133 64, 65+), education (high school or fewer years of education, some college or a two-year 134 degree, four-year degree, graduate degree), and race/ethnicity (Hispanic, non-Hispanic 135 American Indian or Alaska Native, non-Hispanic Asian, non-Hispanic Black, non-Hispanic Native 136 Hawaiian or Other Pacific Islander, non-Hispanic White, non-Hispanic multiracial or other race), 137 and age group of child (unknown, 12 to 17, and 5 to 11), and nested random intercepts on state 138 and county: 139 140 ln( 𝑝𝑖𝑗𝑘 1 − 𝑝𝑖𝑗𝑘 ) = 𝛼𝑗𝑘 + 𝛽𝑋𝑖𝑗𝑘 141 𝛼𝑗𝑘 = 𝛼𝑘 + 𝜇𝑗𝑘; 𝜇𝑗𝑘~𝑁(0, 𝜎𝜇𝑗𝑘 2 ), 142 𝛼𝑘~𝑁(0, 𝜎𝛼𝑘 2 ), 143 𝑖 = individual CTIS responses from parents/guardians; 𝑗 = counties; 𝑘 = states 144 145 We did not perform a weighted regression to include the CTIS survey weights, instead adjusting 146 for the probability of inclusion and non-response through post-stratification.19 We combined 147 data from counties with a sample size of 10 or fewer into grouped counties, by state. To 148 generate county-level predictions of hesitancy, including uncertainty around these predictions, 149 from the first-stage regression, we generated 1,000 draws of subgroup-level predicted 150 probabilities of hesitancy for unique combinations of documented gender, age group, 151 education, race/ethnicity, and county using the estimated regression coefficients, assuming a 152 multivariate normal distribution of the parameters including the fixed and random effects (𝑝̂𝑔𝑗), 153 where 𝑔 corresponds to each unique demographic characteristic combination. We then post-154 stratified county- and subgroup-level predicted probabilities of hesitancy to produce overall 155 county-level hesitancy estimates (𝜃̂𝑗): 156 157 𝜃̂𝑗 = ∑ 𝑤𝑔𝑗𝑝̂𝑔𝑗 ∑ 𝑤𝑔𝑗 . 158 159 The weights (𝑤𝑔𝑗) used for post-stratification were based on an analysis of individual-level data 160 from the American Community Survey that reflected household structure and incorporated 161 children’s sample weights. First, we identified each child’s parents/guardians based on the first 162 available of the following: 1) parents directly coded in the American Community Survey, 2) 163 grandparents designated as responsible for one or more children directly coded in the 164 American Community Survey, 3) adults (18+) in the same household and same family unit, and 165 4) adults (18+) in the same household but different family unit. Next, we assigned the child’s 166 sample weight to each of their parents/guardians, dividing the weight by the total number of 167 identified parents/guardians. Finally, we summed the children’s sample weights across each 168 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 6 subgroup (𝑔), defined by the demographic characteristics of the parents/guardians, and each 169 county (𝑗), resulting in the final weight (𝑤𝑔𝑗) used for post-stratification. 170 171 Predicting County-Level Vaccination Coverage from Hesitancy Estimates 172 We used a second logistic regression model to translate county-level hesitancy estimates to 173 county-level vaccination coverage predictions. We trained the model on paired county-level 174 hesitancy and coverage estimates for children ages 12 to 17 years, and then projected the 175 predictive relationships onto the hesitancy estimates for children ages 5 to 11 years under the 176 assumption that the same relationships would apply across the two age groups. To estimate the 177 regression model for the 12 to 17 year group, we first generated estimates of parental 178 hesitancy for this group using the same regression model specification described above for the 179 5 to 11 year group, in this case predicting hesitancy for parents of children ages 12 to 17 years 180 and post-stratifying estimates based on household structure and sample weights of children 181 ages 12 to 17 years. These county-level hesitancy estimates were used as independent 182 variables in the second logistic regression. For our dependent variable, we used coverage data 183 from February 1, 2022, which was approximately nine months after 12-17 year-olds first 184 became eligible for vaccination (ages 16-17 years in early April 2021 and ages 12-15 years on 185 May 10, 2021). 186 187 For states with at least ten counties reporting vaccination coverage data for children ages 12 to 188 17 years on February 1, 2022, with CDC reporting completeness exceeding 80%, we fit state-189 specific regressions. For all other states and the District of Columbia (n=7), we fit regressions at 190 the census division level. This prevented overfitting to small numbers of counties or low-quality 191 reporting data. Regressions were weighted based on the size of the 12 to 17 population in each 192 county. The second regression was fit to each of the 1,000 draws of county-level hesitancy from 193 the first regression. Uncertainty from the second regression was captured through 1,000 draws 194 from the multivariate normal distribution of the fixed effects plus the residual variance. 195 196 Finally, we used the models fit on the relationship between parental hesitancy and vaccination 197 coverage for children ages 12 to 17 to predict coverage for children ages 5 to 11 years based on 198 our first-stage estimates of hesitancy for this age group. Final prediction intervals were based 199 on the 2.5th and 97.5th percentiles of one million final county-level coverage predictions (1,000 200 draws from the first regression crossed with 1,000 draws from the second regression). 201 202 Performance Evaluation 203 We compared our estimates of parental hesitancy towards vaccinating children ages 12 to 17 204 years to estimates on the same indicator produced by the Office of the Assistant Secretary for 205 Planning and Evaluation (ASPE), including comparing correlation coefficients between 206 estimated county-level hesitancy and observed vaccination coverage on February 1, 2022. 207 208 To evaluate our use of the relationship between hesitancy and coverage for children ages 12 to 209 17, applied to children ages 5 to 11, we assessed interim coverage predictions for the 5 to 11 210 age group at three months after EUA expansion against observed county-level coverage 211 reported by the CDC, based on the intraclass correlation coefficient and percentage of counties 212 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 7 for which the 95% prediction interval contained the observed coverage level nationally and at 213 the state level. Since the CDC does not report separate county-level coverage estimates for 214 ages 12 to 15 versus 16 to 17, the time since an age group first became eligible for vaccination 215 is an imprecise but best-available approach to this interim performance evaluation. 216 217 Monitoring Progress and Equity in Scale-Up 218 To monitor pace of vaccination scale-up, we defined a measure of “3-month progress” as: 219 220 progress = Observed three month coverage Predicted nine month coverage . 221 222 To monitor equity in the pace of vaccination scale-up, we used linear regression to analyze 223 associations between this progress measure and the county-level socioeconomic status domain 224 of the CDC’s Social Vulnerability Index, which reflects measures of poverty, unemployment, 225 income, and education.20 226 227 Study Approval and Data Availability 228 The study was approved by Stanford’s Institutional Review Board, under protocol number 229 56018. All analyses were conducted using the R programming language version 3.6.3. Analytic 230 code is available through GitHub (https://github.com/PPML/CTIS-County-Vaccination-231 Coverage). The COVID-19 Trends and Impact Survey microdata can be accessed through a data 232 use agreement with Carnegie Mellon University, while the American Community Survey data 233 and CDC vaccination data are publicly available. 234 235

236 237 Data 238 Between July 1 and October 31, 2021, a total of 613,460 responses to Wave 11 of the US 239 COVID-19 Trends and Impact Survey (CTIS) were collected from parents/guardians of children 240 under age 18 with complete demographic information. To allow for variation in parental 241 hesitancy by child age group, we supplemented the analyses with 119,465 responses collected 242 from parents/guardians whose oldest children were between the ages of 5 and 17 years in 243 Wave 12, between December 19, 2021 and February 14, 2022. We report exclusions in the 244 sample flowchart (Supplemental Figure S1). Unweighted and weighted distributions of 245 respondents by age, documented gender, education, and race/ethnicity are reported in Table 1. 246 Post-stratification to the American Community Survey reduced bias from the non-247 representative sample. Of 3,142 counties, 1,203 had a sample size of at least 100, while 293 248 had a sample size between one and 10, and 23 had zero respondents. Maps of county-level 249 sample sizes and sample rates are reported in Supplemental Figures S2 and S3. 250 251 Hesitancy Estimates 252 Modeled county-level parental hesitancy toward vaccination for children ages 5 to 11 years 253 ranged from 7% (95% Prediction Interval: 5-9%) in San Mateo County, California to 74% (61-254 84%) in Platte County, Wyoming. Although the population-weighted national average hesitancy 255 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 8 was 31%, 2,787 counties (89% of all counties) had hesitancy levels exceeding this benchmark. 256 The skewed distribution of county-level estimates versus state- and national-aggregates is 257 largely driven by lower hesitancy in urban areas with large populations and higher hesitancy in 258 rural areas with smaller populations. Across counties, median hesitancy towards vaccination 259 was 21% (IQR: 18-25%) higher for parents of children ages 5 to 11, compared to parents of 260 children ages 12 to 17. Our estimates of hesitancy among parents of children ages 12 to 17 261 using data from CTIS reflected substantially more sub-state variation in hesitancy, compared to 262 previously published estimates from ASPE (Figure 2). Additionally, our estimates showed a 263 stronger correlation with vaccination coverage on February 1, 2022, among children ages 12 to 264 17 years (CTIS: -0.78; ASPE: -0.44) (Supplemental Figure S4). 265 266 Predicted Coverage Levels 267 Predicted mean national plateau coverage level among children ages 5 to 11 by August 2022, 268 nine months after EUA, was 46%. There was substantial state-level variation in predicted 269 plateau coverage, ranging from 30% and below in Wyoming, Alabama, Mississippi, Idaho, and 270 Louisiana to 66% and above in Connecticut, District of Columbia, and Massachusetts. Four out 271 of the five counties with the highest predicted coverage were in California. Ninety-two percent 272 of counties are predicted to fall short of a 50% coverage benchmark by August 2022 for children 273 ages 5 to 11, while 56% of counties are predicted to not reach 30% coverage. Eighty-six percent 274 of counties are predicted to fall short of their state average coverage level, highlighting an 275 urban-rural divide in vaccination. Higher levels of predicted coverage are concentrated in the 276 northeast, west coast, and in urban centers across the country (Figure 3). 277 278 Model Validation 279 Figure 4 shows the relationship between predicted three-month coverage among children ages 280 5 to 11 years and observed coverage three months after EUA. The intraclass correlation 281 coefficient for consistency of predicted versus observed three-month coverage at national level 282 was 0.81. Intraclass correlation coefficients were greater than 0.75 for 16 states, between 0.50 283 and 0.75 for 19 states, and less than 0.50 for 10 states. Intraclass correlation coefficients at 284 state level are reported in Supplemental Table S5. The prediction interval for three-month 285 coverage included the observed coverage level for 81% of counties. 286 287 Monitoring Progress and Equity in Scale-Up 288 Relative to long-term predicted coverage levels, at the state level, Vermont, Rhode Island, and 289 Maine had the fastest pace of scale-up of coverage among children ages 5 to 11 years at three 290 months after EUA, while Louisiana, Alabama, and Mississippi had the slowest pace of scale-up 291 (Figure 5). We find that errors in nine-month coverage predictions among ages 12 to 17 years 292 were not significantly associated with the socioeconomic status domain of the CDC’s Social 293 Vulnerability Index (SVI) in all states except South Dakota, Nevada, and Montana. As a result, in 294 addition to predicting plateau coverage levels, we can use county-level predicted coverage 295 levels to monitor equity in the pace of vaccination scale-up. More vulnerable counties, as 296 measured by the socioeconomic status domain of the CDC’s SVI generally made less progress 297 toward reaching their plateau coverage levels over the first three months after EUA expansion, 298 compared to less vulnerable counties. There was significantly slower scale-up progress in more 299 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 9 vulnerable (higher SVI) counties in 36 out of 46 states reporting data on vaccination coverage 300 among children ages 5 to 11 years (Figure 6). 301 302

303 304 We generated bias-adjusted county-level predictions of long-term vaccination coverage for 305 children ages 5 to 11 years that leveraged data on parental vaccination intentions from the 306 COVID-19 Trends and Impact Survey (CTIS). To mitigate the impacts of selection bias in the 307 sample, we combined CTIS data with representative sociodemographic data from the American 308 Community Survey and unbiased programmatic data on vaccination coverage. Our approach to 309 estimation and propagation of multiple sources of uncertainty produced prediction intervals 310 that included observed coverage levels for 81% of counties three-months after EUA. Our 311 estimation framework can be broadly used to generate actionable indicators on the time-scale 312 and at the geographic-scale required for decision-making during the pandemic and beyond. 313 314 Our predictions for vaccination coverage nine months after EUA for children ages 5 to 11 years 315 suggest that 92% counties are likely to fall short of a 50% coverage benchmark. Across and 316 Within states, there is substantial geographic heterogeneity in both parental hesitancy and 317 predicted coverage. These estimates have implications for targeting of efforts to promote 318 vaccination uptake among eligible children and expectations for eventual vaccination uptake 319 among younger children who are not yet eligible. They also underscore the continued need for 320 other protective measures, including masking, testing, and improved ventilation, in schools 321 during periods of significant community transmission.21,22 322 323 Despite consistent messaging about the importance of promoting equity in vaccination scale-324 up, we observe a pervasive pattern of slower vaccination scale-up in more vulnerable counties, 325 as measured by the socioeconomic domain of the CDC’s Social Vulnerability Index.23–25 The 326 socioeconomic status domain of the Social Vulnerability Index comprises measures of income, 327 poverty, employment, and education. Moving forward, as vaccination is extended to even 328 younger children and as new rounds of boosters or new vaccines are authorized, more 329 intensive and explicitly pro-equity policies and programs are required to break the cycle of 330 inequity in vaccination scale-up that has been repeated in every phase of the vaccination 331 campaign.26–29 332 333 Large-scale, low-cost surveys offer a promising approach to population health measurement. 334 They offer advantages for rapid and continuous deployment and allow estimation at smaller 335 geographic scales compared to traditional approaches to data collection, including 336 representative household surveys and retrospective reporting data. These advantages of 337 county-level data in CTIS, compared to the state-level data available from the Census 338 Household Pulse Survey are evident in their respective performance in capturing sub-state 339 heterogeneity in vaccination intentions and coverage. The correlation between parental 340 hesitancy and coverage among children ages 12 to 17 years for CTIS was -0.78, compared to a 341 correlation coefficient of -0.44 for ASPE estimates based on the Census Household Pulse 342 Survey.5 343 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 10 344 Beyond the COVID-19 pandemic, large-scale, low-cost surveys could be applied to regularly 345 generate and update estimates of county-level geographic heterogeneity in determinants of 346 health, healthcare access, and health outcomes. Designing integrated health measurement 347 systems that intentionally combine sources with different advantages across the spectrum of 348 timeliness, geographic granularity, and representativeness can maximize the benefits of data 349 collection relative to their costs. Future large-scale, low-cost data collection efforts should 350 ensure sufficient indicators are incorporated in the survey instrument for post-stratification, as 351 well as availability of appropriate reference indicators for secondary bias-adjustment. 352 353 The results of our study should be interpreted in the context of several limitations. First, to 354 predict plateau coverage for children ages 5 to 11 years we assumed that the relationship 355 between hesitancy and coverage observed for children ages 12 to 17 applies to this younger 356 age group. Our three-month validation supports this assumption, which is necessary for 357 prospective estimation. Second, estimates of hesitancy for children of different age groups only 358 became available in Wave 12 of the CTIS survey, and respondents are only asked about 359 intentions to vaccinate their oldest child. Third, we rely on historical relationships between 360 hesitancy and observed coverage, which will not capture the evolving COVID-19 policy and 361 epidemiologic landscape. Fourth, our analytic framework is designed to capture geographic 362 variation in coverage but not variation by other important population characteristics such as 363 race/ethnicity within small geographic areas. Despite these limitations, our estimates reflect a 364 principled approach to generating bias-adjusted estimates of vaccination coverage that can be 365 used to inform decisions and evaluate actual progress against a reference scenario. 366 367

368 369 A combination of post-stratification and secondary normalization to an unbiased reference can 370 reduce bias in large-scale, low-cost survey data. Applying this method to predict long-term 371 county-level COVID-19 vaccination coverage among children ages 5 to 11 years, we find 372 substantial sub-state geographic heterogeneity and disparities in the pace of scale-up. Although 373 direct estimates of vaccination coverage from the COVID-19 Trends and Impact Survey are 374 biased, a multi-step regression strategy can result in bias-adjusted actionable predictions on the 375 time-scale and geographic-scale required for proactive decision-making in the pandemic. 376 377 378 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 11 Data Availability 379 Survey microdata are not publicly available because survey participants only consented to 380 public disclosure of aggregate data, and because the legal agreement with Facebook governing 381 operation of the survey prohibits disclosure of microdata without confidentiality protections for 382 respondents. Deidentified microdata are available to researchers under a Data Use Agreement 383 that protects the confidentiality of respondents. Access can be requested online (https://cmu-384 delphi.github.io/delphi-epidata/symptom-survey/data-access.html). Requests are reviewed by 385 the Carnegie Mellon University Office of Sponsored Programs and Facebook Data for Good. 386 387

388 We would like to thank the Delphi Group at Carnegie Mellon University for their role in 389 managing the COVID-19 Trends and Impact Survey. We appreciate feedback from members of 390 the SC-COSMO group (https://sc-cosmo.org/) and Prevention Policy Modeling Lab 391 (https://ppml.stanford.edu/) on this work. 392 393 Funding/Support 394 MBR is supported by the National Science Foundation Graduate Research Fellowship Program 395 under Grant No. (DGE-1656518), Stanford’s Knight-Hennessy Scholars Program, and the 396 Stanford Data Science Scholars Program. MBR, JDGF, and JAS are supported by the Stanford 397 Clinical and Translational Science Award to Spectrum (UL1TR003142). JDGF, SR, and JAS are 398 supported by funding from the Centers for Disease Control and Prevention and the Council of 399 State and Territorial Epidemiologists (NU38OT000297) and by funding from the Health Equity 400 Research Project Fund from Stanford’s School of Medicine. JDGF and JAS are supported by 401 funding from the National Institute on Drug Abuse (3R37DA01561217S1). AR is supported by an 402 unrestricted gift from Facebook. Facebook was involved in the design and conduct of the study. 403 No funders had a role in the analysis and interpretation of the data, writing of the manuscript, 404 or the decision to submit the manuscript for publication. 405 406 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 12 Main Figures and Tables 407 408 Table 1. Unweighted and weighted distribution of sociodemographic variables of included 409 guardians from the COVID-19 Trends and Impact Survey (CTIS), compared to distribution of 410 sociodemographic variables of parents/guardians in the American Community Survey (ACS) 411 412 CTIS Survey Data: Unweighted CTIS Survey Data: Weighted Parents/Guardians of Children Ages 12-17 Parents/Guardians of Children Ages 5-11 Documented Gender* Female 65.4% 53.9% 55.4% 55.4% Male 34.6% 46.1% 44.6% 44.6% Age Group 18-24 1.9% 5.8% 0.7% 1.4% 25-34 13.7% 17.7% 8.4% 29.6% 35-44 29.0% 28.3% 42.1% 48.7% 45-54 25.6% 24.4% 39.3% 16.8% 55-64 16.0% 13.6% 7.9% 2.5% 65+ 13.7% 10.2% 1.6% 1.0% Education High school or less 21.7% 25.1% 37.1% 35.9% Some college or two-year degree 35.9% 35.9% 29.7% 29.8% Four-year degree 23.2% 21.7% 20.4% 20.7% Graduate degree 19.2% 17.3% 12.8% 13.6% Race/Ethnicity American Indian or Alaska Native 1.1% 1.0% 0.7% 0.7% Asian 2.8% 3.4% 5.4% 5.9% Black 7.1% 7.2% 11.5% 11.5% Hispanic 17.3% 21.7% 22.0% 23.0% Native Hawaiian or Other Pacific Islander 0.3% 0.3% 0.2% 0.2% Other 4.8% 5.2% 1.8% 2.0% White 66.6% 61.2% 58.4% 56.7% 413 *CTIS collects information on the respondent’s self-reported gender, while the ACS collects information on the 414 respondent’s self-reported sex. 415 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 1 Figure 1. Methods Flowchart. 416 417 418 419 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 2 Figure 2. Comparison of county-level hesitancy estimates for parents of children ages 12-17 produced by CTIS (left) and ASPE 420 (right). 421 422 423 424 425 426 427 428 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 3 Figure 3. County-level map of predicted plateau complete vaccination coverage levels by August 2022 for children ages 5-11 years. 429 The color scale is split at the national average predicted coverage of 46%. 430 431 432 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 4 Figure 4. Three-month validation of county-level predicted versus observed complete coverage among children ages 5 to 11 years. 433 434 435 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 5 Figure 5. State-level three-month complete vaccination scale-up progress for children ages 5-11 years, and nine-month predicted 436 coverage. 437 438 439 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 6 Figure 6. Association between three-month county-level complete vaccination progress for children ages 5-11 years and the 440 socioeconomic status domain of the CDC’s Social Vulnerability Index. The intensity of the trend lines is proportional to the linear 441 regression R2. 442 443 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 1

444 445 1 Kreuter F, Barkay N, Bilinski A, et al. Partnering with a global platform to inform research and 446 public policy making: Survey Research Methods 2020; 14: 159–63. 447 2 Salomon JA, Reinhart A, Bilinski A, et al. The US COVID-19 Trends and Impact Survey: 448 Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, 449 testing, and vaccination. Proceedings of the National Academy of Sciences 2021; 118: 450 e2111454118. 451 3 Bradley VC, Kuriwaki S, Isakov M, Sejdinovic D, Meng X-L, Flaxman S. Unrepresentative big 452 surveys significantly overestimated US vaccine uptake. Nature 2021; 600: 695–700. 453 4 Rosenfeld R, Tibshirani RJ. Epidemic tracking and forecasting: Lessons learned from a 454 tumultuous year. Proceedings of the National Academy of Sciences 2021; 118: e2111456118. 455 5 Parents’ Intentions to Vaccinate Children Ages 12-17 for COVID-19: Demographic Factors, 456 Geographic Patterns, and Reasons for Hesitancy. ASPE. 457 https://aspe.hhs.gov/reports/hesitancy-vaccinate-children (accessed March 16, 2022). 458 6 Ward ZJ, Long MW, Resch SC, et al. Redrawing the US Obesity Landscape: Bias-Corrected 459 Estimates of State-Specific Adult Obesity Prevalence. PLOS ONE 2016; 11: e0150735. 460 7 Zhang X, Holt JB, Yun S, Lu H, Greenlund KJ, Croft JB. Validation of Multilevel Regression and 461 Poststratification Methodology for Small Area Estimation of Health Indicators From the 462 Behavioral Risk Factor Surveillance System. American Journal of Epidemiology 2015; 182: 463 127–37. 464 8 Elliott MR, Davis WW. Obtaining cancer risk factor prevalence estimates in small areas: 465 combining data from two surveys. Journal of the Royal Statistical Society: Series C (Applied 466 Statistics) 2005; 54: 595–609. 467 9 Daly M, Jones A, Robinson E. Public Trust and Willingness to Vaccinate Against COVID-19 in 468 the US From October 14, 2020, to March 29, 2021. JAMA 2021; 325: 2397–9. 469 10 KFF COVID-19 Vaccine Monitor Dashboard. KFF. 2022; published online March 1. 470 https://www.kff.org/coronavirus-covid-19/dashboard/kff-covid-19-vaccine-monitor-471 dashboard/ (accessed March 16, 2022). 472 11 Vaccine Hesitancy for COVID-19: State, County, and Local Estimates. ASPE. 473 https://aspe.hhs.gov/index.php/reports/vaccine-hesitancy-covid-19-state-county-local-474 estimates (accessed Sept 30, 2021). 475 12 IHME | COVID-19 Vaccine Hesitancy. Institute for Health Metrics and Evaluation. 476 https://vaccine-hesitancy.healthdata.org/ (accessed March 16, 2022). 477 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 2 13 SteelFisher GK, Blendon RJ, Caporello H. An Uncertain Public — Encouraging Acceptance of 478 Covid-19 Vaccines. New England Journal of Medicine 2021; 384: 1483–7. 479 14 Commissioner O of the. FDA Authorizes Pfizer-BioNTech COVID-19 Vaccine for Emergency 480 Use in Children 5 through 11 Years of Age. FDA. 2021; published online Oct 29. 481 https://www.fda.gov/news-events/press-announcements/fda-authorizes-pfizer-biontech-482 covid-19-vaccine-emergency-use-children-5-through-11-years-age (accessed April 7, 2022). 483 15 Questions and Coding. Delphi Epidata API. https://cmu-delphi.github.io/delphi-484 epidata/symptom-survey/coding.html (accessed March 16, 2022). 485 16 Ruggles S, Flood S, Foster S, et al. IPUMS USA: Version 11.0 2015-2019 American Community 486 Survey. Minneapolis, MN: IPUMS, 2021 https://doi.org/10.18128/D010.V11.0. 487 17 Geocorr 2018 - MCDC. https://mcdc.missouri.edu/applications/geocorr2018.html (accessed 488 March 16, 2022). 489 18 COVID-19 Vaccinations in the United States,County | Data | Centers for Disease Control and 490 Prevention. https://data.cdc.gov/Vaccinations/COVID-19-Vaccinations-in-the-United-States-491 County/8xkx-amqh (accessed March 16, 2022). 492 19 Gelman A. Struggles with Survey Weighting and Regression Modeling. Statistical Science 493 2007; 22: 153–64. 494 20 CDC SVI Documentation 2018 | Place and Health | ATSDR. 2022; published online Feb 10. 495 https://www.atsdr.cdc.gov/placeandhealth/svi/documentation/SVI_documentation_2018.ht496 ml (accessed March 16, 2022). 497 21 CDC. Guidance for COVID-19 Prevention in K-12 Schools and ECE Programs. Centers for 498 Disease Control and Prevention. 2022; published online Jan 13. 499 https://www.cdc.gov/coronavirus/2019-ncov/community/schools-childcare/k-12-500 guidance.html (accessed April 7, 2022). 501 22 Giardina J, Bilinski A, Fitzpatrick MC, et al. Model-Estimated Association Between Simulated 502 US Elementary School–Related SARS-CoV-2 Transmission, Mitigation Interventions, and 503 Vaccine Coverage Across Local Incidence Levels. JAMA Network Open 2022; 5: e2147827. 504 23 National COVID-19 Preparedness Plan. The White House. 505 https://www.whitehouse.gov/covidplan/ (accessed April 7, 2022). 506 24 Equity in Childhood COVID-19 Vaccination. 2021; published online Dec 29. 507 https://www.cdc.gov/vaccines/covid-19/planning/children/equity.html (accessed April 7, 508 2022). 509 25 Ndugga N, Hill L, Artiga S, Haldar S. Latest Data on COVID-19 Vaccinations by Race/Ethnicity. 510 Kaiser Family Foundation. 2022; published online March 9. https://www.kff.org/coronavirus-511 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint 3 covid-19/issue-brief/latest-data-on-covid-19-vaccinations-by-race-ethnicity/ (accessed April 512 7, 2022). 513 26 Faherty LJ, Ringel JS, Williams MV, et al. The U.S. Equity-First Vaccination Initiative: Early 514 Insights. RAND Corporation, 2022 https://www.rand.org/pubs/research_reports/RRA1627-515 1.html (accessed April 7, 2022). 516 27 Wrigley-Field E, Kiang MV, Riley AR, et al. Geographically targeted COVID-19 vaccination is 517 more equitable and averts more deaths than age-based thresholds alone. Science Advances; 518 7: eabj2099. 519 28 Dada D, Djiometio JN, McFadden SM, et al. Strategies That Promote Equity in COVID-19 520 Vaccine Uptake for Black Communities: a Review. J Urban Health 2022; 99: 15–27. 521 29 Quinn SC, Andrasik MP. Addressing Vaccine Hesitancy in BIPOC Communities — Toward 522 Trustworthiness, Partnership, and Reciprocity. New England Journal of Medicine 2021; 385: 523 97–100. 524 525 . CC-BY-NC-ND 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 21, 2022. ; https://doi.org/10.1101/2022.05.18.22275217doi: medRxiv preprint