Nowcasting and Forecasting the 2022 U.S. Mpox Outbreak: Support for Public Health Decision Making and Lessons Learned

monkeypox virus, modeling, epidemic, real-time analysis, situational awareness 26 27 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.

28 In June of 2022, the U.S. Centers for Disease Control and Prevention (CDC) Mpox Response wanted 29 timely answers to important epidemiological questions which can now be answered more effectively 30 through infectious disease modeling. Infectious disease models have shown to be valuable tool for 31 decision making during outbreaks; however, model complexity often makes communicating the results 32 and limitations of models to decision makers difficult. We performed nowcasting and forecasting for the 33 2022 mpox outbreak in the United States using the R package EpiNow2. We generated 34 nowcasts/forecasts at the national level, by Census region, and for jurisdictions reporting the greatest 35 number of mpox cases. Modeling results were shared for situational awareness within the CDC Mpox 36 Response and publicly on the CDC website. We retrospectively evaluated forecast predictions at four key 37 phases during the outbreak using three metrics, the weighted interval score, mean absolute error, and 38 prediction interval coverage. We compared the performance of EpiNow2 with a naïve Bayesian 39 generalized linear model (GLM). The EpiNow2 model had less probabilistic error than the GLM during 40 every outbreak phase except for the early phase. We share our experiences with an existing tool for 41 nowcasting/forecasting and highlight areas of improvement for the development of future tools. We also 42 reflect on lessons learned regarding data quality issues and adapting modeling results for different 43 audiences. 44 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint

45 The 2022 mpox (formerly known as monkeypox) outbreak is the first major infectious disease 46 outbreak since the COVID-19 pandemic and was declared a Public Health Emergency of International 47 Concern by the World Health Organization on July 23, 2022 (1). As of April 13, 2023, a total of 86,956 48 confirmed cases have been reported in 110 countries and territories (2). Unlike COVID-19, mpox is a 49 disease known to be endemic in West and Central Africa for decades; it is caused by monkeypox virus 50 (MPXV), a zoonotic orthopoxvirus (3). Historically, classical symptoms involved fever, headache, muscle 51 aches, fatigue, lymphadenopathy, and rash (4). Human-to-human MPXV transmission occurs through 52 close contact with infectious material from skin lesions, respiratory secretions during prolonged face-to-53 face contact, and fomites, such as linens and bedding (5). The 2022 mpox outbreak began in May and 54 spread rapidly in non-endemic countries. This outbreak was characterized by human-to-human 55 transmission of MPXV through close physical contact (often associated with sexual activities) and has 56 disproportionately affected gay, bisexual, and other men who have sex with men (6). 57 During a public health crisis such as the mpox outbreak, difficult and rapid decisions with limited 58 available data are often required (7). Infectious disease models may assist with informing policy and 59 practice by predicting the magnitude and duration of an outbreak or epidemic, evaluating characteristics 60 of pathogen transmission such as transmissibility, and designing vaccination strategies, among others (8, 61 9). However, infectious disease models are often complex, integrating data from heterogenous sources 62 with many parameter assumptions that are subject to uncertainty. These aspects make it challenging to 63 effectively implement such models and communicate the results and potential limitations to decision 64 makers, other public health partners, and the general public (10, 11). 65 During the COVID-19 pandemic, the state of the art of outbreak analysis advanced considerably 66 (12). Methods and tools for estimating key epidemiological parameters, such as the effective reproduction 67 number, Rt, were developed and shared in real-time (13). Monitoring Rt, the average number of secondary 68 cases caused by a single infected individual in a large population, during an outbreak is useful for 69 assessing transmission dynamics and evaluating the effectiveness of public health measures (e.g., 70 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint vaccination, contact tracing, isolation, and quarantine) (14). Nowcasts and forecasts have been produced 71 by numerous research groups around the globe (15-18), the results of which were instrumental for 72 decision makers weighing possible control measures (19) such as social distancing measures. Outbreak 73 forecasting predicts specific outcomes (e.g., number of cases, deaths, or hospitalizations) at some specific 74 future times (e.g., weeks, months, etc.), whereas nowcasting estimates those outcomes for the current 75 time, accounting for delays in reporting. 76 In this manuscript, we share our experience nowcasting and forecasting the mpox outbreak, 77 including adapting the modeling output to different audiences. We also describe challenges faced vis-a-78 vis data quality, parameter estimation, and model application and propose ways to improve nowcasting 79 and short-term forecasting efforts for future outbreaks. 80

81 Nowcasting/forecasting the mpox outbreak 82 We used data on probable and confirmed mpox cases in the United States (see “Case definition” 83 in Supplementary methods) reported to CDC by state and local public health jurisdictions from May 17, 84 2022, through March 16, 2023. Data were submitted in several different formats throughout the outbreak 85 period. These formats included: a CDC-operated call center through the Emergency Operations Center 86 (EOC), a long and a short case report form (CRF), and the National Notifiable Diseases Surveillance 87 System (NNDSS). Cases could have data submitted via more than one format and jurisdictions could 88 update data on cases after initial submission (Supplementary methods). All reported data were processed 89 in CDC’s Data Collation and Integration for Public Health Event Response (DCIPHER) platform, an 90 instance of Palantir Foundry (Palantir Technologies Inc, Denver, CO). DCIPHER is a secure, cloud-based 91 data integration, analytics, and situational awareness platform used by the Centers for Disease Control 92 and Prevention (CDC), federal partners, and state, tribal, local, and territorial public health jurisdictions to 93 collect, collaborate on, and share public health data (20). DCIPHER collates data of differing origin, 94 structure, and purpose to provide near real-time insights into public health problems, with the goal of 95 providing a complete picture of situational awareness. 96 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint We considered three approaches for estimating Rt which are implemented in the R packages 97 EpiEstim (version 2.2-4) (21), earlyR (version 0.0.5) (22), and EpiNow2 (version 1.3.2) (23) 98 (Supplementary methods). Initially, we used all three methods to estimate Rt at the national level as well 99 as for jurisdictions reporting the highest incidence of mpox. Although estimates of the historical range of 100 the serial interval of mpox were available at the start of the outbreak, they were based on data from the 101 Democratic Republic of Congo, which reflected largely non-sexual household spread (24). We considered 102 these historical parameter estimates as a starting point for early outbreak analysis, using them (along with 103 sensitivity analyses) until new estimates were generated. Updated estimates characterized by the mean 104 and standard deviation were needed for the global outbreak given the novel mode of transmission. In June 105 2022, we were able to use an estimated mean serial interval (i.e., the period of time between symptom 106 onset in the primary case and symptom onset in the secondary case) of 9.8 days (95% credible interval 107 [CrI]: 5.9 – 21.4) from 17 case pairs reported by the United Kingdom (6). At that time, symptom onset 108 date was available for most reported cases, and imported cases were still contributing to a high proportion 109 of MPXV transmission. EpiEstim results were considered the most appropriate at this stage of the 110 outbreak because this method accounts for imported vs. locally acquired cases, has a stable codebase, is 111 widely used, and is computationally efficient (25). 112 In July 2022, we started exclusively using EpiNow2, which uses a similar approach as EpiEstim 113 (a branching process model, Supplementary methods), but it better accounts for reporting delays and 114 incorporates multiple sources of uncertainty (13); for example, it removes noise associated with weekend 115 effects and uses random walks for temporal smoothing. Forecasting is supported internally for Rt, number 116 of infections, cases by date of report, and growth rate. Unlike EpiEstim, EpiNow2 does not distinguish 117 between imported vs. locally acquired infections. EpiNow2 is the most computationally expensive 118 approach, requiring longer model run times (Supplementary methods). The model assumes that testing 119 procedures, surveillance effort, and reporting delays remain constant over the estimation period. To use 120 EpiNow2, cases by date of report must be provided as well as the generation time distribution (the time 121 between infection of a primary and secondary case), incubation period distribution (the time between 122 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint infection and symptom onset in a case), and any other delay distributions (e.g., the delay between 123 symptom onset and report date). The model estimates the number of new cases by date of report, number 124 of cases by their date of infection, Rt, and time-varying growth rate. Estimates over the last 16 days of the 125 time-series are based on partial data due to the presumption of reporting delays. Input parameters and 126

for adjusting for right-truncation evolved as we learned more about the outbreak. 127 Communication methods 128 Rt estimates were shared internally through Situational Reports and leadership meetings and 129 publicly through CDC’s Technical Reports (26) and CDC’s public-facing mpox website (27). The 130 Technical Reports were co-led by the Center for Forecasting and Outbreak Analytics (CFA) and the 2022 131 Multi-National Mpox Outbreak Response. Estimates were generated for distribution at least once per 132 week. The technical reports were intended for scientific audiences. The purpose of sharing these results 133 was to improve understanding of the outbreak and inform further scientific inquiry. 134 Performance assessment methods 135 We chose eight dates during four key outbreak phases to retrospectively evaluate our short-term 136 (one-week-ahead) forecasts of reported mpox cases generated from EpiNow2: 1. one month into the 137 outbreak prior to exponential growth (June 13 and June 27); 2. during exponential growth (July 5); 3. near 138 the outbreak peak (July 27); and 4. during the declining phase (September 6, September 19, October 11, 139 and December 5). Ideally, the same day of the week would be used, but some historical versions of the 140 dataset were not available for this analysis and some dates fell on national holidays which may have 141 introduced additional delays. Like the real-time analyses, we used rash onset date as the first reference 142 date to define the reporting delay distribution for all eight time points, while the second reference date 143 changed over time (Table S1). 144 We used three metrics to evaluate the forecasts. Our primary metric was the weighted interval 145 score (WIS). For each of the eight time points considered, WIS was computed for each daily prediction 146 and averaged across the seven-day forecast. The WIS measures the consistency of a group of prediction 147 intervals with an observed value (probabilistic accuracy). The WIS is positive, and lower values 148 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint correspond to smaller error (Supplementary methods) (15). To evaluate the error in the forecast’s point 149 estimate, we used the mean absolute error (MAE), which was computed as /g1839/g1827/g1831 /g3404 /g2869 /g3015 ∑| /g1877 /g3036 /g3398/g1877 /g3548 /g3036 |/g3015 /g3036/g2880/g2869 , 150 where /g1877 /g3036 is the observed number of mpox cases on day /g1861 , /g1877 /g3548/g3036 is the median forecast on day /g1861 , and N = 7 151 (15). We also used prediction interval (PI) coverage rates, which check the degree to which the model 152 provides calibrated predictions. Coverage rates are calculated by determining the proportion of times the 153 50% or 90% PIs included the observed value (for example, a well-calibrated forecast would have a 50% 154 PI coverage close to 0.50. Also see Supplementary methods) (15). 155 We compared the performance of EpiNow2 with a naïve Bayesian generalized linear model 156 (GLM, Supplementary methods). We calculated the relative WIS and relative MAE for EpiNow2 and the 157 GLM as /g2016 /g3006/g3043/g3036/g3015/g3042/g3050/g2870,/g3008/g3013/g3014 /g3404 /g3040/g3032/g3028/g3041 /g3046/g3030/g3042/g3045/g3032 /g3042/g3033 /g3006/g3043/g3036/g3015/g3042/g3050/g2870 /g3040/g3032/g3028/g3041 /g3046/g3030/g3042/g3045/g3032 /g3042/g3033 /g3008/g3013/g3014 , where the mean score is the average of the models’ 158 performance (WIS or MAE) over all eight dates evaluated. If /g2016 /g3006/g3043/g3036/g3015/g3042/g3050/g2870,/g3008/g3013/g3014 was less than 1, that indicated 159 the forecasts generated by EpiNow2 had less error than the GLM, whereas /g2016 /g3006/g3043/g3036/g3015/g3042/g3050/g2870,/g3008/g3013/g3014 > 1 indicated 160 EpiNow2 performed worse. 161 For both EpiNow2 and the GLM, we removed recent cases (defined as cases reported in the last 3 162 – 5 days) from the time series to adjust for right truncation of the data for all four outbreak phases (Table 163 S1). Forecasts were evaluated using mpox data as of March 16, 2023. We used the most recent version of 164 event date as the basis for the comparison. 165

166 Challenges of nowcasting/forecasting 167 It is voluntary for jurisdictions to report mpox cases to CDC, with only minimal data needed to 168 submit a case report form (e.g., case ID and reporting jurisdiction) in part, because not all cases may be 169 reached or fully investigated. CDC asks jurisdictions to collect and report additional data variables to 170 achieve situational awareness and surveillance goals. The number of variables on the case report form 171 was decreased to reduce reporting burden. Despite these efforts, jurisdictional case surveillance systems 172 may not have aligned to CDC’s requested data variables, and jurisdictions may choose to limit what data 173 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint are shared with CDC based on local reporting practices. Received data were subjected to additional 174 manual data cleaning to standardize formats and correct obvious data entry errors. As a result, even key 175 data variables such as demographic characteristics (e.g., age, race/ethnicity, HIV status, vaccination) were 176 not consistently available across all jurisdictions and time periods, precluding detailed sub-analyses. For 177 example, out of 29,921 cases in DCIPHER through December 31, 2022, 21,480 (71.8%) were missing 178 HIV status, 16,474 (55.1%) were missing smallpox vaccination, 3,913 (13.1%) were missing gender 179 identity, 3,172 (10.6%) were missing race, 2,928 (9.8%) were missing ethnicity, and 250 (0.8%) were 180 missing age. The timing and frequency of data submission varied between jurisdictions and changed over 181 the course of the outbreak. Some jurisdictions reported case data in near real-time whereas others 182 submitted a large number of cases all at once, the latter of which caused large, artificial spikes in the time-183 series. There were instances of duplicate cases being reported from several jurisdictions which may be 184 attributed to the changes in reporting processes. Spurious cases at the end of the time series had to be 185 investigated (and usually removed) because they artificially inflated the nowcasts/forecasts. These data 186 issues required us to monitor the model output closely and modify the code as needed. 187 In early July 2022, reporting of mpox cases to CDC started to lag in some jurisdictions, especially 188 those most affected by the outbreak. These few jurisdictions were publicly reporting more cases on their 189 websites than what CDC had received reports for. This led to a lengthy case reconciliation process during 190 which case data uncertainty prevented it from being used for nowcasting/forecasting at the national level. 191 Also in July, an increasing proportion of cases were reported with missing symptom onset dates (from 192 26% to 53% for rash onset date between June 13 and July 5). To ensure each case had a date associated 193 with it for plotting epidemic curves, a new event date field was created which we started using for 194 nowcasting/forecasting. The new calculated date field selected the best available date among possible date 195 fields based on the following priority, ordered from most to least preferable: orthopoxvirus test date, date 196 of call to the call center, date the short CRF was created, and the long CRF timestamp (Figure S1). The 197 date in which the record was created was least preferred due to artificial spikes in the time series caused 198 by bulk uploading data. In September 2022, a different date was adopted by the response for reporting 199 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint case data. This date field was defined as the earliest among all available dates for a case including 200 symptom onset, which facilitated improved visualization of epidemic curves. However, this new date 201 presented challenges for its use in the EpiNow2 framework because the delay from symptom onset to this 202 new date would have more variation than the delay using the original event date, including a delay of zero 203 for some cases. Thus, we worked to create a new event date field specifically for nowcasting/forecasting 204 which was similar to the original event date. The definition was expanded to include dates available in 205 NNDSS data. 206 Successes of nowcasting/forecasting 207 During the case reconciliation process in July, publicly available data through health department 208 websites was used for subnational analyses [e.g., California (28) and New York City (29). We used 209 WebPlotDigitizer (30) to extract time series data from pdfs when the underlying data were not available 210 for download. 211 In October, we updated estimates of the serial interval for rash onset of 7.0 days (95% CrI 5.8 – 212 8.4) from 40 case pairs and incubation period for rash onset of 7.5 days (95% CrI 6.0 – 9.8) from 35 U.S. 213 case-patients and used those as model inputs for EpiNow2 (31). These data were obtained through the 214 collaboration of several U.S. jurisdictions on a special study. The estimated serial interval for the 2022 215 outbreak was on the lower end of the historical range observed in the Democratic Republic of Congo (7 – 216 23 days) (24). 217 Adapting model output and communicating nowcasts/forecasts 218 We adapted the presentation of our results for a scientific/technical audience and the general 219 public. The default plots from EpiNow2 included three panels: cases by date of report, cases by date of 220 infection, and Rt. Green represented estimates based on complete data, orange represented estimates based 221 on partial data, and purple represented the forecast (the default is seven days). Gray bars in the top panel 222 showed the actual time series of reported cases, while gray bars in the middle panel showed the back-223 calculated infection time series. For the Technical Reports, Situational Reports, and response updates 224 meetings, we removed the middle panel (Figure 1) (26). For the website, we only showed Rt, removed the 225 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint forecast, and removed the 20% credible intervals to minimize confusion (Figure 2) (27). We included a 226 simple description of the plot that could be understood by non-experts. In accordance with CDC’s Data 227 Modernization Initiative, a national effort aimed at modernizing state and national core data and 228 surveillance infrastructure (32), data for the underlying plots were made available for download as 229 comma-separated values (csv) files with the Technical Reports. 230 Sub-national analyses revealed some differences between regions regarding the start of the 231 outbreak, when it peaked, and how long it lasted (Figures 3 – 4). For example, Figure 4 demonstrates a 232 later introduction date and slightly longer tail for Texas compared to other jurisdictions. 233 Performance assessment 234 The GLM had lower WIS compared to EpiNow2 for early phase of the outbreak (Table 1); 235 however, during all other phases, EpiNow2 had a slight advantage. The relative WIS was 0.89 over all 236 eight time points considered, indicating that EpiNow2 had on average 11% less probabilistic error than 237 the GLM. 238 EpiNow2 had lower MAE than the GLM for six out of eight time points, but performance was 239 similar: the relative MAE was 0.96. In other words, EpiNow2 had only 4% less point error than the GLM. 240 Overall, predictions were moderately well calibrated. For the 90% PI, EpiNow2 achieved 241 coverage rates within 10% of the desired coverage level for seven out of eight time points compared to six 242 out of eight for the GLM. For the 50% PI, EpiNow2 achieved coverage rates within 10% of the desired 243 coverage level for only two out of eight time points compared to five out of eight for the GLM. 244 Qualitative results of the nowcasts/forecasts are shown in Figure S2. The 90% CrIs from 245 EpiNow2 were very wide for the early phase of the outbreak, while the GLM had large uncertainty 246 around the outbreak’s peak. Both models underestimated reported mpox cases for the seventh time point 247 on October 11 which could be due to a discrepancy in the data available at the time versus the ground-248 truth data (Figure S3). This time point had the lowest PI coverage rates. 249

250 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint We performed nowcasting/forecasting to inform the U.S. response to the 2022 mpox outbreak in 251 real-time. Validation showed that the method implemented in EpiNow2 predicted case counts reasonably 252 well, but improvements are needed around key time periods such as the outbreak’s peak. One reason that 253 the nowcasts/forecasts did not always align with reality is because the definition of event date changed 254 over time, while the study data were constructed the most recent version of the event date field. We found 255 a higher WIS for EpiNow2 in the early phase of the outbreak compared to the GLM which could be due 256 to choices of priors for parameters (e.g., wide intervals for Rt). 257 Subnational analyses allowed us to better understand the spatial heterogeneity of the epidemic 258 which may be attributed to differences between jurisdictions in terms of composition (e.g., population age 259 structure, density, and contact patterns) and public health activities (e.g., vaccination, surveillance 260 methods, frequency of testing) (33) as well as the timing and frequency of case reporting. One limitation 261 of nowcasting/forecasting at the subregional or jurisdictional level is that the effects of bulk uploads are 262 more apparent, resulting in greater uncertainty (wider credible intervals). Another limitation is that 263 movement between jurisdictions could have a greater impact on subnational estimates, as mobility is not 264 accounted for in our approach. Finally, some jurisdictions stopped reporting rash onset date, which 265 decreased the sample size available for estimating the reporting delay distribution over time. 266 Data Quality 267 Nowcasting/forecasting methods perform best when the underlying surveillance data are accurate, 268 timely, and complete, but they are often sub-optimal and variable as the outbreak evolves; while data may 269 improve as an outbreak progresses, they may re-deteriorate once the outbreak slows and intensity of effort 270 is low. Fortunately, the quality and frequency of data improved over the course of the U.S. mpox 271 outbreak. Communicating with specific jurisdictions about our priority dates for modeling improved data 272 quality. These prompts to the jurisdictions need to be continued regularly throughout the outbreak. Close 273 collaboration between epidemiologists/modelers and informaticians, including the use of an issue tracking 274 system in DCIPHER, also facilitated quick investigation and resolution of data errors. 275 EpiNow2 Limitations 276 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint The main limitation of EpiNow2 is its steep learning curve due to limited documentation of 277 package functions and few reports of its application to other outbreaks. Increasing commenting in the 278 code, creating more tutorials or vignettes, and developing a graphical user interface could help. 279 Another limitation is the long computing time required for the analyses. We were able to increase 280 computational efficiency by running the model on multiple cores in parallel, but the processing time 281 became particularly cumbersome if an analysis needed to be repeated. In the future, cloud-based 282 computing could be used to obtain more consistent and faster model run times. 283 There were also instances of unusually long run times whereby the first two Markov chain Monte 284 Carlo (MCMC) chains performed as expected, but subsequent chains never finished processing. Some 285 MCMC convergence issues were resolved by reducing the parameter fitting period (e.g., truncating the 286 beginning of the time series). One study reported that EpiNow2 estimates are more reliable when case 287 numbers at each time step are large and there are at least 14 timepoints without zeroes (34). Large daily 288 fluctuations and limited case counts could substantially affect model estimates, which should be 289 interpreted with caution. 290 Another limitation is that the method we used does not account for under-ascertainment, which 291 occurs when not all infections are diagnosed and reported as cases of the disease to the surveillance 292 system. The under-ascertainment rate is needed to understand the true burden of disease caused by the 293 outbreak; however, current estimates for the U.S. mpox outbreak are lacking. Indirect evidence from a 294 recent modeling study (35) suggests that 65% of mpox infections were diagnosed and reported in 295 Washington, D.C. However, the model was not designed to measure the under-ascertainment rate (Patrick 296 Clay, personal communication, March 10, 2023), and this quantity should be assessed by other methods 297 (e.g., models specifically designed to assess under-ascertainment, serological surveys, and community-298 based surveys). 299 Strategies for Forecasting the Next Outbreak 300 For the next outbreak, it is important for CDC to develop strategies for regularly capturing and 301 storing snapshots of surveillance data which remain easily accessible for systematic analysis. For routine 302 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint case-based surveillance of notifiable diseases, such as rabies, most analyses are performed only after the 303 data have undergone a rigorous and routine reconciliation and closeout process by data submitters with 304 further validation by CDC surveillance epidemiologists; however, timely outbreak response decision 305 support does not allow for such processes. Instead, jurisdictions are asked to submit available case data in 306 near real-time and submit additional data or corrections to data entry errors as time and resources permit. 307 Snapshots of the surveillance data were saved in an ad hoc manner (by exporting data on a particular day 308 and saving a csv file locally), and as a consequence, a complete history of the data is not available, 309 especially around key points in the outbreak, such as the peak. A complete history would help to 310 understand key delay distributions and other quirks (e.g., backfilling and revision of reference dates) 311 involved in the data-generating process. Understanding the data generating process is crucial for the 312 improvement of methods and tools for nowcasting/forecasting and aligns with one of the five priorities of 313 CDC’s Data Modernization Initiative (Accelerating Data for Action: Tapping into more data sources, 314 promoting health equity, and increasing capacities for scalable outbreak response, forecasting, and 315 predictive analytics) (32). In the future, the process of saving snapshots of the data could be automated. 316 Ensemble models have been used for a variety of infectious disease outbreaks, such as COVID-317 19 (15, 36), Zika (37), influenza (38), and Ebola (39). Ensembles combine predictions from several 318 models that use different methodology and sometimes input data. Because some models overpredict, 319 while others underpredict, ensemble models often outperform individual models over time. In the future, 320 we may consider using at least two simpler models and comparing them. 321 One potentially useful addition to EpiNow2 and other currently available tools for 322 nowcasting/forecasting outbreaks would be flexibility in handling dates. We frequently encountered 323 missing dates for cases in the mpox surveillance data. Ideally, a method or tool would be able to keep 324 track of multiple dates for a case and estimate missing dates based on the full distribution of dates across 325 all cases. Epinowcast is a new hierarchical nowcasting package that enables more flexibility in adjusting 326 for truncated data (40). Novel nowcasting approaches use hierarchical generalized additive models, which 327 can provide even more flexibility to modify the model in real-time to the evolving data environment (41). 328 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint Another improvement would be to reduce the time required to run the analyses. Rather than focusing on 329 the efficiency of the MCMC algorithm, computation time could be reduced if the model only needed to be 330 run on the new data. Finally, the imputed time series of cases by symptom onset date would be a useful 331 data visualization output that is not currently available in EpiNow2. As described above, defining the date 332 field for the presentation of epidemic curves was a challenge in the mpox outbreak and having an imputed 333 symptom onset date for each case would have been useful for comparison purposes. 334 CFA played an important advisory role in our nowcasting/forecasting efforts. CFA produces 335 models and forecasts to characterize the state of an outbreak and its course, inform public health decision 336 makers on potential consequences of deploying control measures, and support innovation to continuously 337 improve the science of outbreak analytics and modeling (42). In the future, CFA plans to create new tools 338 for outbreak analysis and modeling. CFA could also serve as a link between CDC modelers and 339 jurisdictions with modeling capacity to share experiences and code. Technical Reports represent a new 340 way for CDC to share timely information with the federal government, state and local leaders, and 341 scientists in academia and industry. Technical Reports have been well received within and outside CDC 342 (43-45) and their publication aligns with CDC’s current restructuring efforts aimed at making the agency 343 more response ready, including sharing science and data faster (46). 344

345 Real-time estimation of Rt as well as nowcasting/forecasting is one method for determining the 346 extent to which current public health measures are effective and/or need to be modified but is subject to 347 limitations. The quality and timeliness of reported data pose challenges to these analyses. Ease of use, 348 model computing time, and ability to handle multiple dates are priorities for consideration in the 349 development of future nowcasting/forecasting tools. A naïve model may be superior to a complex one, 350 such as EpiNow2, during the early phase of an outbreak when data scarcity causes Rt to be largely 351 unconstrained, especially once reporting delays are considered. Future outbreak response activities could 352 be enhanced through inclusion of clear and consistent communication about modeling outputs as well as 353 close collaboration between modeling and informatics/data teams. 354 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint 355 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint Tables 356 Table 1. Evaluation of short-term (one-week-ahead) forecasts of reported mpox cases during the 2022 U.S. outbreak. For WIS, bold indicates 357 where one model performed better than the other. 358 EpiNow2 Bayesian GLM with negative binomial Forecast date Outbreak phase 90% PI coverage 50% PI coverage WIS MAE 90% PI coverage 50% PI coverage WIS MAE Monday, June 13 Early 1 0.71 33.8 4.3 0.86 0.57 6.4 5.1 Monday, June 27 Early 0.86 0 34.0 29.0 0.86 0.14 24.7 26.9 Tuesday, July 5 Exponential growth 0.86 0.71 35.6 32.4 1 0.57 38.5 34.1 Wednesday, July 27 Peak 1 0.57 137.0 107.9 1 0.57 184.3 94.4 Tuesday, September 6 Decline 0.86 0.71 107.6 84.3 0.86 0.43 117.1 86.8 Monday, September 19 Decline 0.86 0.43 61.2 52.4 0.71 0.43 81.8 65.9 Tuesday, October 11 Decline 0.43 0.29 30.6 33.4 0.57 0.14 38.3 41.9 Monday, December 5 Decline 1 0.71 4.3 3.1 1 0.71 6.4 4.6 PI: prediction interval; WIS: weighted interval score; MAE: mean absolute error; GLM: generalized linear model 359 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint Figures 360 Figure 1. Effective reproduction number estimates for the U.S. 2022 mpox outbreak intended for a 361 technical/scientific audience. The top panel shows estimates of cases by date of report with actual cases 362 shown by gray bars. The bottom panel shows estimates of the effective reproduction number by date. In 363 all panels, shaded regions reflect 90%, 50%, and 20% credible intervals in order from lightest to darkest. 364 Green shows estimates, red shows estimates based on partial data, and purple shows forecasts. Event date 365 is determined by a hierarchy across the different data streams where priority is given to diagnosis date, 366 orthopoxvirus test date, orthopoxvirus test confirmation date, case investigation start date, orthopoxvirus 367 sample collection date, date of call to CDC call center, report date (to public health department, county, or 368 state), date CDC announced case, and the date the case was entered into DCIPHER, in that order. 369 370 Figure 2. Effective reproduction number (Rt) estimates for the U.S. 2022 mpox outbreak intended for the 371 general public. The graph shows the Rt estimation over time based on complete data (gray) or partial data 372 (blue). The most recent data are considered incomplete due to delays in reporting mpox cases. As a result, 373 there is more uncertainty associated with the most recent Rt estimates. Rt > 1 means the epidemic is 374 growing. Rt < 1 means the epidemic is shrinking. Shading represents the 50% and 90% credible intervals 375 (uncertainty in the estimates) 376 377 Figure 3. Effective reproduction number estimates for the 2022 mpox outbreak in four U.S. Census 378 regions. The left panels show estimates of cases by date of report with actual cases shown by gray bars. 379 The right panels show estimates of the effective reproduction number by date. In all panels, shaded 380 regions reflect 90%, 50%, and 20% credible intervals in order from lightest to darkest. Green shows 381 estimates, red shows estimates based on partial data, and purple shows forecasts. Event date is determined 382 by a hierarchy across the different data streams where priority is given to diagnosis date, orthopoxvirus 383 test date, orthopoxvirus test confirmation date, case investigation start date, orthopoxvirus sample 384 collection date, date of call to CDC call center, report date (to public health department, county, or state), 385 date CDC announced case, and the date the case was entered into DCIPHER, in that order. 386 387 Figure 4. Effective reproduction number estimates of the 2022 mpox outbreak for the six jurisdictions in 388 the U.S. with the highest case counts. The left panels show estimates of cases by date of report with actual 389 cases shown by gray bars. The right panels show estimates of the effective reproduction number by date. 390 In all panels, shaded regions reflect 90%, 50%, and 20% credible intervals in order from lightest to 391 darkest. Green shows estimates, red shows estimates based on partial data, and purple shows forecasts. 392 Event date is determined by a hierarchy across the different data streams where priority is given to 393 diagnosis date, orthopoxvirus test date, orthopoxvirus test confirmation date, case investigation start date, 394 orthopoxvirus sample collection date, date of call to CDC call center, report date (to public health 395 department, county, or state), date CDC announced case, and the date the case was entered into 396 DCIPHER, in that order. 397 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint Data availability 398 Data and code to run the nowcasts/forecasts and perform model validation will be available on GitHub 399 following publication in a peer-reviewed journal. 400 401

402 We thank all public health professionals involved in reporting mpox cases to CDC. We also acknowledge 403 the Mpox Response Data Analytics and Visualization Task Force Informatics Team for data management 404 and support, and we thank Dr. Sam Abbott for helpful discussions about EpiNow2 methods. 405 406 Declaration of interest 407 The authors declare the following financial interests/personal relationships which may be considered as 408 potential competing interests: Kelly Charniga reports a relationship with Systems Planning and Analysis 409 Inc that includes: consulting or advisory. 410 411 Disclaimer 412 The findings and conclusions in this report are those of the authors and do not necessarily represent the 413 official position of the Centers for Disease Control and Prevention, U.S. Department of Health and 414 Human Services. 415 416 Ethics statement 417 This activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC 418 policy (45 C.F.R. part 46, 21 C.F.R. part 56; 42 U.S.C. Sect. 241(d); 5 U.S.C. Sect. 552a; 44 U.S.C. Sect. 419 3501 et seq). 420 421 Funding 422 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint No specific funding was obtained for this work. 423 424 Author contributions 425 Conceptualization (KC, YN), data curation (KC, JA), formal analysis (KC, ZJM), investigation (KC, 426 ZJM), methodology (KC, ZJM, JA), project administration (KC), software (KC, ZJM, NBM), supervision 427 (JA, YN, IHS), validation (KC, ZJM), visualization (KC, ZJM), writing – original draft preparation (KC, 428 ZJM), writing – review & editing (KC, ZJM, NBM, JA, YN, IHS). 429 430

431 1. World He alth O rganiza tion . WHO Direc to r-Gene ral decla res th e ongoing monkeyp ox outb reak a 432 Public Health Emergency of Int erna tional Concern 2022 [updated J uly 23, 2022. Available from: 433 https://www.who.int/eur ope/news/item /23-07-2022 -who-director-gene ral-decla res-the-ongoing-434 monkeypox-outb reak-a-public-heal th-ev ent-of-inte rnati onal-concer n . 435 2. U.S. Cent ers for Disease Con trol and Prev ention . 2022 Monkeypox Outb reak Gl ob al Map 2022 436 [updated November 15, 2022. Available f rom: 437 https://www.cdc.gov/poxvirus/monkeypox/respons e/2022/world-map.html . 438 3. Beer E, Ra o V. A systematic r eview of the epidemiology of human monkeypox ou t breaks and 439 implications for outb reak st rat egy. PLoS Negl Trop Dis. 2019;13(10):e0007791. 440 4. U.S. Cent ers for Disease Con trol and Prev ention . Signs and Symptoms 2022 [updat ed Octob er 441 18, 2022. Available from: h ttps ://www.c dc.gov/poxvirus/monkeypox/symptoms/index .html . 442 5. WHO/ECDC. Interim advice for public he alth au thori ties on summer eve nts durin g the 443 monkeypox outb reak in Euro pe, 2022 20 22 [updated Ju ne 14, 2022. Availab le fro m: 444 https://www.ecdc.eur opa. eu/sites/defa ult/files/documents/Int erim-advice-for-public-health-445 authori ties-on-summer-eve nts-mpx. pdf . 446 6. UK Health Secu rity Agency. Investigati on into monkeypox o utbr eak in England: t ec hnical briefing 447 1 2022 [updated Sep tembe r 23, 2022. Available from: 448 https://www.gov.uk/government/publications/monkeypox-ou tbr eak-technical-b riefings/investigation-449 into-monkeypox-ou tbr eak-in-england-te chnical-briefing-450 1#:~ :te xt =Wi thou t%20correct ing%20for%20right%2Dtruncation ,8.9)%2C%20see%20Figure%202a . 451 7. Thompson J, McClure R, Sco tt N , Hella rd M, Abeysuriya R, Vidan aarachchi R , et al . A framework 452 for considering th e utility of models whe n facing tough decisions in public health : a guideline for policy-453 makers. Heal th Res earch Policy and Syst ems. 2022;20:107. 454 8. Buchwald A, Adams J , Bor tz D, Carlton E. Infectious Disease Tr ansmission Models t o Predict, 455 Evaluate, and Improve Und erst anding of COVID-19 Trajectory and Inte rventi ons. Annals of the Am erican 456 Thoracic Society. 2020;17(10):1204-6. 457 9. Madewell Z, Pastor e Y Piontti A, Zhang Q , Burton N, Yang Y, Longini I, et a l. Using simulated 458 infectious diseas e outb reaks to inform si te selec tion and sampl e size for individua lly randomized vaccine 459 trials during an o ngoing epidemic. Clinica l Trials. 2021;18(5):630-8. 460 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint 10. Lloyd-Smith J, Funk S, McLean A, Ril ey S, Wood J . Nine chall enges in modelling the emergence of 461 novel pathoge ns. Epidemics. 2015;10:35- 9. 462 11. Madewell Z, Dean N, Berlin J, Coplan P, Davis K, Struchiner C, et al. Challenges of e valuating and 463 modelling vaccination in eme rging infecti ous diseases. Epid emics. 2021;37:10050 6. 464 12. Chakraborty T, Gh osh I, Mah ajan T, Aro r a T. Nowcasting of COVID-19 Confirmed Cases: 465 Foundati ons, Trends, and Challenges . In: Azar A, H assanien A , edi tors. M odeling, C ontrol a nd Drug 466 Development for COVID-19 Outbr eak Pre vention. S tudies in Systems, Decision a nd Control: Spring er, 467 Cham; 2022. 468 13. Abbot t S, Hell ewell J, Thompson R , Sher r att K, Gib bs H, Bosse N , et al . Estimating the time-469 varying reproducti on number of SAR S-CoV-2 using national and subnati onal case c ounts [version 2; pe er 470 review: 1 approved , 1 approved with res ervations]. Wellcome Open R esearch . 20 20;5:112. 471 14. Cori A, Ferguson N , Frase r C, Cauchemez S. A New Fram ework and Softwar e to Est imate Time-472 Varying Reproducti on Numbe rs During Epidemics. America n Jour nal of Epidemiol ogy. 2013;178(9):1505-473 12. 474 15. Cramer E, Ray E, Lopez V, Brache r J, B ren nen A, Rivade neira A, e t al. Evaluat ion of individual and 475 ensemble pr obabilist ic forecasts of COVI D-19 mortality in the Uni ted St at es. PNA S. 476 2022;119(15):e2113561119. 477 16. Arslan S, Oz demir M, Uc ar A. Nowcasting and Forecas ting the Sp read of COVID-19 and 478 Healthca re Demand in Turkey, a M odelin g Study. Front Public Heal th. 2021;8 :575 145. 479 17. Birrell P, Blake J, van Leeuwen E, Gen t N, De Angelis D. Real-time nowcasti ng and forecasting of 480 COVID-19 dynamics in England: the first wave. Philos Trans R Soc Lond B Biol Sci. 481 2021;376(1829):20200279. 482 18. Spreco A, Jöud A, Eriksson O, Solt esz K, Källström R, Dahlst röm Ö, e t al. Nowcastin g (Short-Term 483 Forecasti ng) of COVID-19 Hospitalization s Using Syndromic Healthcar e Data, Swe den, 2020. Emerg Infec t 484 Dis. 2022;28:564-71. 485 19. Biggerstaff M, Slayton R , Joha nsson M, B utler J. Impr oving Pandemic Response: E mploying 486 Mathem atical Mo deling to Confron t Cor onavirus Disease 2019. Clinical Infecti ous Diseases. 487 2021;74(5):913-7. 488 20. U.S. Depa rtmen t of Heal th and Human S ervices. Privacy Impact Assessmen t 2020 [updated 489 March 26, 2020. Availabl e from: ht tps://www.hhs.gov/sites/default/files/cdc-dci pher-whr.pdf . 490 21. Cori A, Cauchemez S, Fe rguson N, Fr aser C, Dahlqwist E, Demarsh P, et al. EpiEsti m: Estimate 491 Time Varying Reproduction Numbers fro m Epidemic Curves. 2.2-4 ed. CRAN2021. 492 22. Jombar t T, Cori A, Nouvell et P, Skarp J, Kamvar Z, Taylor T. earlyR: Estima tion of Transmissibility 493 in the Early Stag es of a Disease Outb reak . 0.0.5 ed : CRAN; 2020. 494 23. Abbot t S, Hell ewell J, Sh err att K, Gostic K , Hickson J, Badr H, et al . EpiNow2: Estim ate r eal-time 495 case counts and tim e-varying epidemiolo gical paramet ers. 1 .3.2 ed : EpiForecas ts; 2021. 496 24. Fenner F , Hende rson D, Ari ta I, Jezek Z, Ladnyi I. Smallpo x and its Eradic ation1988 . Available 497 from: https ://apps.who.in t/iris/handle/1 0665/39485 . 498 25. Nash R, No uvellet P, Cori A . Real-tim e est imation of the epidemic re produc tion nu mber: Scoping 499 review of the applica tions and chall enge s. PLOS Digit Health. 2022;1(6):e0000052 . 500 26. U.S. Cent ers for Disease Con trol and Prev ention . Monkeypo x Technical Rep orts 20 22 [updated 501 Octobe r 27, 2022. Availabl e from: ht tps:/ /www.cdc.gov /poxvirus/monkeypox/cases-data/t echnical-502 repor t.h tml . 503 27. U.S. Cent ers for Disease Con trol and Prev ention . Ou tbre ak Repro duction Number Estimates 504 2022 [updated Novemb er 14, 2022. Avail able from: ht tps://www.cdc.gov/poxvirus/monkeypox/cases-505 data/case-nowcasting .html . 506 28. California Departm ent of Public Health . Monkeypox Dat a in California 2022 [updated N ovember 507 10, 2022. Available from: h ttps ://www.c dph.ca.gov/Programs/CID/DC DC /Pages/ Monkeypox-Data .asp x . 508 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint 29. New York City Departmen t of Health and Mental Hygien e. 2022 Monkeypo x Surve illance Data 509 2022 [updated July 19, 2022. Availa ble fr om: 510 https://www1.nyc.gov/assets/doh/downloads/pdf/monkeypox/data-daily-cases- demographics-511 072122.pdf . 512 30. Rohatgi A . WebPlo tDigitizer . Vers. 4.5 ed . Pacifica, California, USA2021. 513 31. Madewell Z, Charniga K, Mas ters N , Ashe r J, Fahr enwald L, Still W , et al . Seri al inte rval and 514 incubation p eriod es timat es of monkeypox virus infection in 12 U.S . jurisdictions, May – August 2022. 515 Emerg Infect Dis. 2023;29(4):818-21. 516 32. U.S. Cent ers for Disease Con trol and Prev ention . Wha t is the Dat a Mode rniza tion I nitiative ? 517 2022 [updated Oc tobe r 11, 2022. Availab le from: htt ps://www.cdc.gov/surveillance/projects/dmi-518 initiative/ind ex .html# :~: te xt =CDC%E2%8 0%99s%20Data%20Moderniza tion%20Ini tiative%20%28DMI%2519 9%20is%20at%20the,da ta%20repo rting %2C%20management%2C%20and%20analytics%20across%20pu520 blic%20health . 521 33. De Nicola G, Schn eble M, Kau ermann G, Berger U . Region al now- and forecasting for data 522 repor ted with d elay: towa rd surveillanc e of COVID-19 infections. Adv Stat Anal . 20 22;106(3):407-26. 523 34. Binny R, Lustig A, Hendy S, Macla ren O , R idings K, Vattiato G, e t al. R eal-time es ti mation of the 524 effective repr oduction numb er of SARS-C oV-2 in Aotear oa New Zeal and. Pee rJ . 2022;10:e14119. 525 35. Clay P, Asher J, Carnes N, Cop en C, Delaney K, Payne D, et al. Modelling the impac t of 526 vaccination and se xual beh avior change on repor ted cas es of mpox in Washingt o n D.C. medRxiv. 2023. 527 36. Dean N, Pastor e Y Piontti A, Madewell Z, Cummings D, Hitchings M, Joshi K, et al . Ensemble 528 forecast modeli ng for the design of COVI D-19 vaccine effi cacy trials. Vaccine. 202 0;38(46):7213-6. 529 37. Oidtman R, Omodei E, Kra emer M , Casta ñeda-Orjuel a C, Cruz-Rivera E, Misnaza-C astrillón S, et 530 al. Trade-offs betwe en individual and en semble forecasts of an em erging infectio us disease. N atu re 531 Communications. 2021;12:5379 . 532 38. Reich N, McG owan C, Yamana T, Tushar A, Ray E, Osthus D, et al. Accur acy of real -time multi-533 model ensembl e forecasts for se asonal i nfluenza in th e U.S. PLOS Computa tiona l Biology. 534 2019;15(11):e1007486. 535 39. Viboud C, Sun K, Gaffey R, Ajelli M, Fuma nelli L, Merle r S, e t al. The R APIDD ebola forecasting 536 challenge: Synth esis and lessons le arn t. E pidemics. 2018;22:13-21. 537 40. Abbot t S, Lison A, Funk S. e pinowcast: Fl exible hi era rchical nowcasting. Zeno do; 2021. 538 41. Overto n C, Abbot t S, Christie R , Cumming F, Day J, Jones O, e t al. Nowcasting th e 2022 mpox 539 outbr eak in England. arXiv. 2023. 540 42. U.S. Cent ers for Disease Con trol and Prev ention . Cent er for For ecasting and Outbr eak Analytics 541 2022 [updated Oc tobe r 28, 2022. Availab le from: htt ps://www.cdc.gov/ forecast-o utbre ak-542 analysis/index.h tml . 543 43. Daskalakis D. You know that you are a #p ublichealth n erd when you wake up on a gloomy 544 Saturd ay to re ad @CDCgo v 545 Twitter; 2022 [upda ted Oc tobe r 1, 2022. [Tweet]]. Available from: 546 https://twit ter .com/dr_d emet re/sta tus/1576196874079735808?s=48&t=9NVB0 dQPIU9S4J58A JJlfw . 547 44. Fox S. These monk eypox t echnical re por t s from the CDC are [three fire emojis]: T witter ; 2022 548 [updated November 8, 2022 . [Tweet]]. A vailable from: 549 https://twit ter .com/Foxan dthe Flu/status /1589978622026461184 . 550 45. Rivers C. Changing trends in monkeypox epidemiology: Subs tack; 2022 [update d Septemb er 2, 551 2022. Available from: h ttps ://caitlinriver s.substack.com/p/changing-trends-in-mo nkeypox-552 epidemiology . 553 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint 46. U.S. Cent ers for Disease Con trol and Prev ention . CDC Moving Forward Summary Report 2022 554 [updated S ept ember 1, 2022Decembe r 2 7, 2022]. Available from: 555 https://www.cdc.gov/about/organiza tio n/cdc-moving -forward-summary-report. html . 556 557 for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint for use under a CC0 license. This article is a US Government work. It is not subject to copyright under 17 USC 105 and is also made available (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 8, 2023. ; https://doi.org/10.1101/2023.04.14.23288570doi: medRxiv preprint