Derivation and Validation of a Point-based Forecasting Tool for SARS-CoV-2 Critical Care Occupancy

Background The requirement for critical care in even a modest fraction of SARS-CoV-2 infected individuals made ICU resources an important societal chokepoint during the recent pandemic. We developed a simple regression-based point score in 2020 based on an objective of forecasting critical care occupancy in the Canadian province of Ontario based on mean age of cases, case numbers, and testing volume. Evolution of the pandemic (variants of concern, vaccination) led us to re-assess and re-calibrate our earlier work, with inclusion of information vaccination which became widespread in 2021.

We obtained complete provincial SARS-CoV-2 case, testing, and vaccination data for the period from March 2020 to September 2022, with data subdivided into 6 major “waves”, following the approach applied by other Canadian investigators. Our initial model was fit only using the first two “wild type” SARS-CoV-2 waves; an updated model included wave 3 (N501Y+ variants). Our model was validated by comparing model projections to waves not used for model fitting; validation model fits were evaluated with Spearman’s rho; counterfactuals without vaccination were modeled to impute fraction of critical care admissions prevented with vaccination. Costing was based on published economic estimates.

Our initial model (fit to waves 1 and 2) was well calibrated (rho 0.85) but predictive validity was modest (rho 0.46). Predictive validity improved in models fit to the first 3 pandemic waves without vaccination (rho 0.60) or with vaccination (rho 0.68) (P for inclusion of vaccination 0.013 by Likelihood Ratio Test). Prevented fraction of ICU admissions attributable to vaccination was 144% (22017 admissions expected vs. 9020 observed); based on published estimates of ICU admission cost for SARS-CoV-2 the 12977 admissions averted $2.9 (CDN) billion in economic costs, in contrast to the $3 billion total cost of the vaccination program.

Simple time series regression incorporating case and testing characteristics continues to be useful as a tool for forecasting critical care occupancy due to SARS-CoV-2 but early pandemic models need to be updated to capture the preventive effects of widespread vaccination. The economic benefit of vaccination for prevention of critical care resource consumption during the pandemic is substantial, achieving near cost neutrality with the province’s entire vaccination program. Competing Interest Statement DNF has served on advisory boards related to influenza and SARS-CoV-2 vaccines for Seqirus, Pfizer, Astrazeneca and Sanofi-Pasteur Vaccines, and has served as a legal expert on issues related to COVID-19 epidemiology for the Elementary Teachers Federation of Ontario and the Registered Nurses Association of Ontario. NJW worked for Johnson and Johnson Innovative Medicine in 2022. ART and AS were employed by the Public Health Agency of Canada when the research was conducted. The work does not represent the views of the Public Health Agency of Canada. Other authors: no competing interests. Funding Statement DNF received a grant from the Canadians Institutes for Health Research (2019 COVID-19 rapid researching funding OV4-170360), and was supported by a grant to the University of Toronto Institute for Pandemics from the R. Howard Webster Foundation. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Approved by the University of Toronto Research Ethics Board, Protocols #39239, #39253 and #44787. Consent to participate: not applicable. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data Availability Aggregate data and guidance on methods for reproduction of analyses are available from Dr. Fisman (david.fisman{at}utoronto.ca).