On the sensitivity of non-pharmaceutical intervention models for SARS-CoV-2 spread estimation

Introduction A series of modelling reports that quantify the effect of non-pharmaceutical interventions (NPIs) on the spread of the SARS-CoV-2 virus have been made available prior to external scientific peer-review. The aim of this study was to investigate the method used by the Imperial College COVID-19 Research Team (ICCRT) for estimation of NPI effects from the system theoretical viewpoint of model identifiability. Methods An input-sensitivity analysis was performed by running the original software code of the systems model that was devised to estimate the impact of NPIs on the reproduction number of the SARS-CoV-2 infection and presented online by ICCRT in Report 13 on March 30 2020. An empirical investigation was complemented by an analysis of practical parameter identifiability, using an estimation theoretical framework. Results Despite being simplistic with few free parameters, the system model was found to suffer from severe input sensitivities. Our analysis indicated that the model lacks practical parameter identifiability from data. The analysis also showed that this limitation is fundamental, and not something readily resolved should the model be driven with data of higher reliability. Discussion Reports based on system models have been instrumental to policymaking during the SARS-CoV-2 pandemic. With much at stake during all phases of a pandemic, we conclude that it is crucial to thoroughly scrutinise any SARS-CoV-2 effect analysis or prediction model prior to considering its use as decision support in policymaking. The enclosed example illustrates what such a review might reveal.