Reproduction as a Means of Evaluating Policy Models: A Case Study of a COVID-19 Simulation

This article proposes (and demonstrates the effectiveness of) a new strategy for assessing the results of epidemic models which we designate reproduction . The strategy is to build an independent model that uses (as far as possible) only the published information about the model to be assessed. In the example presented here, the independent model also follows a different modelling approach (agent-based modelling) to the model being assessed (the London School of Hygiene and Tropical Medicine compartmental model which has been influential in COVID lockdown policy). The argument runs that if the policy prescriptions of the two models match then this independently supports them (and reduces the chance that they are artefacts of assumptions, modelling approach or programming bugs). If, on the other hand, they do not match then either the model being assessed is not provided with sufficient information to be relied on or (perhaps) there is something wrong with it. In addition to justifying the approach, describing the two models and demonstrating the success of the approach, the article also discusses additional benefits of the reproduction strategy independent of whether match between policy prescriptions is actually achieved.