Why is there so much variability in crop multi-model studies?

It has become common to compare crop model results in multi-model simulation experiments. In general, one observes a large variability in such studies, which reduces the confidence one can have in such models. It is important to understand the causes of this variability as a first step toward reducing it. For a given data set, the variability in a multi-model study can arise from uncertainty in structure or in parameter values for a given structure. Previous studies have made assumptions about the origin of parameter uncertainty, and then quantified its contribution, generally finding that parameter uncertainty is less important than structure uncertainty. However, those studies do not take account of the full parameter variability in multi-model studies. Here we propose estimating parameter uncertainty based on open-call multi-model ensembles where the same structure is used by more than one modeling group. The variability in such a case is due to the full variability of parameters among modeling groups. Then structure and parameter contributions can be estimated sing random effects analysis of variance. Based on three multi-model studies for simulating wheat phenology, it is found that the contribution of parameter uncertainty to total uncertainty is, on average, more than twice as large as the uncertainty from structure. A second estimate, based on a comparison of two different calibration approaches for multiple models leads to a very similar result. We conclude that improvement of crop models requires as much attention to parameters as to model structure.