Deep convolutional neural networks exploit high spatial and temporal resolution aerial imagery to predict key traits in miscanthus.

Miscanthus is one of the most promising perennial crops for bioenergy production, with high yield potential and a low environmental footprint. The increasing interest in this crop requires accelerated selection and the development of new screening techniques. New analytical methods that are more accurate and less labor intensive are needed to better characterize the effects of genetics and the environment on key traits under field conditions. We used persistent multispectral and photogrammetric UAV time-series imagery collected 10 times in the season, together with ground-truth data over thousands of miscanthus genotypes to determine flowering time, culm length, and biomass yield traits. We compared the performance of Convolutional Neural Network (CNN) architectures that used image data from single dates (2D-spatial) versus the integration of multiple dates by (3D-spatio-temporal) architectures. The ability of UAV-based remote sensing to rapidly and non-destructively assess large-scale genetic variation in flowering time, height and biomass production was improved through use of 3D-spatio-temporal CNN architectures versus 2D-spatial CNN architectures. The performance gains of the best 3D-spatio-temporal analyses compared to the best 2D-spatial architectures manifested in up to: 23% improvements in R 2 , 17% reductions in RMSE, and 20% reductions in MAE. The integration of photogrammetric and spectral features with 3D- architecture was key to the improved assessment of all traits. In conclusion, this study demonstrates that the integration of high spatiotemporal resolution UAV imagery with 3D-CNNs enables more accurate monitoring of the dynamics of key crop phenological and yield-related traits. This is especially valuable in highly productive, perennial grass crops such as miscanthus, where in-field phenotyping is especially challenging and traditionally limits the rate of crop improvement through breeding.