aradeepopsis: From images to phenotypic traits using deep transfer learning

Linking plant phenotype to genotype, i.e., identifying genetic determinants of phenotypic traits, is a common goal of both plant breeders and geneticists. While the ever-growing genomic resources and rapid decrease of sequencing costs have led to enormous amounts of genomic data, collecting phenotypic data for large numbers of plants remains a bottleneck. Many phenotyping strategies rely on imaging plants, which makes it necessary to extract phenotypic measurements from these images rapidly and robustly. Common image segmentation tools for plant phenotyping mostly rely on color information, which is error-prone when either background or plant color deviate from the underlying expectations. We have developed a versatile, fully open-source pipeline to extract phenotypic measurements from plant images in an unsupervised manner. ara deep opsis was built around the deep-learning model DeepLabV3+ that was re-trained for segmentation of Arabidopsis thaliana rosettes. It uses semantic segmentation to classify leaf tissue into up to three categories: healthy, anthocyanin-rich, and senescent. This makes ara deep opsis particularly powerful at quantitative phenotyping from early to late developmental stages, of mutants with aberrant leaf color and/or phenotype, and of plants growing in stressful conditions where leaf color may deviate from green. Using our tool on a panel of 210 natural Arabidopsis accessions, we were able to not only accurately segment images of phenotypically diverse genotypes but also to map known loci related to anthocyanin production and early necrosis using the ara deep opsis output in genome-wide association analyses. Our pipeline is able to handle images of diverse origins, image quality, and background composition, and could even accurately segment images of a distantly related Brassicaceae. Because it can be deployed on virtually any common operating system and is compatible with several high-performance computing environments, ara deep opsis can be used independently of bioinformatics expertise and computing resources. ara deep opsis is available at https://github.com/Gregor-Mendel-Institute/aradeepopsis .