Breaking the field phenotyping bottleneck in maize with autonomous robots

Breaking the field phenotyping bottleneck in maize with autonomous robots | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Biological Sciences - Article Breaking the field phenotyping bottleneck in maize with autonomous robots Jason DeBruin, Thomas Aref, Sara Tirado Tolosa, Rebecca Hensley, and 8 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5187688/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Mar, 2025 Read the published version in Communications Biology → Version 1 posted You are reading this latest preprint version Abstract Understanding phenotypic plasticity in maize ( Zea mays L.) is a current grand challenge for continued crop improvement. Measuring the interactive effects of genetics, environmental factors, and management practices such as nitrogen rate (GxExM) on crop performance is time-consuming, expensive, and a major bottleneck to continued yield advancement. We demonstrate that a novel autonomous robotic platform, capable of collecting biologically relevant and commonly measured phenotypes, within a maize canopy at high-throughput, low-cost, and high-volume is now a reality. Field teams used multiple TerraSentia autonomous ground robots developed by EarthSense, Inc. (Champaign, IL) to capture data using a suite of low-cost sensors from nearly 200,000 experimental units, located at 142 unique research fields in the USA and Canada, across five years. Novel computer vision and machine learning algorithms, developed by EarthSense, Inc., analyzed these in-canopy multi-sensor data to deliver ground-truth validated plant height, ear height, stem diameter, and leaf area index at multiple time points during each season. We show the robot measured these phenotypes with high accuracy and reliability, at scales sufficient to functionally dissect interactions between genotypes and nitrogen rates in several environments. The results show that within-row autonomous field robots hold great promise to increase understanding of GxExM interactions in maize research and decrease the amount of human labor required for plant phenotyping. Biological sciences/Plant sciences/Plant breeding Biological sciences/Plant sciences/Plant physiology Biological sciences/Plant sciences/Plant development Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction Phenotyping of biologically relevant traits is a high priority to accelerate genetic gain in crop breeding programs 1 and close notable yield gaps 2 . Phenotyping in the field, and more specifically in the target population of environments 3 is critical to capture the genotype by environment (G x E) interactions that define phenotypic outcomes 3,4 . Currently, high-throughput genotyping is routinely used in maize ( Zea mays L.) breeding programs to aid marker assisted selection and whole genome prediction 5 . However, the scale of genomic data now far exceeds phenotypic data 6 , from the perspectives of both data quantity and cost per data point. Phenotypic data are essential to provide context to genomic data 4 and currently present a bottleneck in research and crop improvement activities 7,8 . Although modeling efforts aid in predicting phenotypes from genotype, they typically rely on training data for phenotypes that are limited to either 1) highest priority traits such as grain yield or 2) can be captured at large scale for low cost 9 . Phenotyping for component traits often adds key biological insights to understand yield outcomes and GxE 10,11 , but these are often difficult and time consuming to measure. Plant breeding companies focus on the continued improvement of maize genotypes through testing, advancement, and recycling of germplasm through their testing pipeline 12 . How these new genotypes interact with management factors once in the hands of farmers is of considerable interest as well and must be assessed through small plot and on-farm trials. Management factors such as plant density 13 and nitrogen management 14 can be applied to more fully understand Genotype by Environment by Management (GxExM) interactions 4 . These efforts require phenotypic characterization of the crop throughout the season to specifically determine the morphological changes that occur and further explain grain yield outcomes under a variety of environment and management conditions. Substantial resources are required to collect relevant morphological phenotypes such as stem diameter, leaf area index (LAI) and plant biomass 15, 16,17,18 . Depending on the trait of interest, a physical measurement may require 10 or more minutes of human effort to collect a single datapoint. Time and labor constraints mean that field scientists must limit the number of treatments, genotypes, and locations where morphological traits are collected further contributing to the phenotyping bottleneck observed in maize research trials. In general, currently available maize phenotyping methods are not capable of producing a complete assessment of plant growth processes which leads to an incomplete understanding of GxExM. Unmanned aerial vehicles have been deployed at a broad scale to collect RGB, multi-spectral, hyper-spectral and thermal data. Images from these flights capture biologically relevant traits such as plant height 19 , leaf area index 20 , and NDVI 21 . Data fusion approaches have been implemented to integrate signals captured from different sensor types, which partially overcomes the problem of canopy saturation for some of the most informative spectral bands 22 . However, in-canopy traits, such as ear height, stem width remain challenging, if not impossible, to predict from above-canopy phenotyping tools. Deployment of compact, in-canopy robotic platforms carrying cameras and sensors is an active space of development to unlock access to below-canopy phenotypic traits at similar scales to above canopy trait collection. Ground based, under-canopy, phenotyping platforms have taken various forms. Initial developments were wheeled but human-pushed 23 tractor carried 24 or motor propelled but human driven 25 platforms have also been demonstrated. More recently self-propelled platforms capable of running between crop rows and require little human operation 26 have also been demonstrated. The last category of ground robots, motorized and autonomous, offers the greatest flexibility and best opportunity to enable parallel, simultaneous operation of multiple robots for high-throughput phenotyping. Ground-based platforms can measure several traits from either maize or sorghum. Examples include plant height and volume 24 , stem diameter 27 , stem strength 28 , and leaf area index 29 . However, each of these leveraged a lower-throughput system that required substantial human assistance (i.e. pushing or driving through the platform through the field) to collect the required information for trait calculation. For this reason, automating in-row, under-canopy, self-propelled systems for the measurement of phenotypic traits can have tremendous value for increased throughput with reduced human labor. Autonomous robotic platforms for under-canopy phenotyping have become more available for use by the practitioner with examples including TerraSentia 26 (Fig. 1 A) and MARS 30 . While proof-of-concept demonstrations for collecting new phenotypic traits continue to increase, there are few examples of autonomous robots deployed at a scale sufficient to enable phenotyping required for a large-scale breeding, trait evaluation, or agronomic testing program. A single example leveraged an autonomous robot to collect LiDAR scans on 2103 experimental units with maize hybrids 31 . Novel latent-space (machine learning based system to characterize and discriminate differences among a series of image sequences for a set of treatments 32 ) phenotypes were extracted from these scans that showed heritabilities as high as 0.44 31 . A significant hurdle to deployment of self-propelled under-canopy phenotyping robotic platforms is reliable in-row navigation under real-world, variable, field, and crop-canopy conditions. This problem has been studied with suggested solutions by several groups 33,34,35,36 . The primary challenge in autonomous navigation under the crop canopy is that Global Navigation Satellite System (GNSS), also known as Global Positioning System (GPS) does not provide reliable enough localization for navigation of these ground-based within-row robots, even with the use of augmentation protocols such as Real-Time Kinematic (RTK). To overcome this challenge, algorithms and systems are required that are capable of row-following using other exteroceptive sensors such as LiDAR and cameras 37,38,39,40,41 . Autonomous robots leveraging such algorithms can ensure reliable under-canopy in-row navigation, and can handle obstacles in real-world fields, such as weeds, ditches, ruts, etc. However, prior to this study, such autonomous robotic systems were not deployed at scale sufficient to establish their use in modern breeding and agricultural product development pipelines. Our objectives were 1) determine whether autonomous ground based robots could be used to collect phenotypic data at a scale required for a large commercial corn breeding and research operation, 2) assess whether multi-sensor data collected from underneath the maize plant canopy by these robots could accurately estimate phenotypic traits that are commonly collected in maize hybrid development and associated with grain yield, and 3) investigate GxE and GxExM interactions for grain yield in different experimental designs to gain new insights into how maize hybrids respond to variation in nitrogen supply, planting density, and precipitation during the growing season. Results and Discussion Small Under-Canopy Phenotyping Robots have a High Deployment potential Between 2019 through 2023, through the deployment of 3–5 TerraSentia robots (EarthSense, Inc. Champaign, IL) per year by multiple individuals located in various parts of the United States and Canada (Fig. 1 B) we achieved a collection of multi-sensor data from 198,249 maize plots (experimental units, or EUs) (Fig. 1 C) (5.4 to 9.4 m length by 1.52 m width). The data were collected across 142 unique locations (Fig. 1 D), an average of three to four times each, resulting in 418 unique location by time data collections. The compact size and limited weight of the TerraSentia robots, relative to other ground based autonomous robots, was one of the main factors that allowed for broad deployment across many locations and across experimental units within a location 30 . In addition to size, operators reported continued improvement during the study years of greater robot reliability (i.e. maintain wi-fi connection to tablet, maintain cool operating temperature, hot-swapable batteries), and a simpler user interface to engage more operators (based on a survey of operators). Critically important to high-throughput data collection was the significantly improved autonomous navigation. A typical daily collection could accumulate to a total driving distance between 10 and 11,000 m. Beginning from 27 m in 2019, the average distance between human interventions increased to a peak of 3,629 m of unassisted autonomous navigation between human interventions by 2023 due to improved algorithms 37,38,39,40,41 . This demonstrated for the first time the feasibility of under-canopy robots for high-throughput phenotyping and other applications. From the imaging and LiDAR data, plant height, stem diameter, and leaf area index or ear height were calculated for each of the 132,176 experimental unit, delivering nearly 400,000 individual phenotypic trait measurements. During the multiple years of data collection, the number of experimental units with video data increased 24X and location number increased 20X (Fig. 1 D). A requirement for any high-throughput system to be useful is the collection of high-resolution images that facilitate computer vision and machine learning 42 . Here we note the marked improvement between 2019, when traits were estimated for 982 (36%) of the 2731 experimental units with multi-sensor data, to 2022 when 98% of the 67,148 experimental units with multi-sensor data successfully delivered at least two traits. The dataset collected in 2022 exceeds by an order of magnitude the number of phenotyped experimental units from field-grown maize using UAVs 22 , and by two orders of magnitude the largest phenotyping effort with UGVs 26,43 . Case studies for high-throughput phenotyping in agriculture From this large set of robotically collected data, we have selected five case studies to demonstrate the capability of the TerraSentia phenotyping platform to deliver high volumes of phenotypic traits. These traits both replace commonly manually collected traits, but also provide new opportunities to collect difficult phenotypic data to inform GxExM in the context of a commercial maize breeding and agronomic research program. Case study I. Replacing common, but manual trait collection in a maize breeding program. A commercial program evaluates hundreds of crosses at multiple locations each year 12 , with varying levels of biotic and abiotic stress, with the goal of finding high performing genotypes with acceptable traits. The height of the maize ear is a trait of particular interest due to correlation with lodging risk 44 . Previously this trait was collected manually using a measurement stick and data collection on a few plants within each experimental unit 45 . This method requires at least one individual to enter each experimental unit and measure from the soil surface to the nodal attachment point of the ear, typically for one to four plants, followed by manual data entry. This process requires approximately one to two minutes of human effort per experimental unit. It is important to note, it is not possible to measure maize ear height using above-canopy imagery collected with drones, let alone aircraft or satellites. With the TerraSentia robots, we were able to deliver ear height measurements for 4826 experimental units across five research locations (Table 1). The ear-height algorithm detects the ear, determines upward or downward orientation, and then identifies the node attachment point to the stem. The automated measurement of ear height in this experiment was achieved with approximately 20 hours of in-field data collection, a significant reduction compared to an estimated 150 hours of human labor that would have been required to complete equivalent manual measurements. TerraSentia reported ear height ranged from 59 and 152 cm depending on the location (Table 1). Ear height values are comparable to those obtained from manual measurements of ear height collected from genetically diverse maize populations grown over multiple years and locations in the US 45,46 . A within-location heritability estimation was calculated for each of the five locations utilizing the best linear unbiased estimates (BLUEs) of germplasm effects by trial and ranged between 0.43 and 0.67 demonstrating a strong genotypic response to the trait among the various locations. One experiment was common across three locations and produced a heritability estimate of 0.66. By collecting variable ear height data across a large number of locations and genotypes, we show that the rover can feasibly replace manual data collection for important agronomic traits like ear height and produce data with high heritability both within and across locations that can be selected upon. Table 1 Descriptive statistics and heritability for maize ear height collected in a pre-commercial breeding program using the TerraSentia robot at five locations in 2022 in the Midwest USA. Location Observations Ear Height (cm) Heritability no. minimum maximum mean ratio Griswold, IA 803 62 148 118* (9)** 0.64 Prairie City, IA 757 59 134 110 (7) 0.48 Cameron, IL 1004 64 152 120 (10) 0.62 Table Grove, IL 976 61 143 111 (8) 0.67 Windfall, IN 1286 59 136 108 (8) 0.43 Across locations*** 3484 59 152 111 (5) 0.66 * Single location minimum, maximum, and mean are best linear unbiased estimates. Across location minimum, maximum, and mean values are best linear unbiased predictions. ** Value in parentheses is the standard deviation *** Includes Cameron, IL, Table Grove, IL, and Windfall, IN Case study II. Multi-factor experiments using component trait phenotyping. Agronomic management choices such as planting density and nitrogen rate are ways farmers can optimize crop productivity and profitability at the field and farm level. Small-plot research trials are leveraged to study multiple management factors in multiple combinations to understand the main effects and potential interactions. In this experiment, six hybrids were evaluated at one location in four replicate plots, in each combination of four levels of N fertilizer and four planting densities, for a total of 384 experimental units. The experiment was established to estimate grain yield from these treatment combinations and the TerraSentia robot, operated by one individual, enabled a deeper characterization of how crop morphological traits responded to the treatments. Nitrogen rate and plant density interacted, exhibiting greater leaf area index (LAI) ( p ≤ 0.01) and lower plant height ( p = 0.01) as plant density increased across treatments (Fig. 2A). While LAI was greater with higher N rates the degree of response from LAI to N rates was lower for the lower plant density treatments (Fig. 2B). Compared to LAI, plant height was greater with increasing plant density and interacted with N rate (Fig. 2C). Both plant height and LAI in maize have been previously reported to respond to nitrogen rate and plant density 16,17 . Here, we observed that these traits showed distinct responses to plant density in the unfertilized compared to fertilized plots, with plant height being progressively reduced by each increase in plant density, whereas leaf area index was maximized at a density of 9.3 plants m − 2 . Stem diameter was smaller at greater plant densities regardless of N treatment (Fig. 2D), in line with prior observations 16,47 . The TerraSentia platform offered a dramatic increase in the efficiency at which LAI and stem diameter were measured from these 384 experimental units. On average, multi-sensor data was collected on 250 EUs hr − 1 . Approximately 1.6 hours was spent by a single person with a robot to collect the multi-sensor data that, after processing, provided measures of stem diameter, leaf area index, and plant height. This compares to the approximately 192 hours of human capital that would have been required to collect these traits manually (30 minutes per experimental unit) across three separate operations: 1) remove leaves from four random plants and scan through a leaf area meter in the lab, 2) collect plant height with a measuring stick on four plants and 3) measure stem diameter on ten plants with calipers. Simultaneous measurement of these three traits has already been demonstrated to predict biomass of sorghum 43 . A robotic phenotyping platform thus makes a number of significant contributions to small-plot, multi-factor research trials. First, the number of treatments can be increased, as the desired intensity of phenotyping can be applied to the entire experiment rather than being limited by the human labor available for phenotyping. Second, additional locations can be phenotyped as well, to bring in the treatment-by-location interactions that are critical for broad applicability of research results. Labor constraints limit manual data collection of a few individual plants per plot, whereas the robot collects data from most, if not all, of the plants in each plot. Due to the number of people taking measurements or collecting samples there is also a substantial opportunity for error or variation between data collected by the individual people, limiting the overall data quality and potential usefulness. Identical robotic phenotyping platforms deployed across locations remove many of these measurement errors and facilitates the potential to combine datasets across locations, seasons, and across research groups in meaningful ways. Case study III. Phenotyping genetic diversity for N response traits. Breeding programs often use managed environments for evaluating germplasm responses to key drivers of productivity, such as soil nutrient supply and water availability. Managed environments enable scaling of precision phenotyping to reveal genetic variation under stress conditions that may occur in the target population of environments 12 . The University of Illinois maintains a field location where the field soils have been managed to be limiting for nitrogen supply in an otherwise high yield potential environment. This field site has previously enabled evaluation of nitrogen-responsive phenotypes in maize at population scale 22,48 . The TerraSentia robot was deployed to phenotype replicated plots of at least 44 experimental hybrids in both 2022 and 2023, where each hybrid was grown with either no N fertilizer or 200 kg ha − 1 of N side-dressed at the V3 growth stage. In 2022, data were collected at two times post-flowering (July 22 and August 17) from 312 plots and in 2023 at the R2 growth stage (July 25) from 480 plots. Case Study II indicated that LAI is the only robot produced trait that showed a significant hybrid by N-rate interaction (Fig. 2A). Figure 3A shows the variation in LAI values for the 44 hybrids grown in 2023. Among the 217 total N-response plots, LAI values were greater with nitrogen application for all but three, making LAI measurements an accurate classifier for N treatment regardless of genotype. Each hybrid also displayed a positive average N response for LAI, which ranged from 1.12 to 1.47. Comparisons of LAI values for the six hybrids with the lowest ( 1.48) N response indicates the most responsive hybrids have both lower LAI at low N (mean 3.64 versus 4.01) and higher LAI at high N (mean 5.28 versus 4.83). Figure 3B shows the results from correlation analyses among the robot collected traits at different times over two seasons, ground truth data for plant height and ear height, and component traits for nitrogen use efficiency (NUE) measured at physiological maturity. As indicated from Fig. 3A, among all factors considered, LAI has the highest positive correlation with N rate. Although significant in both years, the stronger correlation between LAI and N rate in the July 2023 compared to August 2022 dataset likely reflects LAI values being collected near their peak values before the onset of nutrient remobilization and leaf senescence. The moderate correlation between LAI values collected in August 2022 and July 2023 for the same hybrids indicates some consistency across seasons, although the rank correlation among genotypes was not statistically significant. The July 2023 LAI values were positively correlated with each of the NUE component traits known to increase with N supply, and negatively correlated with nitrogen utilization that decreases with fertilization. Despite being closer in time to final harvest, the August 2022 LAI values showed only a moderate correlation with only grain yield, suggesting that canopy photosynthetic capacity earlier in the season is driving these relationships. In this experiment, LAI was not correlated with either ear or plant height, which is consistent with LAI capturing spectral rather than architectural features of the crop canopy. Trait values collected with the robots were highly correlated (r 2 > 0.93) with ground-truth measures for both ear and plant height from a subset of the 2022 plots, further demonstrating the accuracy of the onboard algorithms to derive these traits when challenged with a wide range of phenotypic variation due to the interaction of both genetic diversity and N supply. Furthermore, as anticipated for height traits with high heritabilities 31 , robot phenotypes delivered by TerraSentia showed strong correlations across measuring times and seasons. Among NUE component phenotypes, ear and plant height only showed a significant positive correlation with stalk biomass, which increased with data collection later in the season nearer to final harvest. The stronger correlation of ear height over plant height for stalk biomass is likely due to the higher proportion of stalk biomass accumulated below the ear node compared to above the ear. The relationships between the robot measured traits and human measured stalk biomass indicate the potential to measure stalk biomass in an automated and non-destructive manner. In addition to height and LAI, the robots also delivered average stem diameter for each plot. Stem diameter was found to be correlated with LAI, but not to height, and in July 2023 both LAI and stem diameter showed very similar associations with NUE traits. Measuring stem diameter in a high-throughput and automated fashion opens the opportunity to leverage allometric measurements for estimating total plant biomass 15 . Figure 3C illustrates how a linear regression model including ear height, plant height and their interaction term provides good predictions for genetic variation in stalk biomass for both fertilized and unfertilized plots from the 2022 experiment. The robot traits were collected with a single robot in under four hours with one person, and much earlier in the season, compared to approximately 100 person-hours spent on measuring stalk biomass. The estimated broad-sense heritability for stalk biomass in the 2022 experiment was 0.89 when measured manually and 0.76 when predicted from the TerraSentia robot trait model, indicating that the 25-fold reduction in effort with automated phenotyping for stalk biomass may be achieved without a major loss in potential genetic gain. Case study IV. From research trials to on-farm system evaluations. Genotype selection coupled with management choices and environmental events define the outcome in a farmer’s field. This interaction is best understood by phenotyping the genotype x management decisions within the relevant field environment 4 ; however, this is difficult due to field size, accessibility and heterogeneity of the field to determine where trait collection should occur. We were able to cover large transects of the field (Fig. 4A) to capture the influence of soil type (E) on the phenotypic traits which were expected to primarily respond to nitrogen rate (M) and hybrid (G). The LAI trait measurement from the Nebraska and Indiana fields delivered several useful insights. Canopy size was greater at the irrigated Nebraska location compared to the Indiana location for all N rates (Fig. 4B). Specifically of interest was the 0 kg N ha − 1 rate as this represents the potential for the soil to provide N to the crop via available NO 3 -N or through the process of mineralization. An assessment of grain yield at a zero N rate has been shown to be useful for N rate recommendations 49 . As expected, LAI was greater with higher N rate but did not respond to N rates greater than 134 kg N ha − 1 at the Nebraska site, whereas 202 kg N ha − 1 was needed to reach a maximum LAI value at the Indiana site. For total N rates up to 202 kg ha − 1, plots that received split N applications trended to exhibit higher LAI values and grain yields compared to plots that received all N at planting. Grain yield was linearly correlated to LAI (Fig. 4C) and did not exhibit any saturation effect. Given the size of these research plots (740 m long in Indiana and up to 1330 m long in Nebraska; Fig. 4A) collection of LAI, stem diameter and plant height would have been economically and physically infeasible with conventional human measurement methods. Greater leaf area index, a trait strongly sensitive to the management and environment specifically through availability of N and water, brings the potential benefit of greater N storage within the plant biomass 18 which is expected to be advantageous later in the season as a reserve of N to complete grain filling. However, greater LAI could potentially come at a cost if water is limited and the greater leaf area results in a transpiration rate in excess of water available in the soil. Ability to non-destructively and efficiently phenotype LAI throughout the season provides insight to canopy expansion as a result of water and N availability and enables a calculation of grain yield production efficiency relative to investment in leaf area. Case study V. Deeper understanding of hybrid evaluation trials with previously elusive traits. Interactions between genotypes and the environment result in the phenotype present at a particular location 3 . Utilization of multiple locations provides additional information on G x E interaction. A particularly important environmental factor known to influence G x E is precipitation. In 2022, LAI, plant height, and grain yield were collected for a set of precommercial hybrids in Ontario. Rainfall in July and August was below the 20-year historical average at each of these locations but was more limited at Atwood and Hensal compared to Harriston. Traditional methods of LAI data collection are very labor intensive and are unfeasible for collecting LAI in large maize breeding programs. The robot offered the novel opportunity to collect LAI for multiple research locations multiple times throughout crop development, understand how change in LAI through the maize lifecycle is affected by the environmental conditions, and enable investigation of genotypic variation in this response between pre-commercial lines. There was a notable difference in LAI among locations. The precommercial population at Harriston had an average LAI of 6.5 while maximum LAI was 5.25 at Atwood and 5.15 at Hensall (Fig. 5A). Plant height was lower at the sites that received lower precipitation with an average height of 226 cm at Harriston versus 212 cm at Atwood and 208 cm at Hensall (Fig. 5B). Yield was lower as well, with Harriston averaging 14250 kg ha − 1 while Hensall and Atwood averaged 12250 kg ha − 1 each (Fig. 5C). Principal Components Analysis (PCA) indicated a strong correlation between grain yield and maximum LAI as well as with LAI at R5 (Fig. 5D). Greater LAI at the Harriston location indicated the potential for greater solar radiation interception as well as more N storage in the leaves to support kernel filling through remobilization. Notably, LAI has not been reported as a phenotype under active selection in breeding trials. Only recently has LAI been collected via UAS during the growing season on a breeding population 10 . In this novel work they demonstrate how QTLs associated with LAI explain a substantial amount of the yield variation indicating the potential benefit of using LAI as a selection index in maize breeding. Expanding LAI phenotyping to multiple locations across a more diverse set of hybrids and trials will provide insight into how a specific genotype achieves observed grain yield and enable its future use as an informative selection metric. Robotic platforms that facilitate multi-time point data collection during the season provide the opportunity to define rates of canopy expansion and biomass accumulation as well as end-of-season canopy loss. Measurement of LAI and plant height during the season provides an indication of when stress is beginning to occur and to what extent. Identifying the times when stress occurs is useful for grouping locations with similar stress patterns into a combined multi-location analysis 50 . These time course trait measurements are potentially the phenotypic descriptors to inform “enviromics” to enable further understanding of the genotype-by-environment interactions. Screening a large breeding population is required to leverage the genetic diversity available for crop improvement 51 . A critical component to the success of high-throughput phenotyping across large populations is the capacity not only of the phenotyping platform but the capacity to transfer larger data sets, extract data from imagery and other sensor data and organize trait data emerging from the phenotyping pipeline 8 . The objective for high throughput phenotyping is to create a virtuous cycle that automates crop phenotyping, trait computation, analysis and selection followed by the next season of phenotyping 42 . Future applications: Automating “old” traits, developing new traits, and increasing scale. Deployment of the TerraSentia robots at multiple locations with different types of GxExM experimental designs between 2019 and 2023 demonstrates the potential to deploy ground based autonomous robots at sufficient scale to provide data at geographic scales that match data generated by other sources such as UAV, combine harvesters as well as from genotyping. Robotic, in-canopy trait collection opens an entirely new area of research into the linking of these in-canopy traits to marker association, crop selection, and predictive breeding to continue to investigate and elucidate G x E x M effects. Additionally, we demonstrate that the trait measurements derived from the in-canopy multi-sensor data are agronomically relevant and useful to understand crop growth, crop stress, and genetic diversity. Comparison of the data collected via human measurement on multi-rate N trials or plant density trials with the robotic data shows that there are strong similarities in the response of plant height, stem diameter, leaf area index, and ear height estimated by both methods. Each of these traits have historically been collected at lower coverage through less efficient manual methods. Moving forward, the potential exists for phenotyping other rarely studied traits captured via in-canopy sensing, such as the influence of brace root characteristics (size, width, origin node, diameter of ground coverage) on lodging risk 52 . Automated counting of tillers that form around the base of the main stem of maize will also become a phenotypic trait potentially scored for multiple hybrids at multiple locations to determine their relationship to grain yield 53 . The list of novel traits that will become routine will continue to increase with additional sensors, robotic platforms, and data analytics. Over the four years of operation, TerraSentia robots navigated along each corn row autonomously with a user simply following the robot to facilitate any crash recoveries and turning at the end of the field to begin the next pass of data collection. Future capabilities, currently under development and testing, will bring reliable full-field autonomy to data collection. The robots will navigate through the field autonomously staying between the crop rows and recovering independently when an obstruction occurs. Additionally, the robots will turn at the end of each pass autonomously by navigating between GPS waypoints prior to starting the next pass of data collection. Human involvement at this point will be minimal outside of delivering the robot to the field and exchanging batteries. When the potential to “swarm” robots becomes a reality, data collection rates can greatly increase as each robot is assigned a particular section of the field as their mission. The next generation of phenotyping traits will be delivered by autonomous, in-canopy robots. Through their capacity to digitize traits critical to crop performance, at high densities of measurement in both time and space and on many genotypes, phenotyping robots will contribute significantly to increasing grain yield, reducing yield gaps, and optimizing agronomic crop management. This technological advance will enhance both the productivity and sustainability of agriculture and the future of food production. Materials and Methods A ground-based robot, capable of driving between 0.76 m spaced maize rows (Fig. 1A) was developed by EarthSense, Inc. (Champaign, IL) beginning in 2017. A collaborative effort between Corteva Agriscience (Johnston, IA) and EarthSense began in 2019 and a collaboration with the University of Illinois (Champaign-Urban, IL) and EarthSense started in 2021 to determine whether phenotypic traits important to maize growth could be measured from in-canopy multi-sensor data collected on this robot, named TerraSentia. TerraSentia is a configurable, terrestrial robotic platform that autonomously performs under canopy data collection, running between rows of maize or other row-crops. On board the TerraSentia, a number of sensors are mounted including three cameras for RGB video capture, lidar for 3D point clouds, fisheye lens cameras for hemi-spherical photography and depth cameras that provide RGB with depth information. The robot can complete an experimental unit (5.3 m long) in about 6 seconds capturing data for every plant in the unit and autonomously runs for typically 180 m without manual intervention (i.e. user takes control of the robot using a tablet to then drive the robot) once the user has placed the robot in-between the tows and started the navigation. The TerraSentia robot is capable of autonomous row-following, meaning it can effectively navigate long distances down rows of crops with minimal input from the user. Row-following is enabled by a combination of LiDAR sensors, cameras, and GPS mounted on the robot (Fig. S1A). The onboard computer processes the data from these sensors to map the locations of the crop rows on each side and the orientation of the robot relative to the rows. This information about the robot and its surroundings is then used for path planning and decision making. The TerraSentia robot is equipped with three adjustable cameras, one on the front and one on each side. The unit is also equipped with a rear LiDAR sensor for scanning the regions above and to the side of the robot. Depending on robot configuration, the top of the robot can be equipped with an upward facing camera with fisheye lens for LAI data collection, or front facing depth camera angled up at 25 degrees. These sensors are used for data collection and trait estimation. The robot is configurable and maneuverable with a user-friendly tablet interface (Fig. S1C). Details about a data collection mission, such as the dimensions of the field and the planting date, can be specified by the user. Furthermore, the camera angles, resolutions, fields-of-view, and frame rates of the cameras can be programmatically customized for specific traits. These features allow the robot to simultaneously record data for at least five distinct traits (e.g. stem diameter using the left-side camera, plant count on the right-side camera, LAI using the hemispherical lens on top of the robot, plant height and canopy architecture information using the rear lidar) at a given time, with accurate metadata for future reference. Data processing: Videos and sensor data are collected by the robot along with metadata (columns, ranges, crop, desired trait, etc.) input from the user are uploaded to the cloud after the robot returns to a base location with stable electrical power (to recharge the robot) and internet connectivity (not in the field). The videos are pre-processed with automatic checks for quality and integrity and run through an automatic plot splitting algorithm (Fig. S1D). The automatic plot splitter uses a deep learning model to look for periodic gaps visible in the lidar scans of the canopy, gaps which occur between the experimental units (Fig. 1 B) being scanned. These outputs of the automatic data verification and plot splitting processes are manually reviewed to ensure data quality (e.g. an error in the plot split can lead to catastrophic “off-by-one” errors in the alignment of plot results). Alignment of the split plots with their associated metadata (column/range) is compared to GPS coordinates to further verify plot assignment. The multi-sensor data are then fed into the appropriate algorithm, the raw results per scan/frame are aggregated and post processed into the final estimates of the desired traits for each experimental unit, which are then reported back to the user. The plant-height algorithm uses the point cloud of the vertically scanning lidar to extract the highest measured points as the height of the maize plant 54 . This is calibrated to correspond to the tassel height in maize. Tassel height and uppermost leaf height measured manually generally correlate extremely well so validating versus uppermost leaf height is also effective. Stem width uses RGB imagery combined with distance-to-plant-row estimates to measure the diameter of the maize stems. The stems are detected using Mask R-CNN image segmentation algorithm. Stem widths in pixels are extracted at several sampling points on each vertical stem. These pixel widths are converted to real widths using the distance to stem, estimated from the robot’s lidar and odometry data 55 . The correlation to manual data can be complicated by the millimeter scale measurement error and limited measurement window (young stems tend to have many leaves blocking the stem and later stage maize tends to have drooping leaves which block the stems). The ear height algorithm processes data gathered by an RGB + Depth sensor. A machine-learning based object recognition algorithm detects maize ears and then uses the depth information to estimate geometric properties of the maize ear in image space. The geometry of the maize ear is then adjusted by accounting for the location and orientation of the camera, which are informed by IMU sensors in the robot and in the camera itself. From this adjusted maize ear geometry, the height of the maize ear can be computed. The reported ear height averaged over a number of ears in the plot correlates well with manual measurements of ear height (Fig. S2.A). The LAI algorithm processes data gathered by an upward facing camera with a fisheye lens capturing hemispherical imagery. The algorithm identifies every pixel corresponding to blue light penetrating through the canopy from the sky (blue light is the opaquest wavelength for plants), and then estimates LAI from the distribution of sky pixels using Miller’s integral formula. 56 This method is similar to sensor readings provided by commercial LAI instruments, such as the Licor LAI-2200C (Lincoln, NE). This method has also been verified by 57 and specifically for maize by 57,58 . Validation dataset collection For each of the traits currently measured from multi-sensor data collected by the autonomous robot, a specific set of ground truth data were collected manually. Stem diameter was measured manually using a digital caliper at the R1 (silk appearance growth stage) 59 . The caliper was placed around the stem in the same direction as the maize row and the diameter was recorded to the nearest tenth mm. For stem diameter, 44 experimental units having between 10-40 plants were measured with every plant contributing a measurement. Measurements using the TerraSentia robot were well correlated (p<0.001) with human-acquired caliper traits with an R 2 value of 0.89 and a slope of 1.02 (Fig. S2B). Plant height was measured manually using a tall measurement pole at the R2 (10-14 days after R1) growth stage. The height was recorded to the top of the tassel and the top leaf attachment point (leaf node) from the soil level for 27 experimental units having about 40 plants with every third plant measured. The correlation between the manually measured tassel height and leaf node height was extremely high (R 2 =1.00) with the leaf heights being slightly lower than the tassel heights (the robot height measurements are closer in height to the tassel measurements than the leaf node measurements). Measurements delivered by the TerraSentia robots, using LiDAR data were well correlated (p<0.001) with human-acquired plant height measurement with an R 2 value of 0.97 and a slope of 0.97 (Fig. S2D). Ear height measurements were validated on 79 experimental units having approximately 40 plants each. Manual measurements were carried out at R5 (kernels are dented on the ear 59 ) with every third plant measured manually. The distance from the soil surface to the ear attachment point on the stem (ear node) was recorded using the same pole used for plant height. Measurements delivered by the TerraSentia robots were well correlated (p<0.001) with human-acquired plant height measurement with an R 2 value of 0.87 and a slope of 0.83 (Fig. S2A). Leaf area index was validated by recording data in each of the 50 experimental units approximately two weeks after R1, to collect imagery with a 720p camera with a hemispherical lens. Following robotic data collection, the following process was conducted for manual LAI measurement. First, the length of rows on either side of the robot were measured to the nearest cm. The row spacing was 76 cm and thus the exact area could be calculated. Following row measurement, the number of plants in each row were counted to establish the plant density. Following row measurement and plant counting, all of the green leaves, that were orientated upward ≥ 90 o , were removed from four randomly selected plants (two from each row). These leaves were placed in a large plastic bag and transported to the lab. At the lab leaves were run through a LI-3100C leaf scanner (Li-Cor, Lincoln, Nebraska, USA). This machine determined the total leaf area in square cm for the four plants. This value was divided by four to determine an average leaf area per plant and further divided by 10,000 to produce leaf area per square meter. Leaf area per plant was multiplied by plant density to calculate leaf area index (leaf area m -2 land area m -2 ). A total of 50 experimental units were used to validate the TerraSentia LAI measurement. The correlation between robot measured LAI and human measured LAI was significant but the R 2 value was lower (0.5) than the height and diameter traits (Fig. S2C). Case study 1: As part of the commercial breeding program of Corteva Agriscience trials are grown at several locations to expose pre-commercial lines to environments and management to determine crop response prior to further advancement. Five locations near Griswold, IA, Prairie Cty, IA, Cameron, IL, Table Grove, IL and Windfall, IN were selected. The TerraSentia robot, carrying the Real Sense camera, was used approximately three weeks prior to combine harvest to collect video images that would allow for ear detection and ear height calculation. A human operator followed approximately two meters behind the robot to reorientate following any crashes and to save the videos at the end of each pass and turn the robot on the ends to start the next pass. Data were analyzed by looking at genotypic performance for the trait of interest, in this case ear height, both within single locations and across locations. For by-location analyses, we generate the best linear unbiased estimates (BLUEs) of germplasm effects by trial using ASREML 60 . To generate BLUEs, we first fit the spatial model (AR1 by AR1) with genotype effects as random. Next, we set the genotype to a fixed factor along with the other variance parameters and rerun the model. For across location analyses we generate the best linear unbiased predictors (BLUPs) using ASREML. In this model the location effect is considered a fixed effect, the genotype and the genotype by location interaction effects are considered random, and the residual variance is fixed at 1 and relies on the weighting for each genotype from the BLUE analysis. One experiment was carried out at three locations (Cameron, IL, Table Grove, IL and Windfall, IN locations) and was used for the BLUP analysis (Table 1). Heritability is calculated as the ratio of the genotypic variance to total phenotypic variance 61 . Case study 2: A multi-treatment, replicated, small-plot (5.4 m long by 3 m wide) trial was established at a location near Windfall, Indiana in a randomized complete block design. Treatments included a whole plot treatment of N levels 0, 112, 224 and 336 kg N ha -1 , a split-plot treatment of 4.4, 6.9, 9.4 and 11.9 plants m -2 and a split-split plot treatment of six Pioneer (Corteva Agriscience, Johnston, IA, USA) hybrids. This trial was planted on 20 May 2022 using a custom designed vacuum planter (Almaco, Nevada, IA, USA) and the nitrogen treatments were applied on 24 May 2022 using urea-ammonium-nitrate (UAN). The UAN was injected approximately 20 cm from each row of a four-row experimental unit. On 1 July, 29 July and 18 August the TerraSentia robot was used to collect RGB imagery and LiDAR data on all 384 experimental units. The left camera was set at -5 o to image the plant stems, the right camera was set a -30 o to image the soil to plant interface, the front camera was set a 0 o to collect basic plot survey data and the upward facing hemispherical lens was at 90 o to collect video for leaf area index. The robot was operated in row-following mode with an operator following approximately two meters behind the robot to reorientate following any crashes and to save the data at the end of each pass and turn the robot on the ends to start the next pass. Data from the 29 July collection were selected for analysis due to their close proximity (~2 weeks after R1 (silk appearance) when the canopy reaches maximum height and LAI. The analysis was conducted using Proc Mixed in SAS 62 . Fixed effects were N rate, Plant density, and Hybrid. Random effects were Replication x N rate, Replication x Plant density. Case study 3: Maize hybrids were evaluated as described in 63 within a nitrogen-responsive field site at the Crop Sciences Research and Education Center in Urbana, Illinois. The soil type at this field is a Drummer silty clay loam, pH 6.5, which provided 60-75 kg N ha -1 in each year of the experiment based on both soil testing and measures of plant N recovery from unfertilized plots. Hybrids were grown in a split-plot design, where the same hybrid was grown in paired plots (2 rows 5.3m long, 76cm row spacing) receiving either no fertilizer or N fertilizer applied at a rate of 200 kg N ha-1, as granular ammonium sulfate banded at the soil surface when the plants reached the V3 (3 leaf collars) growth stage. All hybrids were planted at a density of 86,000 plants ha -1 , with three replicates of each N treatment. Trials were planted on 7 May (2021), 12 May (2022) and 4 May (2023) using a GPS-guided vacuum planter (SeedPro, Almaco, Nevada, IA, USA). Phenotyping with the robot was performed as described above for Case Study 2, with data collections on 1 September 2021, 17 August and 9 September 2022, and 25 July, 2023. Ground truth measures of plant and ear height were made with a measuring stick on 5 plants per plot. Biomass and nitrogen concentrations of stalk plus leaves, cob and grain were determined from five plants of each plot harvested at physiological maturity (R6 growth stage, month of September) as described in 63 . Data analyses were performed in R Statistical Software 64 (R Core Team 2021) using the corrplot 65 and ggplot2 66 packages. Case study 4: During 2021 a multiple N rate by multiple hybrid experiment was established at two grower fields. One field was located in northeast Nebraska near the city of Homer and one field located in Southwest Indiana near the city of Owensville. The whole plot treatment was all N applied at planting or 76 kg N ha -1 applied at planting with the remainder applied at V5 (5 collared leaves present). The split-plot was five nitrogen rates of 0, 67, 134, 201 and 269 kg N ha -1 . The split-split plot was hybrid consisting of either P1197AM or P1359AM (Corteva Agriscience, Johnston, IA). This pattern continued to be replicated as many times as needed until the strip across the field was filled (Fig. 4). The experimental unit size was 5.4 m by 3 m. The number of experimental unit ranges were from as few as 108 (584 m) ranges long to as long as 250 (1,350 m) ranges long. Approximately two weeks after flowering (silk appearance) and pollen shed, the robots were used to collect leaf area index, stem diameter, and plant height data. The robot was run through the experimental units at a constant speed of 0.3 m sec -1 . The left camera was set at -15 o to video the plant stems, the right camera was set a -30 o to video the soil to plant interface, the front camera was set a 0 o to collect basic plot survey data and the upward facing hemispherical lens was at 90 o to collect video for leaf area index. Grain yield was captured using a specialized plot combine that measured the weight and moisture of the grain for each experimental unit and standardized the grain weight at 155 g kg -1 moisture. Data were analyzed using Proc Mixed in SAS 62 with State (Nebraska or Indiana), N rate and Hybrid treated as fixed effects. Random effects were N rate x Replication nested within the specific field strip, field strip nested within State and Range nested within specific field strip. Case study 5: A set of pre-commercial maize hybrids were evaluated at three locations near the towns of Hensall, Attwood, and Harriston all located in Ontario Canada. The robot was run through the experiment between four and six times depending on the location during the season between the V10 (10 collared leaves) and the R6 (maturity) growth stages. At each time point video images were collected using the hemispherical lens camera mounted on top of the robot. These videos were converted into a measurement of LAI. Plant height was also collected at each time point using the lidar mounted on the back of the robot. From this time course data, the maximum LAI (achieved at R1, flowering, and maintained for approximately two weeks after flowering) could be determined for each location. An additional measurement, occurring on 23 September of 2022, was common among locations and selected to represent potential loss of green leaf area as the crop senesced prior to harvest which occurred 15-30 days later depending on the location. Weather data (rainfall and temperature) from 2022 were collected using weather station data within 10 km of the individual location. Grain yield was captured using a specialized plot combine that measured the weight and moisture of the grain for each experimental unit and standardized the grain weight at 155 g kg -1 moisture. Raw LAI data were reviewed and values greater than three standard deviations away from the mean were considered outliers and removed from the visualization and analysis. Grain yield, plant height, and LAI at the R5 growth stage were subjected to a principal component analysis using R 64 . Declarations Acknowledgements The authors acknowledge Jan-Michael Schulze and Fred Gruis for initializing the collaboration and guiding the budgetary support for the yearly testing. The authors specifically thank Logan Anderson, Carmen Trainer and Melissa Grafton for their commitment to operate the robots at a number of USA field trial locations during multiple years of testing. Phenotyping efforts at the University of Illinois were supported in part by the National Science Foundation Science and Technology Center for Programmable Plant Systems (CROPPS, NSF award DBI-2019674), the Denton E. and Betty Alexander Professorship in Maize Breeding and Genetics, and a graduate fellowship from the Illinois Corn Marketing Board to Catherine Li. Author contributions J.L. DeBruin: data collection, data organization, data analysis, and manuscript development T. Aref: image organization, data analysis, and manuscript development R. Hensley: data collection, data organization, and manuscript development H. Underwood: data collection, data organization, and manuscript development S. Tirado Tolosa: data collection, data organization, data analysis, and manuscript development M. McGuire: image organization, image analysis, and manuscript development C. Soman: image organization, image analysis, and manuscript development G. Nystrom: data collection, data organization E. Parkinson: data collection, data organization C. Li: data collection, data organization, data analysis, and manuscript development S. Moose: data analysis and manuscript development G. Chowdhary: image organization, image analysis, and manuscript development Competing Interest declaration J.L. DeBruin, R. Hensley, H. Underwood, and S. Tirado Tolosa declare no competing interests. G. Chowdhary, C. Soman, M. McGuire, and T. Aref declare no competing interests. S. Moose. C. Li, G. Nystrom and E. Parkinson: declare no competing interests. Acknowledgements The authors acknowledge Jan-Michael Schulze and Fred Gruis for initializing the collaboration and guiding the budgetary support for the yearly testing. The authors specifically thank Logan Anderson, Carmen Trainer and Melissa Grafton for their commitment to operate the robots at a number of USA field trial locations during multiple years of testing. Phenotyping efforts at the University of Illinois were supported in part by the National Science Foundation Science and Technology Center for Programmable Plant Systems (CROPPS, NSF award DBI-2019674), the Denton E. and Betty Alexander Professorship in Maize Breeding and Genetics, and a graduate fellowship from the Illinois Corn Marketing Board to Catherine Li. Additional Information Corresponding author: Jason DeBruin @ [email protected] References Basavaraj, P.S., Rane, J., Prathibha, M.D., Boraiah, K.M. & Kumar, H. Advances in high-throughput phenotyping of maize (zea mays l.) for climate resilience. Maize Improvement. S.H. Wani et al., (eds). (2023). (doi.org/10.1007/978-3-031-21640-4_12) Cassman, K.G. & Grassini, P. A global perspective on sustainable intensification research. Nature Sustainability 3:262-268 (2020). (doi.org/10.1038/s41893-020-0507-8) Cooper, M. & Messina, C.D. Can we harness “enviromics” to accelerate crop improvement by integrating breeding and agronomy. Front. Plant Sci. 12:735143. (2021). (doi:10.3389/fpls.2021.735143) Cobb, J.N., DeClerck, G., Greenberg, A., Clark, R. & McCouch, S. Next-generation phenotyping: requirements and strategies for enhancing our understanding of genotype-phenotype relationships and its relevance to crop improvement. Theor. Appl. Genet. 126:867-887 (2013). (doi 10.1007/s00122-013-2066-0) Merrick, L.F., Herr, A.W., Sandhu, K.S., Lozada, D.N. & Carter, A.H. Optimizing plant breeding programs for genomic selection. Agronomy 12:174 (2022). (https://doi.org/10.3390/agronomy12030714). Chawade, A., van Ham, J., Blomquist, H., Bagge, O., Alexandersson, E. & Ortiz, R. High-throughput field-phenotyping tools for plant breeding and precision agriculture. Agronomy 9:258 (2019). (doi:10.3390/agronomy/9050258). Minervini, M., Scharr, H. & Tsaftaris, S.A. Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Processing Magazine 126 (2015). (doi:10.1109/MSP.2015.2405111) Araus, J.L. & Kefauver, S.C. Breeding to adapt agriculture to climate change: affordable phenotyping solutions. Current Opinion Plant Biology 45:237-247. (2018). (doi.org/10.1016/j.pbi.2018.05.003) Reynolds, D., Baret, F., Welcker, C., Bostrom, A., Ball, J., Cellini, F., Lorence, A., Chawade, A., Khafif, M., Noshita, K., Mueller-Linow, M., Zhou, J. & Tardieu, F. What is cost-efficient phenotyping? Optimizing costs for different scenarios. Plant Sci. 282:14-22 (2019). (https://doi.org/10.1016-j.plantsci.2018.06.015) Blancon, J., Buet, C., Dubreuil, P., Tixier, M.H., Baret, F. & Praud, S. Maize green leaf area index dynamics: genetic basis of a new secondary trait for grain yield in optimal and drought conditions. Theor. Appl. Genet. 137:68 (2024). https://doi.org/10.1007/s00122-024-04572-6. Washburn, J.D., Burch, M.B. & Valdes Franco, J.A. Predictive breeding for maize: making use of molecular phenotypes, machine learning, and physiological crop models. Crop Sci. 1-17 (2019). (https://doi.org/10.1002/csc2.20052) Cooper, M., Messina, C.D., Podlich, D., Totir, L.R., Baumgarten, A., Hausmann, N.J., Wright, D. & Graham, G. Predicting the future of plant breeding: complementing empirical evaluation with genetic prediction. Crop and Pasture Sci. 65:311-336 (2014). (doi.org/10.1071/CP14007) Assefa, Y., Carter, P., Hinds, M., Bhalla, G., Schon, R., Jeschke, M., Paszkiewicz, S., Smith, S. & Ciampitti, I.A. Analysis of long term study indicates both agronomic optimal plant density and increase maize yield per plant contributed to yield gain. Scientific Reports 8:4937 (2018). (doi:10.1038/s41598-018-23362-x) Ransom, C.J., Clark, J., Mac Bean, G., Bandura, C., Shafer, M.E., Kitchen, N.R., Camberato, J.J., Carter, P.R., Ferguson, R.B., Fernández, F., Laboski, C.A.M., Myers, D.B., Nafziger, E.D., Sawyer, J.E. & J. Shanahan. Data from a public-industry partnership for enhancing corn nitrogen research. Agron. J. 113:4429-4436 (2021). (doi:10.1002/agj2.20812) Vega, C.R.C., Andrade, F.H., Sadras, V.O., Uhart, S.A. & Valentinuz, O.R. Seed number as a function of growth. a comparative study in soybean, sunflower, and maize. Crop Sci 41:748-754 (2001). (doi.org/10.2135/cropsci2001.413748x) Boomsma, C.R., Santini, J.B., Tollenaar, M. & Vyn, T.J. Maize morphophysiological responses to intense crowding and low nitrogen availability: an analysis and review. Agron. J. 101:1426-1452 (2009). (doi:10.2134/agronj2009.0082) Ciampitti, I.A., Zhang, H., Friedemann, P. & Vyn, T.J. Potential physiological frameworks for mid-season field phenotyping of final plant nitrogen uptake, nitrogen use efficiency, and grain yield in maize. Crop Sci. 52:2728-2742 (2012). (doi:10.2135/cropsci2012.05.0305) DeBruin, J., Messina, C.D., Munaro, E., Thompson, K., Conlon-Beckner, C., Fallis, L., Sevenich, D.M., Gupta, R. & Dhugga, K.S. N distribution in maize plant as a marker for grain yield and limits on its remobilization after flowering. Plant Breeding 132:500-505 (2013). (doi.10.1111/pbr.12051) Madec, S., Baret, F., de Solan, B., Thomas, S., Dutartre, D., Jezequel, S., Hemmerlé, M., Colombeau, G. & Comar, A. High-throughput phenotyping of plant height; comparing unmanned aerial vehicles and ground lidar estimates. Front. plant Sci. 8:2002 (2017). (doi:10.3389/fpls.2017.02002) Liu, S., Jin, X., Nie, C., Wang, S., Yu, X., Cheng, M., Shao, M., Wang, Z., Tuohuti, N., Bai, Y. & Liu, Y. Estimating leaf area index using unmanned aerial vehicle data: shallow vs. deep machine learning algorithms. Plant Physiology. 187:1551-1576 (2021). (doi:10.1093/plphyso/kiab322) Tamás, A., Kovács, E., Horváth, E., Juhász, C., Radócz, L., Rátonyi, T. & Ragán, P. Assessment of NDVI dynamics of maize (zea mays l.) and its relation to grain yield in a polyfactorial experiment based on remote sensing. Agriculture 13:689 (2023). (doi.org/10.3390/agriculture13030689). Nguyen, C., Sagan, V., Bhadra, S. & Moose, S.P. UAV multisensory data fusion and multi-task deep learning for high-throughput maize phenotyping. Sensors 23(4): 1827 (2023). https://doi.org/10.3390/s23041827 White, J.W. & Conley, M.M. A flexible, low-cost cart for proximal sensing. Crop Sci. 53:1646-1649 (2013). (doi:10.2135/cropsci2013.01.0054) Bao, Y., Tang, L., Srinivasan, S. & Schnable, P.S. Field -based architectural traits characterization of maize plant using time-of-flight 3d imaging. Biosystems Engineering 178:86-101 (2019). (doi.org/10.1016/j.biosystemseng.2018.11.005) Barker III, J., Zhang, N., Sharon, J., Steeves, R., Wang, X., Wei, Y. & Poland, J. Development of a field-based high throughput mobile phenotyping platform. Computers and Electronics in Agriculture 122:74-85 (2016). Zhang, Z., Kayacan, E., Thompson, B. & Chowdhary, G. High precision control and deep learning-based corn standing counting algorithms for agricultural robot. Autonomous Robots 44:1289-1302 (2020). Fan, Z., Sun, N., Qiu, Q., Li, T., Feng, Q. & Zhao, C. In situ measuring stem diameters of maize crops with a high-throughput phenotyping robot. Remote Sens. 14:1030 (2022). (https://doi.org/10.3390/rs14041030) Mueller-Sim, T., Jenkins, M., Abel, J. & Kantor. G. 2017. The robotanist: a ground-based agricultural robot for high-throughput crop phenotyping. Robotics Institute Carnegie Mellon University Paper-ICRA2017-Robotanist.pdf (cmu.edu) (verified 4-19-2024) Banan, D., Paul, R.E., Feldman, M.J., Holmes, M.W., Schlake, H., Baxter, I., Jiang, H. & Leakey, A.D.B. High-fidelity detection of crop biomass quantitative trait loci from low-cost imaging in the field. Plant Direct 2018:1-8 (2018). (doi:10.1002/pld3.41) Xu, R. & Li. C. 2021. A modular agricultural robotic system (MARS) for precision farming: concept and implementation. J. Field Robotics 39:387-409 (2021). (doi:10.1002/rob.22056) Gage, J.L., Richards, E., Lepak, N., Kaczmar, N., Soman, C., Chowdhary, G., Gore, M.A. & Buckler, E.S. In-field whole-plant maize architecture characterized by subcanopy rovers and latent space phenotyping. Plant Phenome J. 2:190011 (2019). (doi:10.2135/tppj2019.07.0011) Ubbens, J., Cieslak, M., Prusinkiewicz, P., Parkin, I., Ebersbach, J. & Staviness, I. Latent space phenotyping: automated image-based phenotyping for treatment studies. Plant Phenomics 2020, Article ID 5801869 (2020). (https://doi.org/10.34133/2020/5801869) Chebrolu, N., Lottes, P., Schaefer, A., Winterhalter, W., Burgard, W. & Stachniss, C. Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields. Robotics Research 36:1045-1052 (2017). (doi:10.1177/0278364917720510) Benet, B., Dubos, C., Maupas, F., Malatesta, G. & Lenain, R. Development of autonomous robotic platforms for sugar beet crop phenotyping using artificial vision. AGENG Conference, Jul 2018, Wageningen, Netherlands. 8 p. hal-02155159 (2018). Murman, J.N. Flex-Ro: a robotic high throughput field phenotyping system. Biological Systems Engineering-Dissertations, Theses, and Student Research 99 (2019). University of Nebraska. https://digitalcommons.unl.edu/biosysengdiss/99 Tu, X., Gai, J. & Tang, L. Robust navigation control of a 4WD/4WS agricultural robotic vehicle. Computers and Electronics in Agriculture 164:104892 (2019). (https://doi.org/10.1016/j/compag.2019.104892) Higuti, V. A., Velasquez, A.E., Magalhaes, D.V., Becker, M. & Chowdhary, G. Under canopy light detection and ranging‐based autonomous navigation. Journal of Field Robotics, 36(3), 547-567 (2019). Sivakumar, A.N., Modi, S., Gasparino, M.V., Ellis, C., Velasquez, A.E.B., Chowdhary, G. & Gupta, S. Learned visual navigation for under-canopy agricultural robots. Robotics Science and Systems, (2021). Sivakumar, A.N., Gasparino, M.V., McGuire, M., Higuti, V.A., Akcal, M.U. & Chowdhary, G. Lessons from deploying CropFollow++: under-canopy agricultural navigation with keypoints. Robotics Science and Systems, Delft, Netherlands, (2024). Gasparino, M.V., Higuti, V.A., Sivakumar, A.N., Velasquez, A.E., Becker, M. & Chowdhary, G. Cropnav: a framework for autonomous navigation in real farms. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 11824-11830). IEEE (2023) Velasquez, A.E.B., Higuti, V.A.H., Gasparino, M.V., Sivakumar, A.N., Becker, M. & Chowdhary, G. Multi-sensor fusion based robust row following for compact agricultural robots. Journal of Field Robotics, 2:1291–1319 (2022). Shakoor, N., Lee, S. & Mockler, T.C. High throughput phenotyping to accelerate crop breeding and monitoring of diseases in the field. Current Opinion in Plant Biology 38:184-192. (2017). (http://dx.doi.org/10.1016/j.pbi.2017.05.006. Young, S.N. Kayacan, E. & Peschel, J.M. Design and field evaluation of a ground robot for high-throughput phenotyping of energy sorghum. Precision Agriculture (2018). (doi.org/10.1007/s11119-018-9601-6) Tirado, S.B., Hirsch, C.N. & Springer, N.M. Utilizing temporal measurements from UAVs to assess root lodging in maize and its impact on productivity. FCR 262:108014 (2021). (doi.org/10.1016/j.fcr.2020.108014) Vanous, A., Gardner, G., Blanco, M., Martin-Schwarze, A., Lipka, A.E., Flint-Garcia, S., Bohn, M., Edwards, J. & Lübberstedt, T. Association mapping of flowering and heigh traits in germplasm enhancement of maize doubled haploid (gem-dh) lines. Plant Genome 11:170083 (2018). (doi10.3835/plantgenome2017.09.0083) Peiffer, J.A., Romay, M.C., Gore, M.A., Flint-Garcia, S.A., Zhang, Z., Millar, M.J., Gardner, C.A.C., McMullen, M.D., Holland, J.B., Bradbury, P.J. & Buckler, E.S. The genetic architecture of maize height. Genetic 196:1337-1356 (2014). (doi:10.1534/genetics.113.159152/-/DC1) Fromme, D.D., Spivey, T.A. & Grichar, W.J. Agronomic response of corn (zea mays l.) hybrids to plant populations. International Journal of Agronomy 3589768 (2019). (doi.org/10.1155/2019/3589768) Cheng, C.Y., Li, Y., Varala, K., Huang, J., Kim, G.J., Halim, J., Arp, J., Bubert, J., Chih, H-J.S., Levinson, G.L., Park, S.H., Cho, H.Y., Moose, S.P. & Coruzzi, G.M. Evolutionarily informed machine learning enhances the power of predictive gene-to-phenotype relationships. Nature Communications 12: 5627 (2021). https://doi.org/10.1038/s41467-021-25893-w Correndo, A.A., Tremblay, N., Coulter, J.A., Ruiz-Diaz, D., Franzen, D., Nafziger, E., Prasad, V., Rosso, L.H.M., Steinke, K., Du, J., Messina, C.D. & Ciampitti, I.A. Unraveling uncertainty drivers of the maize yield response to nitrogen: a Bayesian and machine learning approach. Agricultural and Forest Meteorology 311:108668 (2021). (doi.org/10.1016/j.agrformet.2021.108668) Gaffney, J., Schussler, J., Löffler, C., Cai, W., Paszkiewicz, S., Messina, C., Groeteke, J., Keaschall, J. & Cooper. M. Industry-scale evaluation of maize hybrids selected for increased yield in drought-stress conditions of the US corn belt. Crop Sci. 55:1608-1618 (2015). (doi.org/10.2135/cropsci2014.09.0654). Rebetzke, G.J., Jimenez-Berni, J., Fischer, R.A., Deery, D.M. & Smith, D.J. Review: high-throughput phenotyping to enhance the use of crop genetic resources. Plant Sci. 282:40-48 (2019). (doi.org/10.1016/j-plantsci.2018.06.017) Berry, P.M., Baker, C.J., Hatley, D., Dong, R., Wang, X., Blackburn, G.A., Miao, Y., Sterling, M. & Whyatt, J.D. Development and application of a model for calculating the risk of stem and root lodging in maize. FCR 262:108037 (2021). Veenstra, R.L., Messina, C.D., Berning, D., Haag, L.A., Carter, P., Hefley, T.J., Prasad, P.V.V. & Ciampitti, I.A. Tiller biomass in low plant-density corn enhances transient C sink without direct harvest index detriment. FCR 292:1068804 (2023). (doi.org/10.1016/j.fcr.2022.108804) Wallman, M. & Zakhor, A. Height estimation of biomass sorghum in the field using li-dar. IS&T International Symposium on Electronic Imaging. Computational Imaging XVII (2019). (doi.org/10.2352/ISSN.2470-1173.2019.13.COIMG-137) McGuire, M., Aref, T., Syraj, A., Shukla, S., Kumar, A., Moose, S., Diers, B. & Chowdhary, G. High-throughput phenotyping for essential crop traits. Association for the advancement of Artificial Intelligence (2022). (www.aaai.org) Miller, J.B. A formula for average foliage density. Australian Journal of Botany 15:141-144 (1967). (doi.org/10.1071/BT9670141) Liu, S., Jin, X., Nie, C., Wang, S., Yu, X., Cheng, M., Shao, M., Wang, Z., Tuohuti, N., Bai, Y. & Liu, Y. Estimating leaf area index using unmanned aerial vehicle data: shallow vs. deep machine learning algorithms. Plant Physiology. 187:1551-1576 (2021). (doi:10.1093/plphyso/kiab322) Chen, Y., Jiao, S., Cheng, Y., Wei, H., Sun, L. & Sun, Y. LAI-NOS: an automatic network observation systems for leaf area index on hemispherical photography. Agricultural and Forest Meteorology 322:108999 (2022). (doi.org/10.1016/j.agrformet.2022.108999) Ritchie, S.W. & Hanway, J.J. How a corn plant develops (Special Report No. 48). Iowa State University of Science and Technology, Cooperative Extension Service. (1982) Gilmour, A.R., Cullis, B.R. & Verbyla, A.P. Accounting for natural and extraneous variation in the analysis of field experiments. Journal of Agricultural, Biological, and Environmental Statistics 2:269-273 (1997). (doi.org/10.2307/1400446) Falconer, D.S. & Mackay, T.F.C. Introduction to Quantitative Genetics, 4 th Edition, Longmans Green, Harlow, Essex, UK (1996). SAS Instituste Inc. SAS/STAT® 14.3 User’s Guide. Cary, NC: SAS Institute (2017). Cheng, C.Y., Li, Y., Varala, K., Huang, J., Kim, G.J., Halim, J., Arp, J., Bubert, J., Chih, H.-J. S., Levinson, G.L., Park, S. H., Cho, H. Y., Moose, S.P. & Coruzzi, G.M. Evolutionarily informed machine learning enhances the power of predictive gene-to-phenotype relationships. Nature Communications 12: 5627 (2021). https://doi.org/10.1038/s41467-021-25893-w R Core Team R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, (2024) Vienna, Austria. <https://www.R-project.org/. Taiyun W. & Viliam S. R package 'corrplot': Visualization of a Correlation Matrix (Version 0.92) (2021). Available from https://github.com/taiyun/corrplot Wickham, H. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016. Additional Declarations There is NO Competing Interest. Supplementary Files FigureS1S2.docx Cite Share Download PDF Status: Published Journal Publication published 21 Mar, 2025 Read the published version in Communications Biology → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5187688","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Biological Sciences - Article","associatedPublications":[],"authors":[{"id":366916591,"identity":"dc1107db-510c-494f-9b11-0d693d7de4d4","order_by":0,"name":"Jason DeBruin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+klEQVRIiWNgGAWjYBACxgYwxcxgACQPMFSQruUMkGQjzjKIFgbGNiK0MLcff/i4gMFazlz6jOHhwnl2cubzm49J/GCwk9NtwOGwnhxj4xkM6caWfTkGh2duSzaWOcaWJtnDkGxsdgCXX3LYpHkYDiduOMNjcJh324HEGWw8xgY8DAcSt+HS0v/8GUhLPUTLHJAW/s+Gf/BpmZFgBtKSYADW0gC2hfExXltmvDE25jFIN9zZw1ZwmOdYsrEEW5rhYxkD3H4x7E9/+JinwlrenId582eeGjs5CebDDw6+qbCTw6mlAUQaYIhjisCBPG6pUTAKRsEoGAVQAAAykVPDpnF6YgAAAABJRU5ErkJggg==","orcid":"","institution":"Corteva Agriscience","correspondingAuthor":true,"prefix":"","firstName":"Jason","middleName":"","lastName":"DeBruin","suffix":""},{"id":366916592,"identity":"c7d5ba7d-42fb-4c47-bd0c-d42060b8e312","order_by":1,"name":"Thomas Aref","email":"","orcid":"","institution":"Chalmers University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Thomas","middleName":"","lastName":"Aref","suffix":""},{"id":366916593,"identity":"e99ded37-d7df-4da3-b9f3-af844b1f8124","order_by":2,"name":"Sara Tirado Tolosa","email":"","orcid":"","institution":"Corteva Agriscience","correspondingAuthor":false,"prefix":"","firstName":"Sara","middleName":"Tirado","lastName":"Tolosa","suffix":""},{"id":366916594,"identity":"bbc38505-7a8d-4947-af79-7a5acf94e5bd","order_by":3,"name":"Rebecca Hensley","email":"","orcid":"","institution":"Corteva Agriscience","correspondingAuthor":false,"prefix":"","firstName":"Rebecca","middleName":"","lastName":"Hensley","suffix":""},{"id":366916595,"identity":"7eef80ae-fe80-4610-afb1-02f1272d6041","order_by":4,"name":"Haley Underwood","email":"","orcid":"","institution":"Corteva Agriscience","correspondingAuthor":false,"prefix":"","firstName":"Haley","middleName":"","lastName":"Underwood","suffix":""},{"id":366916596,"identity":"e954ee28-86d4-45ec-a916-3e8adcc7155e","order_by":5,"name":"Michael McGuire","email":"","orcid":"","institution":"EarthSense","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"McGuire","suffix":""},{"id":366916597,"identity":"97a25682-3d1e-42f4-ad42-9a2915c38476","order_by":6,"name":"Chinmay Soman","email":"","orcid":"","institution":"EarthSense","correspondingAuthor":false,"prefix":"","firstName":"Chinmay","middleName":"","lastName":"Soman","suffix":""},{"id":366916598,"identity":"3d8b0712-4c6d-439b-ad91-ae24b007590f","order_by":7,"name":"Grace Nystrom","email":"","orcid":"","institution":"University of Illinois","correspondingAuthor":false,"prefix":"","firstName":"Grace","middleName":"","lastName":"Nystrom","suffix":""},{"id":366916599,"identity":"9d320822-7bfc-4cfa-8828-4a314a6198d4","order_by":8,"name":"Emma Parkinson","email":"","orcid":"","institution":"University of Illinois","correspondingAuthor":false,"prefix":"","firstName":"Emma","middleName":"","lastName":"Parkinson","suffix":""},{"id":366916600,"identity":"69f5e504-87b7-4dde-81a8-98d5f66b03c8","order_by":9,"name":"Catherine Li","email":"","orcid":"","institution":"University of Illinois","correspondingAuthor":false,"prefix":"","firstName":"Catherine","middleName":"","lastName":"Li","suffix":""},{"id":366916601,"identity":"95ad19dc-aa14-4e71-ba6d-49b93624b308","order_by":10,"name":"Stephen Moose","email":"","orcid":"","institution":"University of Illinois","correspondingAuthor":false,"prefix":"","firstName":"Stephen","middleName":"","lastName":"Moose","suffix":""},{"id":366916602,"identity":"6a9dfbe1-9ca6-4468-b769-42e8a91e7a75","order_by":11,"name":"Girish Chowdhary","email":"","orcid":"","institution":"EarthSense","correspondingAuthor":false,"prefix":"","firstName":"Girish","middleName":"","lastName":"Chowdhary","suffix":""}],"badges":[],"createdAt":"2024-10-01 13:31:45","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5187688/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5187688/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s42003-025-07890-7","type":"published","date":"2025-03-21T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":67738686,"identity":"1adc8b42-73f9-4119-8ccc-d0b308e11b1c","added_by":"auto","created_at":"2024-10-29 08:25:58","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":697802,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003e(A)\u003c/strong\u003e A TerraSentia® robot in a typical corn breeding experimental unit with 0.76 m spaced corn rows. \u003cstrong\u003e(B)\u003c/strong\u003e locations where the robot was operated in the United States and Canada. \u003cstrong\u003e(C)\u003c/strong\u003e aerial image of a typical research trial location showing the corn plots laid out in columns and ranges. Each experimental unit is typically 5.3 to 9.3 m long with two 0.76 m spaced rows. \u003cstrong\u003e(D)\u003c/strong\u003e the number of locations, number of location by collection dates, experimental units with videos, and number of experimental units with at least one trait delivered.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/fb4bf8d6668991873c7c3cab.png"},{"id":67739082,"identity":"33ecc520-75a6-4a44-992b-ead94bbf247a","added_by":"auto","created_at":"2024-10-29 08:33:58","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":182552,"visible":true,"origin":"","legend":"\u003cp\u003ePlant canopy traits collected shortly after silk appearance. Three traits were analyzed and the ANOVA indicates level of significance of main effects and interactions \u003cstrong\u003e(A)\u003c/strong\u003e. The values of each trait at each of the plant density treatments are reported along with the standard error for LAI (n=321) (\u003cstrong\u003eB\u003c/strong\u003e), Plant Height (n=384) (\u003cstrong\u003eC\u003c/strong\u003e) and Stem Diameter (n=371) (\u003cstrong\u003eD\u003c/strong\u003e). Letters used within panels B, C, and D indicate significant separation of means for N rate within a given planting density. Data collected during 2022 near Windfall, IN.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/59bde1582a3aa4e95ce44aab.png"},{"id":67740465,"identity":"b6530c09-a803-4186-a6df-79d69351cace","added_by":"auto","created_at":"2024-10-29 08:41:58","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":378584,"visible":true,"origin":"","legend":"\u003cp\u003eRobotic phenotyping from a managed soil-environment enables assessment of maize nitrogen response. (\u003cstrong\u003eA\u003c/strong\u003e) Nitrogen response of leaf area index (LAI) for 44 maize hybrids collected from three replicate plots in Urbana, Illinois on July 25, 2023. (\u003cstrong\u003eB\u003c/strong\u003e) Correlations among the traits collected by the robot at multiple times during the 2022 and 2023 seasons, manual measurement of ear and plant height, and the component traits for nitrogen use efficiency. (\u003cstrong\u003eC\u003c/strong\u003e) Regression between stalk biomass predicted from robotically collected traits and manual measurements from the same plots collected in 2022.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/71cf71cadeb06ea6f65922da.png"},{"id":67738690,"identity":"fdb1a5d6-2906-4fde-93a7-adc6f2c058ee","added_by":"auto","created_at":"2024-10-29 08:25:59","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":370300,"visible":true,"origin":"","legend":"\u003cp\u003eData collection from an on-farm, multi-N, multi-hybrid experiment 6 July in Indiana and 22 July in Nebraska during 2021. (\u003cstrong\u003eA\u003c/strong\u003e) layout of multi-N treatments across a farmer’s field in southern Indiana. Colors indicate the N rates of 0, 67, 134, 202, and 269 kg N ha\u003csup\u003e-1\u003c/sup\u003e applied either as a complete application at planting or as a split application. The length of each experimental strip is provided for reference. (\u003cstrong\u003eB\u003c/strong\u003e) Leaf area index and grain yield produced at the field trial shown in \u003cstrong\u003eA\u003c/strong\u003e and a similarly-designed trial in Nebraska. Fitted trait responses were significantly different between the two locations for both Grain Yield (\u003cem\u003ep\u003c/em\u003e-value \u0026lt; 0.001) and Leaf Area Index (\u003cem\u003ep\u003c/em\u003e-value of 0.0085).\u0026nbsp; (\u003cstrong\u003eC\u003c/strong\u003e) Regression analysis supports a strong linear relationship (R\u003csup\u003e2\u003c/sup\u003e 0.89, \u003cem\u003ep\u003c/em\u003e-value \u0026lt; 0.001) between green leaf area index and grain yield using N treatment and hybrid mean values from the Indiana and Nebraska locations.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/13e8895403bededcbd5b450c.png"},{"id":67738692,"identity":"23372180-4adf-465c-9b76-c2ff408f4966","added_by":"auto","created_at":"2024-10-29 08:25:59","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":479239,"visible":true,"origin":"","legend":"\u003cp\u003e(\u003cstrong\u003eA\u003c/strong\u003e) Robot Leaf area Index (LAI), (\u003cstrong\u003eB\u003c/strong\u003e) Robot Plant Height (PHT) and (\u003cstrong\u003eC\u003c/strong\u003e) Combine Grain Yield reported for a precommercial breeding trial conducted at locations near Hensal, Harriston, and Ashworth, Ontario during 2022. (D) Principal component analysis among LAI (measured at two weeks after silk appearance to obtain maximum LAI and at the R5 growth stage to capture remaining LAI at the end of the growing season), plant height and grain yield.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/bdd8b9e0febaff24bd29752c.png"},{"id":79004530,"identity":"dc6675c8-c330-4231-98c7-33010dd91b4c","added_by":"auto","created_at":"2025-03-22 07:10:40","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3082831,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/9568e43b-40d4-4393-a432-577924b26e52.pdf"},{"id":67738689,"identity":"298aa733-e6fb-4b13-9c65-c130a6cb6cfb","added_by":"auto","created_at":"2024-10-29 08:25:59","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":692050,"visible":true,"origin":"","legend":"","description":"","filename":"FigureS1S2.docx","url":"https://assets-eu.researchsquare.com/files/rs-5187688/v1/c5da004a8c7ec5da173b2522.docx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Breaking the field phenotyping bottleneck in maize with autonomous robots","fulltext":[{"header":"Introduction","content":"\u003cp\u003ePhenotyping of biologically relevant traits is a high priority to accelerate genetic gain in crop breeding programs\u003csup\u003e1\u003c/sup\u003e and close notable yield gaps\u003csup\u003e2\u003c/sup\u003e. Phenotyping in the field, and more specifically in the target population of environments\u003csup\u003e3\u003c/sup\u003e is critical to capture the genotype by environment (G x E) interactions that define phenotypic outcomes\u003csup\u003e3,4\u003c/sup\u003e. Currently, high-throughput genotyping is routinely used in maize (\u003cem\u003eZea mays\u003c/em\u003e L.) breeding programs to aid marker assisted selection and whole genome prediction\u003csup\u003e5\u003c/sup\u003e. However, the scale of genomic data now far exceeds phenotypic data\u003csup\u003e6\u003c/sup\u003e, from the perspectives of both data quantity and cost per data point. Phenotypic data are essential to provide context to genomic data\u003csup\u003e4\u003c/sup\u003e and currently present a bottleneck in research and crop improvement activities\u003csup\u003e7,8\u003c/sup\u003e. Although modeling efforts aid in predicting phenotypes from genotype, they typically rely on training data for phenotypes that are limited to either 1) highest priority traits such as grain yield or 2) can be captured at large scale for low cost\u003csup\u003e9\u003c/sup\u003e. Phenotyping for component traits often adds key biological insights to understand yield outcomes and GxE\u003csup\u003e10,11\u003c/sup\u003e, but these are often difficult and time consuming to measure.\u003c/p\u003e \u003cp\u003ePlant breeding companies focus on the continued improvement of maize genotypes through testing, advancement, and recycling of germplasm through their testing pipeline\u003csup\u003e12\u003c/sup\u003e. How these new genotypes interact with management factors once in the hands of farmers is of considerable interest as well and must be assessed through small plot and on-farm trials. Management factors such as plant density\u003csup\u003e13\u003c/sup\u003e and nitrogen management\u003csup\u003e14\u003c/sup\u003e can be applied to more fully understand Genotype by Environment by Management (GxExM) interactions\u003csup\u003e4\u003c/sup\u003e. These efforts require phenotypic characterization of the crop throughout the season to specifically determine the morphological changes that occur and further explain grain yield outcomes under a variety of environment and management conditions. Substantial resources are required to collect relevant morphological phenotypes such as stem diameter, leaf area index (LAI) and plant biomass\u003csup\u003e15, 16,17,18\u003c/sup\u003e. Depending on the trait of interest, a physical measurement may require 10 or more minutes of human effort to collect a single datapoint. Time and labor constraints mean that field scientists must limit the number of treatments, genotypes, and locations where morphological traits are collected further contributing to the phenotyping bottleneck observed in maize research trials. In general, currently available maize phenotyping methods are not capable of producing a complete assessment of plant growth processes which leads to an incomplete understanding of GxExM.\u003c/p\u003e \u003cp\u003eUnmanned aerial vehicles have been deployed at a broad scale to collect RGB, multi-spectral, hyper-spectral and thermal data. Images from these flights capture biologically relevant traits such as plant height\u003csup\u003e19\u003c/sup\u003e, leaf area index\u003csup\u003e20\u003c/sup\u003e, and NDVI\u003csup\u003e21\u003c/sup\u003e. Data fusion approaches have been implemented to integrate signals captured from different sensor types, which partially overcomes the problem of canopy saturation for some of the most informative spectral bands\u003csup\u003e22\u003c/sup\u003e. However, in-canopy traits, such as ear height, stem width remain challenging, if not impossible, to predict from above-canopy phenotyping tools. Deployment of compact, in-canopy robotic platforms carrying cameras and sensors is an active space of development to unlock access to below-canopy phenotypic traits at similar scales to above canopy trait collection.\u003c/p\u003e \u003cp\u003eGround based, under-canopy, phenotyping platforms have taken various forms. Initial developments were wheeled but human-pushed\u003csup\u003e23\u003c/sup\u003e tractor carried\u003csup\u003e24\u003c/sup\u003e or motor propelled but human driven\u003csup\u003e25\u003c/sup\u003e platforms have also been demonstrated. More recently self-propelled platforms capable of running between crop rows and require little human operation\u003csup\u003e26\u003c/sup\u003e have also been demonstrated. The last category of ground robots, motorized and autonomous, offers the greatest flexibility and best opportunity to enable parallel, simultaneous operation of multiple robots for high-throughput phenotyping. Ground-based platforms can measure several traits from either maize or sorghum. Examples include plant height and volume\u003csup\u003e24\u003c/sup\u003e, stem diameter\u003csup\u003e27\u003c/sup\u003e, stem strength\u003csup\u003e28\u003c/sup\u003e, and leaf area index\u003csup\u003e29\u003c/sup\u003e. However, each of these leveraged a lower-throughput system that required substantial human assistance (i.e. pushing or driving through the platform through the field) to collect the required information for trait calculation. For this reason, automating in-row, under-canopy, self-propelled systems for the measurement of phenotypic traits can have tremendous value for increased throughput with reduced human labor.\u003c/p\u003e \u003cp\u003eAutonomous robotic platforms for under-canopy phenotyping have become more available for use by the practitioner with examples including TerraSentia\u003csup\u003e26\u003c/sup\u003e (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eA) and MARS\u003csup\u003e30\u003c/sup\u003e. While proof-of-concept demonstrations for collecting new phenotypic traits continue to increase, there are few examples of autonomous robots deployed at a scale sufficient to enable phenotyping required for a large-scale breeding, trait evaluation, or agronomic testing program. A single example leveraged an autonomous robot to collect LiDAR scans on 2103 experimental units with maize hybrids\u003csup\u003e31\u003c/sup\u003e. Novel latent-space (machine learning based system to characterize and discriminate differences among a series of image sequences for a set of treatments\u003csup\u003e32\u003c/sup\u003e) phenotypes were extracted from these scans that showed heritabilities as high as 0.44\u003csup\u003e31\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eA significant hurdle to deployment of self-propelled under-canopy phenotyping robotic platforms is reliable in-row navigation under real-world, variable, field, and crop-canopy conditions. This problem has been studied with suggested solutions by several groups\u003csup\u003e33,34,35,36\u003c/sup\u003e. The primary challenge in autonomous navigation under the crop canopy is that Global Navigation Satellite System (GNSS), also known as Global Positioning System (GPS) does not provide reliable enough localization for navigation of these ground-based within-row robots, even with the use of augmentation protocols such as Real-Time Kinematic (RTK). To overcome this challenge, algorithms and systems are required that are capable of row-following using other exteroceptive sensors such as LiDAR and cameras\u003csup\u003e37,38,39,40,41\u003c/sup\u003e. Autonomous robots leveraging such algorithms can ensure reliable under-canopy in-row navigation, and can handle obstacles in real-world fields, such as weeds, ditches, ruts, etc. However, prior to this study, such autonomous robotic systems were not deployed at scale sufficient to establish their use in modern breeding and agricultural product development pipelines.\u003c/p\u003e \u003cp\u003eOur objectives were 1) determine whether autonomous ground based robots could be used to collect phenotypic data at a scale required for a large commercial corn breeding and research operation, 2) assess whether multi-sensor data collected from underneath the maize plant canopy by these robots could accurately estimate phenotypic traits that are commonly collected in maize hybrid development and associated with grain yield, and 3) investigate GxE and GxExM interactions for grain yield in different experimental designs to gain new insights into how maize hybrids respond to variation in nitrogen supply, planting density, and precipitation during the growing season.\u003c/p\u003e"},{"header":"Results and Discussion","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eSmall Under-Canopy Phenotyping Robots have a High Deployment potential\u003c/h2\u003e \u003cp\u003eBetween 2019 through 2023, through the deployment of 3\u0026ndash;5 TerraSentia robots (EarthSense, Inc. Champaign, IL) per year by multiple individuals located in various parts of the United States and Canada (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eB) we achieved a collection of multi-sensor data from 198,249 maize plots (experimental units, or EUs) (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eC) (5.4 to 9.4 m length by 1.52 m width). The data were collected across 142 unique locations (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eD), an average of three to four times each, resulting in 418 unique location by time data collections. The compact size and limited weight of the TerraSentia robots, relative to other ground based autonomous robots, was one of the main factors that allowed for broad deployment across many locations and across experimental units within a location\u003csup\u003e30\u003c/sup\u003e. In addition to size, operators reported continued improvement during the study years of greater robot reliability (i.e. maintain wi-fi connection to tablet, maintain cool operating temperature, hot-swapable batteries), and a simpler user interface to engage more operators (based on a survey of operators). Critically important to high-throughput data collection was the significantly improved autonomous navigation. A typical daily collection could accumulate to a total driving distance between 10 and 11,000 m. Beginning from 27 m in 2019, the average distance between human interventions increased to a peak of 3,629 m of unassisted autonomous navigation between human interventions by 2023 due to improved algorithms\u003csup\u003e37,38,39,40,41\u003c/sup\u003e. This demonstrated for the first time the feasibility of under-canopy robots for high-throughput phenotyping and other applications.\u003c/p\u003e \u003cp\u003eFrom the imaging and LiDAR data, plant height, stem diameter, and leaf area index or ear height were calculated for each of the 132,176 experimental unit, delivering nearly 400,000 individual phenotypic trait measurements. During the multiple years of data collection, the number of experimental units with video data increased 24X and location number increased 20X (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eD). A requirement for any high-throughput system to be useful is the collection of high-resolution images that facilitate computer vision and machine learning\u003csup\u003e42\u003c/sup\u003e. Here we note the marked improvement between 2019, when traits were estimated for 982 (36%) of the 2731 experimental units with multi-sensor data, to 2022 when 98% of the 67,148 experimental units with multi-sensor data successfully delivered at least two traits. The dataset collected in 2022 exceeds by an order of magnitude the number of phenotyped experimental units from field-grown maize using UAVs\u003csup\u003e22\u003c/sup\u003e, and by two orders of magnitude the largest phenotyping effort with UGVs\u003csup\u003e26,43\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eCase studies for high-throughput phenotyping in agriculture\u003c/h3\u003e\n\u003cp\u003eFrom this large set of robotically collected data, we have selected five case studies to demonstrate the capability of the TerraSentia phenotyping platform to deliver high volumes of phenotypic traits. These traits both replace commonly manually collected traits, but also provide new opportunities to collect difficult phenotypic data to inform GxExM in the context of a commercial maize breeding and agronomic research program.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study I. Replacing common, but manual trait collection in a maize breeding program.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA commercial program evaluates hundreds of crosses at multiple locations each year\u003csup\u003e12\u003c/sup\u003e, with varying levels of biotic and abiotic stress, with the goal of finding high performing genotypes with acceptable traits. The height of the maize ear is a trait of particular interest due to correlation with lodging risk\u003csup\u003e44\u003c/sup\u003e. Previously this trait was collected manually using a measurement stick and data collection on a few plants within each experimental unit\u003csup\u003e45\u003c/sup\u003e. This method requires at least one individual to enter each experimental unit and measure from the soil surface to the nodal attachment point of the ear, typically for one to four plants, followed by manual data entry. This process requires approximately one to two minutes of human effort per experimental unit. It is important to note, it is not possible to measure maize ear height using above-canopy imagery collected with drones, let alone aircraft or satellites.\u003c/p\u003e\n\u003cp\u003eWith the TerraSentia robots, we were able to deliver ear height measurements for 4826 experimental units across five research locations (Table 1). The ear-height algorithm detects the ear, determines upward or downward orientation, and then identifies the node attachment point to the stem. The automated measurement of ear height in this experiment was achieved with approximately 20 hours of in-field data collection, a significant reduction compared to an estimated 150 hours of human labor that would have been required to complete equivalent manual measurements. TerraSentia reported ear height ranged from 59 and 152 cm depending on the location (Table 1). Ear height values are comparable to those obtained from manual measurements of ear height collected from genetically diverse maize populations grown over multiple years and locations in the US\u003csup\u003e45,46\u003c/sup\u003e. A within-location heritability estimation was calculated for each of the five locations utilizing the best linear unbiased estimates (BLUEs) of germplasm effects by trial and ranged between 0.43 and 0.67 demonstrating a strong genotypic response to the trait among the various locations. One experiment was common across three locations and produced a heritability estimate of 0.66. By collecting variable ear height data across a large number of locations and genotypes, we show that the rover can feasibly replace manual data collection for important agronomic traits like ear height and produce data with high heritability both within and across locations that can be selected upon.\u003c/p\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv\u003eTable 1\u003c/div\u003e\n \u003cdiv\u003e\n \u003cp\u003eDescriptive statistics and heritability for maize ear height collected in a pre-commercial breeding program using the TerraSentia robot at five locations in 2022 in the Midwest USA.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eLocation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eObservations\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"3\"\u003e\n \u003cp\u003eEar Height (cm)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHeritability\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eno.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eminimum\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003emaximum\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003emean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eratio\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGriswold, IA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e803\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e62\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e148\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e118* (9)**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.64\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrairie City, IA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e757\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e134\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e110 (7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.48\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCameron, IL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1004\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e64\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e120 (10)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.62\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTable Grove, IL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e976\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e143\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e111 (8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.67\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWindfall, IN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1286\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e136\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e108 (8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.43\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAcross locations***\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3484\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e111 (5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.66\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003cp\u003e* Single location minimum, maximum, and mean are best linear unbiased estimates. Across location minimum, maximum, and mean values are best linear unbiased predictions.\u003c/p\u003e\n\u003cp\u003e** Value in parentheses is the standard deviation\u003c/p\u003e\n\u003cp\u003e*** Includes Cameron, IL, Table Grove, IL, and Windfall, IN\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study II. Multi-factor experiments using component trait phenotyping.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAgronomic management choices such as planting density and nitrogen rate are ways farmers can optimize crop productivity and profitability at the field and farm level. Small-plot research trials are leveraged to study multiple management factors in multiple combinations to understand the main effects and potential interactions. In this experiment, six hybrids were evaluated at one location in four replicate plots, in each combination of four levels of N fertilizer and four planting densities, for a total of 384 experimental units. The experiment was established to estimate grain yield from these treatment combinations and the TerraSentia robot, operated by one individual, enabled a deeper characterization of how crop morphological traits responded to the treatments.\u003c/p\u003e\n\u003cp\u003eNitrogen rate and plant density interacted, exhibiting greater leaf area index (LAI) (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026le;\u0026thinsp;0.01) and lower plant height (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.01) as plant density increased across treatments (Fig. 2A). While LAI was greater with higher N rates the degree of response from LAI to N rates was lower for the lower plant density treatments (Fig. 2B). Compared to LAI, plant height was greater with increasing plant density and interacted with N rate (Fig. 2C). Both plant height and LAI in maize have been previously reported to respond to nitrogen rate and plant density\u003csup\u003e16,17\u003c/sup\u003e. Here, we observed that these traits showed distinct responses to plant density in the unfertilized compared to fertilized plots, with plant height being progressively reduced by each increase in plant density, whereas leaf area index was maximized at a density of 9.3 plants m\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e. Stem diameter was smaller at greater plant densities regardless of N treatment (Fig. 2D), in line with prior observations\u003csup\u003e16,47\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eThe TerraSentia platform offered a dramatic increase in the efficiency at which LAI and stem diameter were measured from these 384 experimental units. On average, multi-sensor data was collected on 250 EUs hr\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e. Approximately 1.6 hours was spent by a single person with a robot to collect the multi-sensor data that, after processing, provided measures of stem diameter, leaf area index, and plant height. This compares to the approximately 192 hours of human capital that would have been required to collect these traits manually (30 minutes per experimental unit) across three separate operations: 1) remove leaves from four random plants and scan through a leaf area meter in the lab, 2) collect plant height with a measuring stick on four plants and 3) measure stem diameter on ten plants with calipers. Simultaneous measurement of these three traits has already been demonstrated to predict biomass of sorghum\u003csup\u003e43\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eA robotic phenotyping platform thus makes a number of significant contributions to small-plot, multi-factor research trials. First, the number of treatments can be increased, as the desired intensity of phenotyping can be applied to the entire experiment rather than being limited by the human labor available for phenotyping. Second, additional locations can be phenotyped as well, to bring in the treatment-by-location interactions that are critical for broad applicability of research results. Labor constraints limit manual data collection of a few individual plants per plot, whereas the robot collects data from most, if not all, of the plants in each plot. Due to the number of people taking measurements or collecting samples there is also a substantial opportunity for error or variation between data collected by the individual people, limiting the overall data quality and potential usefulness. Identical robotic phenotyping platforms deployed across locations remove many of these measurement errors and facilitates the potential to combine datasets across locations, seasons, and across research groups in meaningful ways.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study III. Phenotyping genetic diversity for N response traits.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBreeding programs often use managed environments for evaluating germplasm responses to key drivers of productivity, such as soil nutrient supply and water availability. Managed environments enable scaling of precision phenotyping to reveal genetic variation under stress conditions that may occur in the target population of environments\u003csup\u003e12\u003c/sup\u003e. The University of Illinois maintains a field location where the field soils have been managed to be limiting for nitrogen supply in an otherwise high yield potential environment. This field site has previously enabled evaluation of nitrogen-responsive phenotypes in maize at population scale\u003csup\u003e22,48\u003c/sup\u003e. The TerraSentia robot was deployed to phenotype replicated plots of at least 44 experimental hybrids in both 2022 and 2023, where each hybrid was grown with either no N fertilizer or 200 kg ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e of N side-dressed at the V3 growth stage. In 2022, data were collected at two times post-flowering (July 22 and August 17) from 312 plots and in 2023 at the R2 growth stage (July 25) from 480 plots.\u003c/p\u003e\n\u003cp\u003eCase Study II indicated that LAI is the only robot produced trait that showed a significant hybrid by N-rate interaction (Fig. 2A). Figure 3A shows the variation in LAI values for the 44 hybrids grown in 2023. Among the 217 total N-response plots, LAI values were greater with nitrogen application for all but three, making LAI measurements an accurate classifier for N treatment regardless of genotype. Each hybrid also displayed a positive average N response for LAI, which ranged from 1.12 to 1.47. Comparisons of LAI values for the six hybrids with the lowest (\u0026lt;\u0026thinsp;1.2) or highest (\u0026gt;\u0026thinsp;1.48) N response indicates the most responsive hybrids have both lower LAI at low N (mean 3.64 versus 4.01) and higher LAI at high N (mean 5.28 versus 4.83).\u003c/p\u003e\n\u003cp\u003eFigure 3B shows the results from correlation analyses among the robot collected traits at different times over two seasons, ground truth data for plant height and ear height, and component traits for nitrogen use efficiency (NUE) measured at physiological maturity. As indicated from Fig. 3A, among all factors considered, LAI has the highest positive correlation with N rate. Although significant in both years, the stronger correlation between LAI and N rate in the July 2023 compared to August 2022 dataset likely reflects LAI values being collected near their peak values before the onset of nutrient remobilization and leaf senescence. The moderate correlation between LAI values collected in August 2022 and July 2023 for the same hybrids indicates some consistency across seasons, although the rank correlation among genotypes was not statistically significant. The July 2023 LAI values were positively correlated with each of the NUE component traits known to increase with N supply, and negatively correlated with nitrogen utilization that decreases with fertilization. Despite being closer in time to final harvest, the August 2022 LAI values showed only a moderate correlation with only grain yield, suggesting that canopy photosynthetic capacity earlier in the season is driving these relationships.\u003c/p\u003e\n\u003cp\u003eIn this experiment, LAI was not correlated with either ear or plant height, which is consistent with LAI capturing spectral rather than architectural features of the crop canopy. Trait values collected with the robots were highly correlated (r\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;\u0026gt;\u0026thinsp;0.93) with ground-truth measures for both ear and plant height from a subset of the 2022 plots, further demonstrating the accuracy of the onboard algorithms to derive these traits when challenged with a wide range of phenotypic variation due to the interaction of both genetic diversity and N supply. Furthermore, as anticipated for height traits with high heritabilities\u003csup\u003e31\u003c/sup\u003e, robot phenotypes delivered by TerraSentia showed strong correlations across measuring times and seasons. Among NUE component phenotypes, ear and plant height only showed a significant positive correlation with stalk biomass, which increased with data collection later in the season nearer to final harvest. The stronger correlation of ear height over plant height for stalk biomass is likely due to the higher proportion of stalk biomass accumulated below the ear node compared to above the ear.\u003c/p\u003e\n\u003cp\u003eThe relationships between the robot measured traits and human measured stalk biomass indicate the potential to measure stalk biomass in an automated and non-destructive manner. In addition to height and LAI, the robots also delivered average stem diameter for each plot. Stem diameter was found to be correlated with LAI, but not to height, and in July 2023 both LAI and stem diameter showed very similar associations with NUE traits. Measuring stem diameter in a high-throughput and automated fashion opens the opportunity to leverage allometric measurements for estimating total plant biomass\u003csup\u003e15\u003c/sup\u003e. Figure 3C illustrates how a linear regression model including ear height, plant height and their interaction term provides good predictions for genetic variation in stalk biomass for both fertilized and unfertilized plots from the 2022 experiment. The robot traits were collected with a single robot in under four hours with one person, and much earlier in the season, compared to approximately 100 person-hours spent on measuring stalk biomass. The estimated broad-sense heritability for stalk biomass in the 2022 experiment was 0.89 when measured manually and 0.76 when predicted from the TerraSentia robot trait model, indicating that the 25-fold reduction in effort with automated phenotyping for stalk biomass may be achieved without a major loss in potential genetic gain.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study IV. From research trials to on-farm system evaluations.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eGenotype selection coupled with management choices and environmental events define the outcome in a farmer\u0026rsquo;s field. This interaction is best understood by phenotyping the genotype x management decisions within the relevant field environment\u003csup\u003e4\u003c/sup\u003e; however, this is difficult due to field size, accessibility and heterogeneity of the field to determine where trait collection should occur.\u003c/p\u003e\n\u003cp\u003eWe were able to cover large transects of the field (Fig. 4A) to capture the influence of soil type (E) on the phenotypic traits which were expected to primarily respond to nitrogen rate (M) and hybrid (G). The LAI trait measurement from the Nebraska and Indiana fields delivered several useful insights. Canopy size was greater at the irrigated Nebraska location compared to the Indiana location for all N rates (Fig. 4B). Specifically of interest was the 0 kg N ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e rate as this represents the potential for the soil to provide N to the crop via available NO\u003csub\u003e3\u003c/sub\u003e-N or through the process of mineralization. An assessment of grain yield at a zero N rate has been shown to be useful for N rate recommendations\u003csup\u003e49\u003c/sup\u003e. As expected, LAI was greater with higher N rate but did not respond to N rates greater than 134 kg N ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e at the Nebraska site, whereas 202 kg N ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e was needed to reach a maximum LAI value at the Indiana site. For total N rates up to 202 kg ha\u003csup\u003e\u0026minus;\u0026thinsp;1,\u003c/sup\u003e plots that received split N applications trended to exhibit higher LAI values and grain yields compared to plots that received all N at planting. Grain yield was linearly correlated to LAI (Fig. 4C) and did not exhibit any saturation effect.\u003c/p\u003e\n\u003cp\u003eGiven the size of these research plots (740 m long in Indiana and up to 1330 m long in Nebraska; Fig. 4A) collection of LAI, stem diameter and plant height would have been economically and physically infeasible with conventional human measurement methods. Greater leaf area index, a trait strongly sensitive to the management and environment specifically through availability of N and water, brings the potential benefit of greater N storage within the plant biomass\u003csup\u003e18\u003c/sup\u003e which is expected to be advantageous later in the season as a reserve of N to complete grain filling. However, greater LAI could potentially come at a cost if water is limited and the greater leaf area results in a transpiration rate in excess of water available in the soil. Ability to non-destructively and efficiently phenotype LAI throughout the season provides insight to canopy expansion as a result of water and N availability and enables a calculation of grain yield production efficiency relative to investment in leaf area.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study V. Deeper understanding of hybrid evaluation trials with previously elusive traits.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eInteractions between genotypes and the environment result in the phenotype present at a particular location\u003csup\u003e3\u003c/sup\u003e. Utilization of multiple locations provides additional information on G x E interaction. A particularly important environmental factor known to influence G x E is precipitation. In 2022, LAI, plant height, and grain yield were collected for a set of precommercial hybrids in Ontario. Rainfall in July and August was below the 20-year historical average at each of these locations but was more limited at Atwood and Hensal compared to Harriston. Traditional methods of LAI data collection are very labor intensive and are unfeasible for collecting LAI in large maize breeding programs. The robot offered the novel opportunity to collect LAI for multiple research locations multiple times throughout crop development, understand how change in LAI through the maize lifecycle is affected by the environmental conditions, and enable investigation of genotypic variation in this response between pre-commercial lines.\u003c/p\u003e\n\u003cp\u003eThere was a notable difference in LAI among locations. The precommercial population at Harriston had an average LAI of 6.5 while maximum LAI was 5.25 at Atwood and 5.15 at Hensall (Fig. 5A). Plant height was lower at the sites that received lower precipitation with an average height of 226 cm at Harriston versus 212 cm at Atwood and 208 cm at Hensall (Fig. 5B). Yield was lower as well, with Harriston averaging 14250 kg ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e while Hensall and Atwood averaged 12250 kg ha\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e each (Fig. 5C). Principal Components Analysis (PCA) indicated a strong correlation between grain yield and maximum LAI as well as with LAI at R5 (Fig. 5D). Greater LAI at the Harriston location indicated the potential for greater solar radiation interception as well as more N storage in the leaves to support kernel filling through remobilization. Notably, LAI has not been reported as a phenotype under active selection in breeding trials. Only recently has LAI been collected via UAS during the growing season on a breeding population\u003csup\u003e10\u003c/sup\u003e. In this novel work they demonstrate how QTLs associated with LAI explain a substantial amount of the yield variation indicating the potential benefit of using LAI as a selection index in maize breeding. Expanding LAI phenotyping to multiple locations across a more diverse set of hybrids and trials will provide insight into how a specific genotype achieves observed grain yield and enable its future use as an informative selection metric.\u003c/p\u003e\n\u003cp\u003eRobotic platforms that facilitate multi-time point data collection during the season provide the opportunity to define rates of canopy expansion and biomass accumulation as well as end-of-season canopy loss. Measurement of LAI and plant height during the season provides an indication of when stress is beginning to occur and to what extent. Identifying the times when stress occurs is useful for grouping locations with similar stress patterns into a combined multi-location analysis\u003csup\u003e50\u003c/sup\u003e. These time course trait measurements are potentially the phenotypic descriptors to inform \u0026ldquo;enviromics\u0026rdquo; to enable further understanding of the genotype-by-environment interactions.\u003c/p\u003e\n\u003cp\u003eScreening a large breeding population is required to leverage the genetic diversity available for crop improvement\u003csup\u003e51\u003c/sup\u003e. A critical component to the success of high-throughput phenotyping across large populations is the capacity not only of the phenotyping platform but the capacity to transfer larger data sets, extract data from imagery and other sensor data and organize trait data emerging from the phenotyping pipeline\u003csup\u003e8\u003c/sup\u003e. The objective for high throughput phenotyping is to create a virtuous cycle that automates crop phenotyping, trait computation, analysis and selection followed by the next season of phenotyping\u003csup\u003e42\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFuture applications: Automating \u0026ldquo;old\u0026rdquo; traits, developing new traits, and increasing scale.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDeployment of the TerraSentia robots at multiple locations with different types of GxExM experimental designs between 2019 and 2023 demonstrates the potential to deploy ground based autonomous robots at sufficient scale to provide data at geographic scales that match data generated by other sources such as UAV, combine harvesters as well as from genotyping. Robotic, in-canopy trait collection opens an entirely new area of research into the linking of these in-canopy traits to marker association, crop selection, and predictive breeding to continue to investigate and elucidate G x E x M effects. Additionally, we demonstrate that the trait measurements derived from the in-canopy multi-sensor data are agronomically relevant and useful to understand crop growth, crop stress, and genetic diversity.\u003c/p\u003e\n\u003cp\u003eComparison of the data collected via human measurement on multi-rate N trials or plant density trials with the robotic data shows that there are strong similarities in the response of plant height, stem diameter, leaf area index, and ear height estimated by both methods. Each of these traits have historically been collected at lower coverage through less efficient manual methods. Moving forward, the potential exists for phenotyping other rarely studied traits captured via in-canopy sensing, such as the influence of brace root characteristics (size, width, origin node, diameter of ground coverage) on lodging risk\u003csup\u003e52\u003c/sup\u003e. Automated counting of tillers that form around the base of the main stem of maize will also become a phenotypic trait potentially scored for multiple hybrids at multiple locations to determine their relationship to grain yield\u003csup\u003e53\u003c/sup\u003e. The list of novel traits that will become routine will continue to increase with additional sensors, robotic platforms, and data analytics.\u003c/p\u003e\n\u003cp\u003eOver the four years of operation, TerraSentia robots navigated along each corn row autonomously with a user simply following the robot to facilitate any crash recoveries and turning at the end of the field to begin the next pass of data collection. Future capabilities, currently under development and testing, will bring reliable full-field autonomy to data collection. The robots will navigate through the field autonomously staying between the crop rows and recovering independently when an obstruction occurs. Additionally, the robots will turn at the end of each pass autonomously by navigating between GPS waypoints prior to starting the next pass of data collection. Human involvement at this point will be minimal outside of delivering the robot to the field and exchanging batteries. When the potential to \u0026ldquo;swarm\u0026rdquo; robots becomes a reality, data collection rates can greatly increase as each robot is assigned a particular section of the field as their mission.\u003c/p\u003e\n\u003cp\u003eThe next generation of phenotyping traits will be delivered by autonomous, in-canopy robots. Through their capacity to digitize traits critical to crop performance, at high densities of measurement in both time and space and on many genotypes, phenotyping robots will contribute significantly to increasing grain yield, reducing yield gaps, and optimizing agronomic crop management. This technological advance will enhance both the productivity and sustainability of agriculture and the future of food production.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003eA ground-based robot, capable of driving between 0.76 m spaced maize rows (Fig. 1A) was developed by EarthSense, Inc. (Champaign, IL) beginning in 2017. A collaborative effort between Corteva Agriscience (Johnston, IA) and EarthSense began in 2019 and a collaboration with the University of Illinois (Champaign-Urban, IL) and EarthSense started in 2021 to determine whether phenotypic traits important to maize growth could be measured from in-canopy multi-sensor data collected on this robot, named TerraSentia.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTerraSentia is a configurable, terrestrial robotic platform that autonomously performs under canopy data collection, running between rows of maize or other row-crops. On board the TerraSentia, a number of sensors are mounted including three cameras for RGB video capture, lidar for 3D point clouds, fisheye lens cameras for hemi-spherical photography and depth cameras that provide RGB with depth\u0026nbsp;information. The robot can complete an experimental unit (5.3 m long) in about 6 seconds capturing data for every plant in the unit and autonomously runs for typically 180 m without manual intervention\u0026nbsp;(i.e. user takes control of the robot using a tablet to then drive the robot)\u0026nbsp;once the user has placed the robot in-between the tows and started the navigation.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;The TerraSentia robot is capable of autonomous row-following, meaning it can effectively navigate long distances down rows of crops with minimal input from the user. Row-following is enabled by a combination of LiDAR sensors, cameras, and GPS mounted on the robot (Fig. S1A). The onboard computer processes the data from these sensors to map the locations of the crop rows on each side and the orientation of the robot relative to the rows. This information about the robot and its surroundings is then used for path planning and decision making.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe TerraSentia robot is equipped with three adjustable cameras, one on the front and one on each side. The unit is also equipped with a rear LiDAR sensor for scanning the regions above and to the side of the robot. Depending on robot configuration, the top of the robot can be equipped with an upward facing camera with fisheye lens for LAI data collection, or front facing depth camera angled up at 25 degrees. These sensors are used for data collection and trait estimation.\u003c/p\u003e\n\u003cp\u003eThe robot is configurable and maneuverable with a user-friendly tablet interface (Fig. S1C). Details about a data collection mission, such as the dimensions of the field and the planting date, can be specified by the user. Furthermore, the camera angles, resolutions, fields-of-view, and frame rates of the cameras can be programmatically customized for specific traits. These features allow the robot to simultaneously record data for at least five distinct traits (e.g. stem diameter using the left-side camera, plant count on the right-side camera, LAI using the hemispherical lens on top of the robot, plant height and canopy architecture information using the rear lidar) at a given time, with accurate metadata for future reference.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData processing:\u003c/strong\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eVideos and sensor data are collected by the robot along with metadata (columns, ranges, crop, desired trait, etc.) input from the user are uploaded to the cloud after the robot returns to a base location with stable electrical power (to recharge the robot) and internet connectivity (not in the field). The videos are pre-processed with automatic checks for quality and integrity and run through an automatic plot splitting algorithm\u0026nbsp;(Fig. S1D).\u0026nbsp;The automatic plot splitter uses a deep learning model to look for periodic gaps visible in the lidar scans of the canopy, gaps which occur between the\u0026nbsp;experimental units (Fig. 1 B) being scanned. These outputs of the automatic data verification and plot splitting processes are manually reviewed to ensure data quality (e.g. an error in the plot split can lead to catastrophic \u0026ldquo;off-by-one\u0026rdquo; errors in the alignment of plot results). Alignment of the split plots with their associated metadata (column/range) is compared to GPS coordinates to further verify plot assignment. The multi-sensor data\u0026nbsp;are\u0026nbsp;then fed into the appropriate algorithm, the raw results per scan/frame are aggregated and post processed into\u0026nbsp;the\u0026nbsp;final\u0026nbsp;estimates of the\u0026nbsp;desired traits\u0026nbsp;for each experimental unit, which are then reported back to the user.\u003c/p\u003e\n\u003cp\u003eThe plant-height algorithm uses the point cloud of the vertically scanning lidar to extract the highest measured points as the height of the maize\u0026nbsp;plant\u003csup\u003e54\u003c/sup\u003e. This is calibrated to correspond to the tassel height in maize. Tassel height and uppermost leaf height measured manually generally correlate extremely well so validating versus uppermost leaf height is also effective.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eStem width uses RGB imagery combined with distance-to-plant-row estimates to measure the diameter of the maize\u0026nbsp;stems. The stems are detected using Mask R-CNN image segmentation algorithm. Stem widths in pixels are extracted at several sampling points\u0026nbsp;on each vertical stem.\u0026nbsp;These pixel widths are converted to real widths using the distance to stem, estimated from the robot\u0026rsquo;s lidar and odometry\u0026nbsp;data\u003csup\u003e55\u003c/sup\u003e.\u0026nbsp;The correlation to manual data can be complicated by the millimeter scale measurement error and limited measurement window (young stems tend to have many leaves blocking the stem and later stage maize tends to have drooping leaves which block the\u0026nbsp;stems).\u003c/p\u003e\n\u003cp\u003eThe ear height algorithm processes data gathered by an RGB + Depth sensor. A machine-learning based object recognition algorithm detects maize ears and then uses the depth information to estimate geometric properties of the maize ear in image space. The geometry of the maize ear is then adjusted by accounting for the location and orientation of the camera, which are informed by IMU sensors in the robot and in the camera itself. From this adjusted maize ear geometry, the height of the maize ear can be computed. The reported ear height averaged over a number of ears in the plot correlates well with manual measurements of ear height (Fig. S2.A).\u003c/p\u003e\n\u003cp\u003eThe LAI algorithm processes data gathered by an upward facing camera with a fisheye lens capturing hemispherical imagery. The algorithm identifies every pixel corresponding to blue light penetrating through the canopy from the sky (blue light is the opaquest wavelength for plants), and then estimates LAI from the distribution of sky pixels using Miller\u0026rsquo;s integral\u0026nbsp;formula.\u003csup\u003e56\u003c/sup\u003e This method is similar to sensor readings provided by commercial LAI instruments, such as the Licor LAI-2200C (Lincoln, NE). This method has also been verified by\u003csup\u003e57\u003c/sup\u003e and specifically for maize by\u003csup\u003e57,58\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eValidation dataset collection\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor each of the traits currently measured from multi-sensor data collected by the autonomous robot, a specific set of ground truth data were collected manually. Stem diameter was measured manually using a digital caliper\u0026nbsp;at the R1 (silk appearance growth stage)\u003csup\u003e59\u003c/sup\u003e.\u0026nbsp;The caliper was placed around the stem in the same direction as the maize row and the diameter was recorded to the nearest tenth mm. For stem diameter, 44\u0026nbsp;experimental units\u0026nbsp;having between 10-40 plants were measured with every plant contributing a measurement.\u0026nbsp;Measurements using the TerraSentia robot were well correlated (p\u0026lt;0.001) with human-acquired caliper traits with an R\u003csup\u003e2\u003c/sup\u003e value of 0.89 and a slope of 1.02 (Fig. S2B).\u003c/p\u003e\n\u003cp\u003ePlant height was measured manually using a tall measurement pole\u0026nbsp;at the R2 (10-14 days after R1) growth stage. The height was recorded to the top of the tassel and the top leaf attachment point (leaf node) from the soil level for 27 experimental units having about 40 plants with every third plant measured. The correlation between the manually measured tassel height and leaf node height was extremely high (R\u003csup\u003e2\u003c/sup\u003e=1.00) with the leaf heights being slightly lower than the tassel heights (the robot height measurements are closer in height to the tassel measurements than the leaf node measurements).\u0026nbsp;Measurements delivered by the TerraSentia robots, using LiDAR data were well correlated (p\u0026lt;0.001) with human-acquired plant height measurement with an R\u003csup\u003e2\u003c/sup\u003e value of 0.97 and a slope of 0.97 (Fig. S2D).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eEar height measurements were validated on 79 experimental units having approximately 40 plants each. Manual measurements were carried out\u0026nbsp;at R5 (kernels are dented on the ear\u003csup\u003e59\u003c/sup\u003e)\u0026nbsp;with every third plant measured manually. The distance from the soil surface to the ear attachment point on the stem (ear node) was recorded using the same pole used for plant height.\u0026nbsp;Measurements delivered by the TerraSentia robots were well correlated (p\u0026lt;0.001) with human-acquired plant height measurement with an R\u003csup\u003e2\u003c/sup\u003e value of 0.87 and a slope of 0.83 (Fig. S2A).\u003c/p\u003e\n\u003cp\u003eLeaf area index was validated by recording data in each of the 50 experimental units\u0026nbsp;approximately two weeks after R1,\u0026nbsp;to collect imagery with a 720p camera with a hemispherical lens.\u0026nbsp;Following robotic data collection, the following process was conducted for manual LAI measurement. First, the length of rows on either side of the robot were measured to the nearest cm. The row spacing was 76 cm and thus the exact area could be calculated. Following row measurement, the number of plants in each row were counted to establish the plant density. Following row measurement and plant counting, all of the green leaves, that were orientated upward \u0026ge; 90\u003csup\u003eo\u003c/sup\u003e, were removed from four randomly selected plants (two from each row). These leaves were placed in a large plastic bag and transported to the lab. At the lab leaves were run through a LI-3100C leaf scanner (Li-Cor, Lincoln, Nebraska, USA). This machine determined the total leaf area in square cm for the four plants. This value was divided by four to determine an average leaf area per plant and further divided by 10,000 to produce leaf area per square meter. Leaf area per plant was multiplied by plant density to calculate leaf area index (leaf area m\u003csup\u003e-2\u003c/sup\u003e land area m\u003csup\u003e-2\u003c/sup\u003e). A total of 50 experimental units were used to validate the TerraSentia LAI measurement.\u0026nbsp;The correlation between robot measured LAI and human measured LAI was significant but the R\u003csup\u003e2\u003c/sup\u003e value was lower (0.5) than the height and diameter traits (Fig. S2C). \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study 1:\u003c/strong\u003e As part of the commercial breeding program of Corteva Agriscience trials are grown at several locations to expose pre-commercial lines to environments and management to determine crop response prior to further advancement. Five locations near Griswold, IA, Prairie Cty, IA, Cameron, IL, Table Grove, IL and Windfall, IN were selected. The TerraSentia robot, carrying the Real Sense camera, was used approximately three weeks prior to combine harvest to collect video images that would allow for ear detection and ear height calculation. \u0026nbsp;A human operator followed approximately two meters behind the robot to reorientate following any crashes and to save the videos at the end of each pass and turn the robot on the ends to start the next pass. Data were analyzed by looking at genotypic performance for the trait of interest, in this case ear height, both within single locations and across locations. For by-location analyses, we generate the best linear unbiased estimates (BLUEs) of germplasm effects by trial using ASREML\u003csup\u003e60\u003c/sup\u003e. To generate BLUEs, we first fit the spatial model (AR1 by AR1) with genotype effects as random. Next, we set the genotype to a fixed factor along with the other variance parameters and rerun the model. For across location analyses we generate the best linear unbiased predictors (BLUPs) using ASREML. In this model the location effect is considered a fixed effect, the genotype and the genotype by location interaction effects are considered random, and the residual variance is fixed at 1 and relies on the weighting for each genotype from the BLUE analysis. \u0026nbsp;One experiment was carried out at three locations (Cameron, IL, Table Grove, IL and Windfall, IN locations) and was used for the BLUP analysis (Table 1). Heritability is calculated as the ratio of the genotypic variance to total phenotypic variance\u003csup\u003e61\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study 2:\u0026nbsp;\u003c/strong\u003eA multi-treatment, replicated, small-plot (5.4 m long by 3 m wide) trial was established at a location near Windfall, Indiana in a randomized complete block design. Treatments included a whole plot treatment of N levels 0, 112, 224 and 336 kg N ha\u003csup\u003e-1\u003c/sup\u003e, a split-plot treatment of 4.4, 6.9, 9.4 and 11.9 plants m\u003csup\u003e-2\u003c/sup\u003e and a split-split plot treatment of six Pioneer (Corteva Agriscience, Johnston, IA, USA) hybrids. This trial was planted on 20 May 2022 using a custom designed vacuum planter (Almaco, Nevada, IA, USA) and the nitrogen treatments were applied on 24 May 2022 using urea-ammonium-nitrate (UAN). The UAN was injected approximately 20 cm from each row of a four-row experimental unit.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOn 1 July, 29 July and 18 August the TerraSentia robot was used to collect RGB imagery and LiDAR data on all 384 experimental units. The left camera was set at -5\u003csup\u003eo\u003c/sup\u003e to image the plant stems, the right camera was set a -30\u003csup\u003eo\u003c/sup\u003e to image the soil to plant interface, the front camera was set a 0\u003csup\u003eo\u003c/sup\u003e to collect basic plot survey data and the upward facing hemispherical lens was at 90\u003csup\u003eo\u003c/sup\u003e to collect video for leaf area index. The robot was operated in row-following mode with an operator following approximately two meters behind the robot to reorientate following any crashes and to save the data at the end of each pass and turn the robot on the ends to start the next pass. Data from the 29 July collection were selected for analysis due to their close proximity (~2 weeks after R1 (silk appearance) when the canopy reaches maximum height and LAI. The analysis was conducted using Proc Mixed in SAS\u003csup\u003e62\u003c/sup\u003e.\u0026nbsp;Fixed effects were N rate, Plant density, and Hybrid. Random effects were Replication x N rate, Replication x Plant density.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study 3:\u0026nbsp;\u003c/strong\u003eMaize hybrids were evaluated as described\u0026nbsp;in\u003csup\u003e63\u003c/sup\u003e within a nitrogen-responsive field site at the Crop Sciences Research and Education Center in Urbana, Illinois. The soil type at this field is a Drummer silty clay loam, pH 6.5, which provided 60-75 kg N ha\u003csup\u003e-1\u003c/sup\u003e in each year of the experiment based on both soil testing and measures of plant N recovery from unfertilized plots. \u0026nbsp;Hybrids were grown in a split-plot design, where the same hybrid was grown in paired plots (2 rows 5.3m long, 76cm row spacing) receiving either no fertilizer or N fertilizer applied at a rate of 200 kg N ha-1, as granular ammonium sulfate banded at the soil surface when the plants reached the V3 (3 leaf collars) growth stage. All hybrids were planted at a density of 86,000 plants ha\u003csup\u003e-1\u003c/sup\u003e, with three replicates of each N treatment. Trials were planted on 7 May (2021), 12 May (2022) and 4 May (2023) using a GPS-guided vacuum planter (SeedPro, Almaco, Nevada, IA, USA).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ePhenotyping with the robot was performed as described above for Case Study 2, with data collections on 1 September 2021, 17 August and 9 September 2022, and 25 July, 2023. \u0026nbsp;Ground truth measures of plant and ear height were made with a measuring stick on 5 plants per plot. Biomass and nitrogen concentrations of stalk plus leaves, cob and grain were determined from five plants of each plot harvested at physiological maturity (R6 growth stage, month of September) as described in\u003csup\u003e63\u003c/sup\u003e\u003cstrong\u003e.\u003c/strong\u003e\u0026nbsp; Data analyses were performed in R Statistical Software\u003csup\u003e64\u003c/sup\u003e (R Core Team 2021) using the corrplot\u003csup\u003e65\u003c/sup\u003e and ggplot2\u003csup\u003e66\u003c/sup\u003e packages. \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eCase study 4:\u003c/strong\u003e During 2021 a multiple N rate by multiple hybrid experiment was established at two grower fields. One field was located in northeast Nebraska near the city of Homer and one field located in Southwest Indiana near the city of Owensville. The whole plot treatment was all N applied at planting or 76 kg N ha\u003csup\u003e-1\u003c/sup\u003e applied at planting with the remainder applied at V5 (5 collared leaves present). The split-plot was five nitrogen rates of 0, 67, 134, 201 and 269 kg N ha\u003csup\u003e-1\u003c/sup\u003e. The split-split plot was hybrid consisting of either P1197AM or P1359AM (Corteva Agriscience, Johnston, IA). This pattern continued to be replicated as many times as needed until the strip across the field was filled (Fig. 4). The experimental unit size was 5.4 m by 3 m. The number of experimental unit ranges were from as few as 108 (584 m) ranges long to as long as 250 (1,350 m) ranges long.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eApproximately two weeks after flowering (silk appearance) and pollen shed, the robots were used to collect leaf area index, stem diameter, and plant height data. The robot was run through the experimental units at a constant speed of 0.3 m sec\u003csup\u003e-1\u003c/sup\u003e. The left camera was set at -15\u003csup\u003eo\u003c/sup\u003e to video the plant stems, the right camera was set a -30\u003csup\u003eo\u003c/sup\u003e to video the soil to plant interface, the front camera was set a 0\u003csup\u003eo\u003c/sup\u003e to collect basic plot survey data and the upward facing hemispherical lens was at 90\u003csup\u003eo\u003c/sup\u003e to collect video for leaf area index. Grain yield was captured using a specialized plot combine that measured the weight and moisture of the grain for each experimental unit and standardized the grain weight at 155 g kg\u003csup\u003e-1\u003c/sup\u003e moisture. \u0026nbsp;Data were analyzed using Proc Mixed in SAS\u003csup\u003e62\u003c/sup\u003e with State (Nebraska or Indiana), N rate and Hybrid treated as fixed effects. Random effects were N rate x Replication nested within the specific field strip, field strip nested within State and Range nested within specific field strip.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase study 5:\u003c/strong\u003e A set of pre-commercial maize hybrids were evaluated at three locations near the towns of Hensall, Attwood, and Harriston all located in Ontario Canada. The robot was run through the experiment between four and six times depending on the location during the season between the V10 (10 collared leaves) and the R6 (maturity) growth stages. At each time point video images were collected using the hemispherical lens camera mounted on top of the robot. These videos were converted into a measurement of LAI. Plant height was also collected at each time point using the lidar mounted on the back of the robot.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFrom this time course data, the maximum LAI (achieved at R1, flowering, and maintained for approximately two weeks after flowering) could be determined for each location. An additional measurement, occurring on 23 September of 2022, was common among locations and selected to represent potential loss of green leaf area as the crop senesced prior to harvest which occurred 15-30 days later depending on the location. Weather data (rainfall and temperature) from 2022 were collected using weather station data within 10 km of the individual location. Grain yield was captured using a specialized plot combine that measured the weight and moisture of the grain for each experimental unit and standardized the grain weight at 155 g kg\u003csup\u003e-1\u003c/sup\u003e moisture. Raw LAI data were reviewed and values greater than three standard deviations away from the mean were considered outliers and removed from the visualization and analysis. Grain yield, plant height, and LAI at the R5 growth stage were subjected to a principal component analysis using R\u003csup\u003e64\u003c/sup\u003e.\u0026nbsp;\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors acknowledge Jan-Michael Schulze and Fred Gruis for initializing the collaboration and guiding the budgetary support for the yearly testing. The authors specifically thank Logan Anderson, Carmen Trainer and Melissa Grafton for their commitment to operate the robots at a number of USA field trial locations during multiple years of testing. Phenotyping efforts at the University of Illinois were supported in part by the National Science Foundation Science and Technology Center for Programmable Plant Systems (CROPPS, NSF award DBI-2019674), the Denton E. and Betty Alexander Professorship in Maize Breeding and Genetics, and a graduate fellowship from the Illinois Corn Marketing Board to Catherine Li.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eJ.L. DeBruin: data collection, data organization, data analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eT. Aref: image organization, data analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eR. Hensley: data collection, data organization, and manuscript development\u003c/p\u003e\n\u003cp\u003eH. Underwood: data collection, data organization, and manuscript development\u003c/p\u003e\n\u003cp\u003eS. Tirado Tolosa: data collection, data organization, data analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eM. McGuire: image organization, image analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eC. Soman: image organization, image analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eG. Nystrom: data collection, data organization\u003c/p\u003e\n\u003cp\u003eE. Parkinson: data collection, data organization\u003c/p\u003e\n\u003cp\u003eC. Li: data collection, data organization, data analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003eS. Moose: data analysis and manuscript development\u003c/p\u003e\n\u003cp\u003eG. Chowdhary: image organization, image analysis, and manuscript development\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interest declaration\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003eJ.L. DeBruin, R. Hensley, H. Underwood, and S. Tirado Tolosa declare no competing interests.\u003c/li\u003e\n \u003cli\u003eG. Chowdhary, C. Soman, M. McGuire, and T. Aref declare no competing interests.\u003c/li\u003e\n \u003cli\u003eS. Moose. C. Li, G. Nystrom and E. Parkinson: declare no competing interests.\u003c/li\u003e\n\u003c/ul\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors acknowledge Jan-Michael Schulze and Fred Gruis for initializing the collaboration and guiding the budgetary support for the yearly testing. The authors specifically thank Logan Anderson, Carmen Trainer and Melissa Grafton for their commitment to operate the robots at a number of USA field trial locations during multiple years of testing. Phenotyping efforts at the University of Illinois were supported in part by the National Science Foundation Science and Technology Center for Programmable Plant Systems (CROPPS, NSF award DBI-2019674), the Denton E. and Betty Alexander Professorship in Maize Breeding and Genetics, and a graduate fellowship from the Illinois Corn Marketing Board to Catherine Li.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eAdditional Information\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eCorresponding author: Jason DeBruin @ [email protected]\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBasavaraj, P.S., Rane, J., Prathibha, M.D., Boraiah, K.M. \u0026amp; Kumar, H. Advances in high-throughput phenotyping of maize (zea mays l.) for climate resilience. Maize Improvement. S.H. Wani et al., (eds). (2023). (doi.org/10.1007/978-3-031-21640-4_12)\u003c/li\u003e\n\u003cli\u003eCassman, K.G. \u0026amp; Grassini, P. A global perspective on sustainable intensification research. Nature Sustainability 3:262-268 (2020). (doi.org/10.1038/s41893-020-0507-8)\u003c/li\u003e\n\u003cli\u003eCooper, M. \u0026amp; Messina, C.D. Can we harness \u0026ldquo;enviromics\u0026rdquo; to accelerate crop improvement by integrating breeding and agronomy. Front. Plant Sci. 12:735143. (2021). (doi:10.3389/fpls.2021.735143)\u003c/li\u003e\n\u003cli\u003eCobb, J.N., DeClerck, G., Greenberg, A., Clark, R. \u0026amp; McCouch, S. Next-generation phenotyping: requirements and strategies for enhancing our understanding of genotype-phenotype relationships and its relevance to crop improvement. Theor. Appl. Genet. 126:867-887 (2013). (doi 10.1007/s00122-013-2066-0)\u003c/li\u003e\n\u003cli\u003eMerrick, L.F., Herr, A.W., Sandhu, K.S., Lozada, D.N. \u0026amp; Carter, A.H. Optimizing plant breeding programs for genomic selection. Agronomy 12:174 (2022). (https://doi.org/10.3390/agronomy12030714).\u003c/li\u003e\n\u003cli\u003eChawade, A., van Ham, J., Blomquist, H., Bagge, O., Alexandersson, E. \u0026amp; Ortiz, R. High-throughput field-phenotyping tools for plant breeding and precision agriculture. Agronomy 9:258 (2019). (doi:10.3390/agronomy/9050258).\u003c/li\u003e\n\u003cli\u003eMinervini, M., Scharr, H. \u0026amp; Tsaftaris, S.A. Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Processing Magazine 126 (2015). (doi:10.1109/MSP.2015.2405111)\u003c/li\u003e\n\u003cli\u003eAraus, J.L. \u0026amp; Kefauver, S.C. Breeding to adapt agriculture to climate change: affordable phenotyping solutions. Current Opinion Plant Biology 45:237-247. (2018). (doi.org/10.1016/j.pbi.2018.05.003)\u003c/li\u003e\n\u003cli\u003eReynolds, D., Baret, F., Welcker, C., Bostrom, A., Ball, J., Cellini, F., Lorence, A., Chawade, A., Khafif, M., Noshita, K., Mueller-Linow, M., Zhou, J. \u0026amp; Tardieu, F. What is cost-efficient phenotyping? Optimizing costs for different scenarios. Plant Sci. 282:14-22 (2019). (https://doi.org/10.1016-j.plantsci.2018.06.015)\u003c/li\u003e\n\u003cli\u003eBlancon, J., Buet, C., Dubreuil, P., Tixier, M.H., Baret, F. \u0026amp; Praud, S. Maize green leaf area index dynamics: genetic basis of a new secondary trait for grain yield in optimal and drought conditions. Theor. Appl. Genet. 137:68 (2024). https://doi.org/10.1007/s00122-024-04572-6.\u003c/li\u003e\n\u003cli\u003eWashburn, J.D., Burch, M.B. \u0026amp; Valdes Franco, J.A. Predictive breeding for maize: making use of molecular phenotypes, machine learning, and physiological crop models. Crop Sci. 1-17 (2019). (https://doi.org/10.1002/csc2.20052)\u003c/li\u003e\n\u003cli\u003eCooper, M., Messina, C.D., Podlich, D., Totir, L.R., Baumgarten, A., Hausmann, N.J., Wright, D. \u0026amp; Graham, G. Predicting the future of plant breeding: complementing empirical evaluation with genetic prediction. Crop and Pasture Sci. 65:311-336 (2014). (doi.org/10.1071/CP14007)\u003c/li\u003e\n\u003cli\u003eAssefa, Y., Carter, P., Hinds, M., Bhalla, G., Schon, R., Jeschke, M., Paszkiewicz, S., Smith, S. \u0026amp; Ciampitti, I.A. Analysis of long term study indicates both agronomic optimal plant density and increase maize yield per plant contributed to yield gain. Scientific Reports 8:4937 (2018). (doi:10.1038/s41598-018-23362-x)\u003c/li\u003e\n\u003cli\u003eRansom, C.J., Clark, J., Mac Bean, G., Bandura, C., Shafer, M.E., Kitchen, N.R., Camberato, J.J., Carter, P.R., Ferguson, R.B., Fern\u0026aacute;ndez, F., Laboski, C.A.M., Myers, D.B., Nafziger, E.D., Sawyer, J.E. \u0026amp; J. Shanahan. Data from a public-industry partnership for enhancing corn nitrogen research. Agron. J. 113:4429-4436 (2021). (doi:10.1002/agj2.20812)\u003c/li\u003e\n\u003cli\u003eVega, C.R.C., Andrade, F.H., Sadras, V.O., Uhart, S.A. \u0026amp; Valentinuz, O.R. Seed number as a function of growth. a comparative study in soybean, sunflower, and maize. Crop Sci 41:748-754 (2001). (doi.org/10.2135/cropsci2001.413748x)\u003c/li\u003e\n\u003cli\u003eBoomsma, C.R., Santini, J.B., Tollenaar, M. \u0026amp; Vyn, T.J. Maize morphophysiological responses to intense crowding and low nitrogen availability: an analysis and review. Agron. J. 101:1426-1452 (2009). (doi:10.2134/agronj2009.0082)\u003c/li\u003e\n\u003cli\u003eCiampitti, I.A., Zhang, H., Friedemann, P. \u0026amp; Vyn, T.J. Potential physiological frameworks for mid-season field phenotyping of final plant nitrogen uptake, nitrogen use efficiency, and grain yield in maize. Crop Sci. 52:2728-2742 (2012). (doi:10.2135/cropsci2012.05.0305)\u003c/li\u003e\n\u003cli\u003eDeBruin, J., Messina, C.D., Munaro, E., Thompson, K., Conlon-Beckner, C., Fallis, L., Sevenich, D.M., Gupta, R. \u0026amp; Dhugga, K.S. N distribution in maize plant as a marker for grain yield and limits on its remobilization after flowering. Plant Breeding 132:500-505 (2013). (doi.10.1111/pbr.12051)\u003c/li\u003e\n\u003cli\u003eMadec, S., Baret, F., de Solan, B., Thomas, S., Dutartre, D., Jezequel, S., Hemmerl\u0026eacute;, M., Colombeau, G. \u0026amp; Comar, A. High-throughput phenotyping of plant height; comparing unmanned aerial vehicles and ground lidar estimates. Front. plant Sci. 8:2002 (2017). (doi:10.3389/fpls.2017.02002)\u003c/li\u003e\n\u003cli\u003eLiu, S., Jin, X., Nie, C., Wang, S., Yu, X., Cheng, M., Shao, M., Wang, Z., Tuohuti, N., Bai, Y. \u0026amp; Liu, Y. Estimating leaf area index using unmanned aerial vehicle data: shallow vs. deep machine learning algorithms. Plant Physiology. 187:1551-1576 (2021). (doi:10.1093/plphyso/kiab322)\u003c/li\u003e\n\u003cli\u003eTam\u0026aacute;s, A., Kov\u0026aacute;cs, E., Horv\u0026aacute;th, E., Juh\u0026aacute;sz, C., Rad\u0026oacute;cz, L., R\u0026aacute;tonyi, T. \u0026amp; Rag\u0026aacute;n, P. Assessment of NDVI dynamics of maize (zea mays l.) and its relation to grain yield in a polyfactorial experiment based on remote sensing. Agriculture 13:689 (2023). (doi.org/10.3390/agriculture13030689).\u003c/li\u003e\n\u003cli\u003eNguyen, C., Sagan, V., Bhadra, S. \u0026amp; Moose, S.P. UAV multisensory data fusion and multi-task deep learning for high-throughput maize phenotyping. Sensors 23(4): 1827 (2023). \u003cem\u003ehttps://doi.org/10.3390/s23041827\u003c/em\u003e \u003c/li\u003e\n\u003cli\u003eWhite, J.W. \u0026amp; Conley, M.M. A flexible, low-cost cart for proximal sensing. Crop Sci. 53:1646-1649 (2013). (doi:10.2135/cropsci2013.01.0054)\u003c/li\u003e\n\u003cli\u003eBao, Y., Tang, L., Srinivasan, S. \u0026amp; Schnable, P.S. Field -based architectural traits characterization of maize plant using time-of-flight 3d imaging. Biosystems Engineering 178:86-101 (2019). (doi.org/10.1016/j.biosystemseng.2018.11.005) \u003c/li\u003e\n\u003cli\u003eBarker III, J., Zhang, N., Sharon, J., Steeves, R., Wang, X., Wei, Y. \u0026amp; Poland, J. Development of a field-based high throughput mobile phenotyping platform. Computers and Electronics in Agriculture 122:74-85 (2016). \u003c/li\u003e\n\u003cli\u003eZhang, Z., Kayacan, E., Thompson, B. \u0026amp; Chowdhary, G. High precision control and deep learning-based corn standing counting algorithms for agricultural robot. Autonomous Robots 44:1289-1302 (2020). \u003c/li\u003e\n\u003cli\u003eFan, Z., Sun, N., Qiu, Q., Li, T., Feng, Q. \u0026amp; Zhao, C. In situ measuring stem diameters of maize crops with a high-throughput phenotyping robot. Remote Sens. 14:1030 (2022). (https://doi.org/10.3390/rs14041030)\u003c/li\u003e\n\u003cli\u003eMueller-Sim, T., Jenkins, M., Abel, J. \u0026amp; Kantor. G. 2017. The robotanist: a ground-based agricultural robot for high-throughput crop phenotyping. Robotics Institute Carnegie Mellon University Paper-ICRA2017-Robotanist.pdf (cmu.edu) (verified 4-19-2024)\u003c/li\u003e\n\u003cli\u003eBanan, D., Paul, R.E., Feldman, M.J., Holmes, M.W., Schlake, H., Baxter, I., Jiang, H. \u0026amp; Leakey, A.D.B. High-fidelity detection of crop biomass quantitative trait loci from low-cost imaging in the field. Plant Direct 2018:1-8 (2018). (doi:10.1002/pld3.41)\u003c/li\u003e\n\u003cli\u003eXu, R. \u0026amp; Li. C. 2021. A modular agricultural robotic system (MARS) for precision farming: concept and implementation. J. Field Robotics 39:387-409 (2021). (doi:10.1002/rob.22056)\u003c/li\u003e\n\u003cli\u003eGage, J.L., Richards, E., Lepak, N., Kaczmar, N., Soman, C., Chowdhary, G., Gore, M.A. \u0026amp; Buckler, E.S. In-field whole-plant maize architecture characterized by subcanopy rovers and latent space phenotyping. Plant Phenome J. 2:190011 (2019). (doi:10.2135/tppj2019.07.0011)\u003c/li\u003e\n\u003cli\u003eUbbens, J., Cieslak, M., Prusinkiewicz, P., Parkin, I., Ebersbach, J. \u0026amp; Staviness, I. Latent space phenotyping: automated image-based phenotyping for treatment studies. Plant Phenomics 2020, Article ID 5801869 (2020). (https://doi.org/10.34133/2020/5801869)\u003c/li\u003e\n\u003cli\u003eChebrolu, N., Lottes, P., Schaefer, A., Winterhalter, W., Burgard, W. \u0026amp; Stachniss, C. Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields. Robotics Research 36:1045-1052 (2017). (doi:10.1177/0278364917720510)\u003c/li\u003e\n\u003cli\u003eBenet, B., Dubos, C., Maupas, F., Malatesta, G. \u0026amp; Lenain, R. Development of autonomous robotic platforms for sugar beet crop phenotyping using artificial vision. AGENG Conference, Jul 2018, Wageningen, Netherlands. 8 p. hal-02155159 (2018). \u003c/li\u003e\n\u003cli\u003eMurman, J.N. Flex-Ro: a robotic high throughput field phenotyping system. Biological Systems Engineering-Dissertations, Theses, and Student Research 99 (2019). University of Nebraska. https://digitalcommons.unl.edu/biosysengdiss/99\u003c/li\u003e\n\u003cli\u003eTu, X., Gai, J. \u0026amp; Tang, L. Robust navigation control of a 4WD/4WS agricultural robotic vehicle. Computers and Electronics in Agriculture 164:104892 (2019). (https://doi.org/10.1016/j/compag.2019.104892)\u003c/li\u003e\n\u003cli\u003eHiguti, V. A., Velasquez, A.E., Magalhaes, D.V., Becker, M. \u0026amp; Chowdhary, G. Under canopy light detection and ranging‐based autonomous navigation. Journal of Field Robotics, 36(3), 547-567 (2019).\u003c/li\u003e\n\u003cli\u003eSivakumar, A.N., Modi, S., Gasparino, M.V., Ellis, C., Velasquez, A.E.B., Chowdhary, G. \u0026amp; Gupta, S. Learned visual navigation for under-canopy agricultural robots. Robotics Science and Systems, (2021).\u003c/li\u003e\n\u003cli\u003eSivakumar, A.N., Gasparino, M.V., McGuire, M., Higuti, V.A., Akcal, M.U. \u0026amp; Chowdhary, G. Lessons from deploying CropFollow++: under-canopy agricultural navigation with keypoints. Robotics Science and Systems, Delft, Netherlands, (2024).\u003c/li\u003e\n\u003cli\u003eGasparino, M.V., Higuti, V.A., Sivakumar, A.N., Velasquez, A.E., Becker, M. \u0026amp; Chowdhary, G. Cropnav: a framework for autonomous navigation in real farms. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 11824-11830). IEEE (2023)\u003c/li\u003e\n\u003cli\u003eVelasquez, A.E.B., Higuti, V.A.H., Gasparino, M.V., Sivakumar, A.N., Becker, M. \u0026amp; Chowdhary, G. Multi-sensor fusion based robust row following for compact agricultural robots. Journal of Field Robotics, 2:1291\u0026ndash;1319 (2022).\u003c/li\u003e\n\u003cli\u003eShakoor, N., Lee, S. \u0026amp; Mockler, T.C. High throughput phenotyping to accelerate crop breeding and monitoring of diseases in the field. Current Opinion in Plant Biology 38:184-192. (2017). (http://dx.doi.org/10.1016/j.pbi.2017.05.006.\u003c/li\u003e\n\u003cli\u003eYoung, S.N. Kayacan, E. \u0026amp; Peschel, J.M. Design and field evaluation of a ground robot for high-throughput phenotyping of energy sorghum. Precision Agriculture (2018). (doi.org/10.1007/s11119-018-9601-6)\u003c/li\u003e\n\u003cli\u003eTirado, S.B., Hirsch, C.N. \u0026amp; Springer, N.M. Utilizing temporal measurements from UAVs to assess root lodging in maize and its impact on productivity. FCR 262:108014 (2021). (doi.org/10.1016/j.fcr.2020.108014)\u003c/li\u003e\n\u003cli\u003eVanous, A., Gardner, G., Blanco, M., Martin-Schwarze, A., Lipka, A.E., Flint-Garcia, S., Bohn, M., Edwards, J. \u0026amp; L\u0026uuml;bberstedt, T. Association mapping of flowering and heigh traits in germplasm enhancement of maize doubled haploid (gem-dh) lines. Plant Genome 11:170083 (2018). (doi10.3835/plantgenome2017.09.0083)\u003c/li\u003e\n\u003cli\u003ePeiffer, J.A., Romay, M.C., Gore, M.A., Flint-Garcia, S.A., Zhang, Z., Millar, M.J., Gardner, C.A.C., McMullen, M.D., Holland, J.B., Bradbury, P.J. \u0026amp; Buckler, E.S. The genetic architecture of maize height. Genetic 196:1337-1356 (2014). (doi:10.1534/genetics.113.159152/-/DC1)\u003c/li\u003e\n\u003cli\u003eFromme, D.D., Spivey, T.A. \u0026amp; Grichar, W.J. Agronomic response of corn (zea mays l.) hybrids to plant populations. International Journal of Agronomy 3589768 (2019). (doi.org/10.1155/2019/3589768)\u003c/li\u003e\n\u003cli\u003eCheng, C.Y., Li, Y., Varala, K., Huang, J., Kim, G.J., Halim, J., Arp, J., Bubert, J., Chih, H-J.S., Levinson, G.L., Park, S.H., Cho, H.Y., Moose, S.P. \u0026amp; Coruzzi, G.M. Evolutionarily informed machine learning enhances the power of predictive gene-to-phenotype relationships. Nature Communications 12: 5627 (2021). https://doi.org/10.1038/s41467-021-25893-w\u003c/li\u003e\n\u003cli\u003eCorrendo, A.A., Tremblay, N., Coulter, J.A., Ruiz-Diaz, D., Franzen, D., Nafziger, E., Prasad, V., Rosso, L.H.M., Steinke, K., Du, J., Messina, C.D. \u0026amp; Ciampitti, I.A. Unraveling uncertainty drivers of the maize yield response to nitrogen: a Bayesian and machine learning approach. Agricultural and Forest Meteorology 311:108668 (2021). (doi.org/10.1016/j.agrformet.2021.108668)\u003c/li\u003e\n\u003cli\u003eGaffney, J., Schussler, J., L\u0026ouml;ffler, C., Cai, W., Paszkiewicz, S., Messina, C., Groeteke, J., Keaschall, J. \u0026amp; Cooper. M. Industry-scale evaluation of maize hybrids selected for increased yield in drought-stress conditions of the US corn belt. Crop Sci. 55:1608-1618 (2015). (doi.org/10.2135/cropsci2014.09.0654).\u003c/li\u003e\n\u003cli\u003eRebetzke, G.J., Jimenez-Berni, J., Fischer, R.A., Deery, D.M. \u0026amp; Smith, D.J. Review: high-throughput phenotyping to enhance the use of crop genetic resources. Plant Sci. 282:40-48 (2019). (doi.org/10.1016/j-plantsci.2018.06.017)\u003c/li\u003e\n\u003cli\u003eBerry, P.M., Baker, C.J., Hatley, D., Dong, R., Wang, X., Blackburn, G.A., Miao, Y., Sterling, M. \u0026amp; Whyatt, J.D. Development and application of a model for calculating the risk of stem and root lodging in maize. FCR 262:108037 (2021).\u003c/li\u003e\n\u003cli\u003eVeenstra, R.L., Messina, C.D., Berning, D., Haag, L.A., Carter, P., Hefley, T.J., Prasad, P.V.V. \u0026amp; Ciampitti, I.A. Tiller biomass in low plant-density corn enhances transient C sink without direct harvest index detriment. FCR 292:1068804 (2023). (doi.org/10.1016/j.fcr.2022.108804)\u003c/li\u003e\n\u003cli\u003eWallman, M. \u0026amp; Zakhor, A. Height estimation of biomass sorghum in the field using li-dar. IS\u0026amp;T International Symposium on Electronic Imaging. Computational Imaging XVII (2019). (doi.org/10.2352/ISSN.2470-1173.2019.13.COIMG-137)\u003c/li\u003e\n\u003cli\u003eMcGuire, M., Aref, T., Syraj, A., Shukla, S., Kumar, A., Moose, S., Diers, B. \u0026amp; Chowdhary, G. High-throughput phenotyping for essential crop traits. Association for the advancement of Artificial Intelligence (2022). (www.aaai.org)\u003c/li\u003e\n\u003cli\u003eMiller, J.B. A formula for average foliage density. Australian Journal of Botany 15:141-144 (1967). (doi.org/10.1071/BT9670141)\u003c/li\u003e\n\u003cli\u003eLiu, S., Jin, X., Nie, C., Wang, S., Yu, X., Cheng, M., Shao, M., Wang, Z., Tuohuti, N., Bai, Y. \u0026amp; Liu, Y. Estimating leaf area index using unmanned aerial vehicle data: shallow vs. deep machine learning algorithms. Plant Physiology. 187:1551-1576 (2021). (doi:10.1093/plphyso/kiab322)\u003c/li\u003e\n\u003cli\u003eChen, Y., Jiao, S., Cheng, Y., Wei, H., Sun, L. \u0026amp; Sun, Y. LAI-NOS: an automatic network observation systems for leaf area index on hemispherical photography. Agricultural and Forest Meteorology 322:108999 (2022). (doi.org/10.1016/j.agrformet.2022.108999)\u003c/li\u003e\n\u003cli\u003eRitchie, S.W. \u0026amp; Hanway, J.J. How a corn plant develops (Special Report No. 48). Iowa State University of Science and Technology, Cooperative Extension Service. (1982)\u003c/li\u003e\n\u003cli\u003eGilmour, A.R., Cullis, B.R. \u0026amp; Verbyla, A.P. Accounting for natural and extraneous variation in the analysis of field experiments. Journal of Agricultural, Biological, and Environmental Statistics 2:269-273 (1997). (doi.org/10.2307/1400446)\u003c/li\u003e\n\u003cli\u003eFalconer, D.S. \u0026amp; Mackay, T.F.C. Introduction to Quantitative Genetics, 4\u003csup\u003eth\u003c/sup\u003e Edition, Longmans Green, Harlow, Essex, UK (1996).\u003c/li\u003e\n\u003cli\u003eSAS Instituste Inc. SAS/STAT\u0026reg; 14.3 User\u0026rsquo;s Guide. Cary, NC: SAS Institute (2017).\u003c/li\u003e\n\u003cli\u003eCheng, C.Y., Li, Y., Varala, K., Huang, J., Kim, G.J., Halim, J., Arp, J., Bubert, J., Chih, H.-J. S., Levinson, G.L., Park, S. H., Cho, H. Y., Moose, S.P. \u0026amp; Coruzzi, G.M. Evolutionarily informed machine learning enhances the power of predictive gene-to-phenotype relationships. \u003cem\u003eNature Communications\u003c/em\u003e 12: 5627 (2021). https://doi.org/10.1038/s41467-021-25893-w\u003c/li\u003e\n\u003cli\u003eR Core Team R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, (2024) Vienna, Austria. \u0026lt;https://www.R-project.org/.\u003c/li\u003e\n\u003cli\u003eTaiyun W. \u0026amp; Viliam S. R package \u0026apos;corrplot\u0026apos;: Visualization of a Correlation Matrix (Version 0.92) (2021). Available from https://github.com/taiyun/corrplot\u003c/li\u003e\n\u003cli\u003eWickham, H. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5187688/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5187688/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eUnderstanding phenotypic plasticity in maize (\u003cem\u003eZea mays\u003c/em\u003eL.) is a current grand challenge for continued crop improvement. \u0026nbsp;Measuring the interactive effects of genetics, environmental factors, and management practices such as nitrogen rate (GxExM) on crop performance is time-consuming, expensive, and a major bottleneck to continued yield advancement. We demonstrate that a novel autonomous robotic platform, capable of collecting biologically relevant and commonly measured phenotypes, within a maize canopy at high-throughput, low-cost, and high-volume is now a reality. Field teams used multiple TerraSentia autonomous ground robots developed by EarthSense, Inc. (Champaign, IL) to capture data using a suite of low-cost sensors from nearly 200,000 experimental units, located at 142 unique research fields in the USA and Canada, across five years. Novel computer vision and machine learning algorithms, developed by EarthSense, Inc., analyzed these in-canopy multi-sensor data to deliver ground-truth validated plant height, ear height, stem diameter, and leaf area index at multiple time points during each season. We show the robot measured these phenotypes with high accuracy and reliability, at scales sufficient to functionally dissect interactions between genotypes and nitrogen rates in several environments. The results show that within-row autonomous field robots hold great promise to increase understanding of GxExM interactions in maize research and decrease the amount of human labor required for plant phenotyping.\u003c/p\u003e","manuscriptTitle":"Breaking the field phenotyping bottleneck in maize with autonomous robots","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-29 08:25:54","doi":"10.21203/rs.3.rs-5187688/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"communications-biology","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"commsbio","sideBox":"Learn more about [Communications Biology](http://www.nature.com/commsbio/)","snPcode":"","submissionUrl":"","title":"Communications Biology","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Communications Series","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"811e0142-26b6-4947-b4fc-d7618dbaf54b","owner":[],"postedDate":"October 29th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":39042929,"name":"Biological sciences/Plant sciences/Plant breeding"},{"id":39042930,"name":"Biological sciences/Plant sciences/Plant physiology"},{"id":39042931,"name":"Biological sciences/Plant sciences/Plant development"}],"tags":[],"updatedAt":"2025-03-22T07:10:34+00:00","versionOfRecord":{"articleIdentity":"rs-5187688","link":"https://doi.org/10.1038/s42003-025-07890-7","journal":{"identity":"communications-biology","isVorOnly":false,"title":"Communications Biology"},"publishedOn":"2025-03-21 04:00:00","publishedOnDateReadable":"March 21st, 2025"},"versionCreatedAt":"2024-10-29 08:25:54","video":"","vorDoi":"10.1038/s42003-025-07890-7","vorDoiUrl":"https://doi.org/10.1038/s42003-025-07890-7","workflowStages":[]},"version":"v1","identity":"rs-5187688","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5187688","identity":"rs-5187688","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}