Recognition of citrus fruit and planning the robotic picking sequence in orchards

To improve the operational efficiency of and to prevent possible collision damage in the picking of citruses by robots in densely planted complex orchards, this study proposes an algorithm based on YOLOv5 for recognizing citruses and planning a picking sequence. First, the convolutional block attention module (CBAM) is embedded into YOLOv5 to improve the feature extraction capability of the network and mitigate missed detection of occluded targets and small targets. Simultaneously, the bounding loss function is optimized to improve the positioning accuracy of the bounding box. This combined model is used to recognize and localize citruses. Then, a three-dimensional model of citrus fruit was established, and an adaptive pheromone concentration updating mechanism was introduced on the basis of the ant colony algorithm to dynamically judge the picking order of citruses and determine the optimal picking sequence. We show the quantitative and qualitative results of our method in comparison to previous methods. In tests, the F1-score of this method for citrus in a densely planted environment was 92.41%, which is a 2.81% improvement compared to YOLOv5. Compared to stochastic planning, the proposed method can plan the picking sequence of citruses in the field of view in advance and shorten the picking path. In addition, extensive sequence planning experiments on other fruits validate the superior of the proposed method. Therefore, the method in this paper may provide new solutions for citrus anti-collision picking and orchard yield forecasting and new ideas for the intelligent fruit industry.