Adaptive Active Positioning of Camellia oleifera Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm

Abstract

Camellia oleifera fruits are randomly distributed in an orchard, and the fruits are easily blocked or covered by leaves. In addition, the colors of leaves and fruits are alike, and flowers and fruits grow at the same time, presenting many ambiguities. The large shock force will cause flowers to fall and affect the yield. As a result, accurate positioning becomes a difficult problem for robot picking. Therefore, studying target recognition and localization of Camellia oleifera fruits in complex environments has many difficulties. In this paper, a fusion method of deep learning based on visual perception and image processing is proposed to adaptively and actively locate fruit recognition and picking points for Camellia oleifera fruits. First, to adapt to the target classification and recognition of complex scenes in the field, the parameters of the You Only Live Once v7 (YOLOv7) model were optimized and selected to achieve Camellia oleifera fruits’ detection and determine the center point of the fruit recognition frame. Then, image processing and a geometric algorithm are used to process the image, segment, and determine the morphology of the fruit, extract the centroid of the outline of Camellia oleifera fruit, and then analyze the position deviation of its centroid point and the center point in the YOLO recognition frame. The frontlighting, backlight, partial occlusion, and other test conditions for the perceptual recognition processing were validated with several experiments. The results demonstrate that the precision of YOLOv7 is close to that of YOLOv5s, and the mean average precision of YOLOv7 is higher than that of YOLOv5s. For some occluded Camellia oleifera fruits, the YOLOv7 algorithm is better than the YOLOv5s algorithm, which improves the detection accuracy of Camellia oleifera fruits. The contour of Camellia oleifera fruits can be extracted entirely via image processing. The average position deviation between the centroid point of the image extraction and the center point of the YOLO recognition frame is 2.86 pixels; thus, the center point of the YOLO recognition frame is approximately considered to be consistent with the centroid point of the image extraction.

Tea leaf disease detection and identification based on YOLOv7 (YOLO-T)

“…We compared our version (YOLO-T) to the most recent version (YOLOv5). A new study found that YOLOv7 required less training time than YOLOv5, which contradicts our findings 47 . This variance in training duration may be due to the utilization of graphics processing units (GPUs).…”

“…According to a study, YOLOv7 has higher inference in speed and accuracy when compared with other algorithms such as YOLOR, PP-YOLOE, YOLOX, Scaled-YOLOv4, and YOLOv5 (r6.1) 57 . In several recent research, the detection accuracy and precision of the YOLOv7 algorithm have also been evaluated and reported 47,56,[58][59][60] .…”

“…Despite a little drop in speed, this method can still meet the real-time requirements of practical tea leaf disease detection applications. www.nature.com/scientificreports/ among researchers for precision agriculture applications such as disease detection, weed control, fruit recognition, etc [45][46][47] . By identifying the diseased portion, farmers can employ more effective measures for disease control.…”

“…Gallo et al 45 applied the YOLOv7 model on a dataset of Chicory plants to identify weeds. Similarly, YOLOv7 architecture was used to detect fruits in orchards, making it easier for harvesting robots to locate and collect fruits 46,47 .…”

A longan yield estimation approach based on UAV images and deep learning

“…With the development of sensor and computer technology, deep learning approaches have been widely developed and applied by researchers, and deep learning exhibits an excellent learning ability in cases involving the extraction of features from complex images. In recent years, with the demand for intelligence in the agricultural field, an increasing number of researchers have used deep learning technology to process collected image data for various tasks ( Wang et al., 2019 ; da Silva et al., 2021 ; de Medeiros et al., 2021 ; Zhou et al., 2022 ), including fruit recognition ( Gao et al., 2020 ; Xiong et al., 2020 ), classification of plants ( Flores et al., 2021 ), classification of pests and diseases ( Anagnostis et al., 2021 ; Singh et al., 2021 ), monitoring of crop growth state based on remote sensing ( Ma et al., 2019 ; Paoletti et al., 2019 ), nondestructive testing and grading of fruit ( Koirala et al., 2019 ), and animal behavior analysis ( Norouzzadeh et al., 2018 ). To sum up, the deep learning model has stronger feature extraction ability, and it can effectively solve complex nonlinear problems.…”