Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods

Abstract

With the increasing popularity of online fruit sales, accurately predicting fruit yields has become crucial for optimizing logistics and storage strategies. However, existing manual vision-based systems and sensor methods have proven inadequate for solving the complex problem of fruit yield counting, as they struggle with issues such as crop overlap and variable lighting conditions. Recently CNN-based object detection models have emerged as a promising solution in the field of computer vision, but their effectiveness is limited in agricultural scenarios due to challenges such as occlusion and dissimilarity among the same fruits. To address this issue, we propose a novel variant model that combines the self-attentive mechanism of Vision Transform, a non-CNN network architecture, with Yolov7, a state-of-the-art object detection model. Our model utilizes two attention mechanisms, CBAM and CA, and is trained and tested on a dataset of apple images. In order to enable fruit counting across video frames in complex environments, we incorporate two multi-objective tracking methods based on Kalman filtering and motion trajectory prediction, namely SORT, and Cascade-SORT. Our results show that the Yolov7-CA model achieved a 91.3% mAP and 0.85 F1 score, representing a 4% improvement in mAP and 0.02 improvement in F1 score compared to using Yolov7 alone. Furthermore, three multi-object tracking methods demonstrated a significant improvement in MAE for inter-frame counting across all three test videos, with an 0.642 improvement over using yolov7 alone achieved using our multi-object tracking method. These findings suggest that our proposed model has the potential to improve fruit yield assessment methods and could have implications for decision-making in the fruit industry.

A Seamless Deep Learning Approach for Apple Detection, Depth Estimation, and Tracking Using YOLO Models Enhanced by Multi-Head Attention Mechanism

“…One study, for example, developed a better method built on the YOLOv7 model to solve the poor performance of apple fruit detection due to the complex backdrop and occluded apple fruit. Other research employed an updated YOLOv7 framework and multi-object tracking algorithms to recognize and count apples in apple orchards [29,30]. The approach Computers 2024, 13, 83 7 of 25 dealt with transformers to determine apple ripeness from digitized photos of several apple varieties.…”

An SAR Imaging and Detection Model of Multiple Maritime Targets Based on the Electromagnetic Approach and the Modified CBAM-YOLOv7 Neural Network

“…The added prediction head is produced from the low-level and high-resolution feature maps to improve the capability of target feature extractions in the complex environment. The ELAN-H [57] is employed to increasingly improve the training and learning ability of the network, whose structure is given in Figure 12.…”

“…The SAR images are obtained by the SAR imaging model with the EM calculations of the scattering from the long and short cone targets, and sea surface samples are generated at different times in different sea states and radar incident conditions. Then, the SAR image data are framed and labeled by well-trained researchers with the help of the image annotation tool labelImg [58] ELAN-H [57] is employed to increasingly improve the training and learning ability of the network, whose structure is given in Figure 12.…”