Apple detection and instance segmentation in natural environments using an improved Mask Scoring R-CNN Model

Abstract

The accurate detection and segmentation of apples during growth stage is essential for yield estimation, timely harvesting, and retrieving growth information. However, factors such as the uncertain illumination, overlaps and occlusions of apples, homochromatic background and the gradual change in the ground color of apples from green to red, bring great challenges to the detection and segmentation of apples. To solve these problems, this study proposed an improved Mask Scoring region-based convolutional neural network (Mask Scoring R-CNN), known as MS-ADS, for accurate apple detection and instance segmentation in a natural environment. First, the ResNeSt, a variant of ResNet, combined with a feature pyramid network was used as backbone network to improve the feature extraction ability. Second, high-level architectures including R-CNN head and mask head were modified to improve the utilization of high-level features. Convolutional layers were added to the original R-CNN head to improve the accuracy of bounding box detection (bbox_mAP), and the Dual Attention Network was added to the original mask head to improve the accuracy of instance segmentation (mask_mAP). The experimental results showed that the proposed MS-ADS model effectively detected and segmented apples under various conditions, such as apples occluded by branches, leaves and other apples, apples with different ground colors and shadows, and apples divided into parts by branches and petioles. The recall, precision, false detection rate, and F1 score were 97.4%, 96.5%, 3.5%, and 96.9%, respectively. A bbox_mAP and mask_mAP of 0.932 and 0.920, respectively, were achieved on the test set, and the average run-time was 0.27 s per image. The experimental results indicated that the MS-ADS method detected and segmented apples in the orchard robustly and accurately with real-time performance. This study lays a foundation for follow-up work, such as yield estimation, harvesting, and automatic and long-term acquisition of apple growth information.

A Machine Learning Approach for Automated Detection of Critical PCB Flaws in Optical Sensing Systems

“…Their main feature is fast speed but low accuracy. The two-stage algorithms mainly include RCNN [20], Fast RCNN [21], Faster RCNN [22], etc. Figure 3 compares the process differences between the two algorithms.…”

Apple Fruit Edge Detection Model Using a Rough Set and Convolutional Neural Network

“…However, the improvement in accuracy brought by this work also leads to an increase in the model's complexity, resulting in longer training and detection times. Moreover, it requires a larger dataset to encompass real-world scenarios [12]. Tian et al improved the YOLO-V3 model to effectively perform apple fruit object detection tasks across different growth stages in natural scenes.…”

High-precision apple recognition and localization method based on RGB-D and improved SOLOv2 instance segmentation

“…In recent years, along with the development of machine vision, the recognition and localization of apples have been extensively researched (Xia et al, 2022(Xia et al, , 2023Gai et al, 2023). Specific methods can be classified into three main categories: object detection-based (Huang et al, 2017;Hu et al, 2023), semantic segmentation-based (Jia et al, 2022b), and instance segmentation-based (Wang and He, 2022a). Object detection involves identifying and localizing objects in images and marking them with bounding boxes.…”