Detection and Counting of Maize Leaves Based on Two-Stage Deep Learning with UAV-Based RGB Image

Abstract

Leaf age is an important trait in the process of maize (Zea mays L.) growth. It is significant to estimate the seed activity and yield of maize by counting leaves. Detection and counting of the maize leaves in the field are very difficult due to the complexity of the field scenes and the cross-covering of adjacent seedling leaves. A method was proposed in this study for detecting and counting maize leaves based on deep learning with RGB images collected by unmanned aerial vehicles (UAVs). The Mask R-CNN was used to separate the complete maize seedlings from the complex background to reduce the impact of weeds on leaf counting. We proposed a new loss function SmoothLR for Mask R-CNN to improve the segmentation performance of the model. Then, YOLOv5 was used to detect and count the individual leaves of maize seedlings after segmentation. The 1005 field seedlings images were randomly divided into the training, validation, and test set with the ratio of 7:2:1. The results showed that the segmentation performance of Mask R-CNN with Resnet50 and SmoothLR was better than that with LI Loss. The average precision of the bounding box (Bbox) and mask (Mask) was 96.9% and 95.2%, respectively. The inference time of single image detection and segmentation was 0.05 s and 0.07 s, respectively. YOLOv5 performed better in leaf detection compared with Faster R-CNN and SSD. YOLOv5x with the largest parameter had the best detection performance. The detection precision of fully unfolded leaves and newly appeared leaves was 92.0% and 68.8%, and the recall rates were 84.4% and 50.0%, respectively. The average precision (AP) was 89.6% and 54.0%, respectively. The rates of counting accuracy for newly appeared leaves and fully unfolded leaves were 75.3% and 72.9%, respectively. The experimental results showed the possibility of current research on exploring leaf counting for field-grown crops based on UAV images.

Performance evaluation of newly released cameras for fruit detection and localization in complex kiwifruit orchard environments

“…You Only Look Once version 5 extra large (YOLOv5x) shows great potential for object detection in machine vision. Compared with the other three versions of YOLOv5 series, YOLOv5x has large complexity (model depth and layer channel), relative fast detection speed and high detection accuracy (Cao et al, 2023; Xu et al, 2022). Accordingly, considering the advantages of this model, YOLOv5x was employed for kiwifruit and calyx detection in this paper.…”

Progress in the Application of CNN-Based Image Classification and Recognition in Whole Crop Growth Cycles

“…Wang et al [111] detected anomalies in tomato plants through improvements to the YOLO-Dense algorithm. Xu et al [112] detected and counted corn plants by acquiring corn leaf images using a UAV.…”

A Survey of Object Detection for UAVs Based on Deep Learning

“…Small objects R-DFPN [84] Google Earth 2018 Ship RSOD [85] UAVDT [68] 2022 Traffic Complex background SCRDet [86] DOTA [64] 2019 Multicategory Shao et al [87] UAV-head [87] 2021 Pedestrian FR-Transformer [88] UWHD [88] 2022 Agriculture Category imbalance Deng et al [14] NWPU VHR-10 [89] 2018 Multicategory MLD [90] Leaf [90] 2022 Agriculture…”

“…However, detecting and counting maize leaves is difficult because of a variety of plant disturbances and the cross-covering of the adjacent seedling leaves. Thus, Xu et al [90] proposed a method of detecting and counting maize leaves based on UAV images. To reduce the effect of weeds on leaf counting, maize seedlings were separated from the complex background by R-CNN technique.…”

“…Then, Yolov5 was used to detect and count the segmented leaves of maize seedlings. The flowchart of maize leaves method (MLD) presented in [90] is shown in Figure 17. The results of MLD method on the detection of maize leaves are shown in Figure 18 and Table 8.…”