The effect of data augmentation and network simplification on the image‐based detection of broccoli heads with Mask R‐CNN

Abstract

In current practice, broccoli heads are selectively harvested by hand. The goal of our work is to develop a robot that can selectively harvest broccoli heads, thereby reducing labor costs. An essential element of such a robot is an image‐processing algorithm that can detect broccoli heads. In this study, we developed a deep learning algorithm for this purpose, using the Mask Region‐based Convolutional Neural Network. To be applied on a robot, the algorithm must detect broccoli heads from any cultivar, meaning that it can generalize on the broccoli images. We hypothesized that our algorithm can be generalized through network simplification and data augmentation. We found that network simplification decreased the generalization performance, whereas data augmentation increased the generalization performance. In data augmentation, the geometric transformations (rotation, cropping, and scaling) led to a better image generalization than the photometric transformations (light, color, and texture). Furthermore, the algorithm was generalized on a broccoli cultivar when 5% of the training images were images of that cultivar. Our algorithm detected 229 of the 232 harvestable broccoli heads from three cultivars. We also tested our algorithm on an online broccoli data set, which our algorithm was not previously trained on. On this data set, our algorithm detected 175 of the 176 harvestable broccoli heads, proving that the algorithm was successfully generalized. Finally, we performed a cost‐benefit analysis for a robot equipped with our algorithm. We concluded that the robot was more profitable than the human harvest and that our algorithm provided a sufficient basis for robot commercialization.

Deep learning-based decision support system for weeds detection in wheat fields

“…The technical options that add random variations to generate more training data and improve the performance of the model [11]. The model used is the you only look once (YOLO) algorithm which is an efficient choice when we need real-time detection [12], without loss of too much accuracy and able to predict class labels and detects locations of weeds in order to take the better decision Table 1 [13]. Due to its high-speed inference, this model has the potential to be deployed on a single-board platform like the Raspberry Pi for weed detection and control it in real time through a set of features included in it.…”

Selective Harvesting Robotics: Current Research, Trends, and Future Directions

“…Kusumam et al [35] developed a 3D-vision algorithm using machine learning to detect broccoli heads in RGB-D images. Blok et al [36] studied the detection of broccoli heads using deep learning with a specific focus on the generalization of the method to the selective harvesting of new cultivars. Klein et al [37] presented a feasibility study for the development of a selective harvesting robot for cauliflower.…”

Automatic Detection Method of Mining subsidence basin based on InSAR and CNN-AFSA-SVM

“…After repeatedly trying the maximum number of times, if the advance conditions of the arti cial sh are still not met, move forward one step at random. The formula is (2) (2) Clustering behavior. The number n 0 of arti cial sh in the current eld of vision, the position X c of arti cial sh in the cluster center, and the food concentration Y c of arti cial sh in the center.…”

“…Subsequently, AlexNet, VGG19, and ResNet50 convolutional neural networks (CNNs) were used to extract feature vectors of mining subsidence basins for the SVM classi er, and mining subsidence basins were detected in a large-area InSAR interferogram; The arti cial sh swarm algorithm with strong optimization ability and good global convergence is introduced into SVM parameter optimization to construct an improved ResNet50_ SVM model. The experimental results show that (1) the three CNN_SVM methods can accurately detect dry mining subsidence basins automatically in large regional interference maps, providing an essential scienti c basis for the government to monitor illegal mining activities and prevent and control geological disasters in mining areas; (2) the accuracy of the CNN_SVM automatic detection methods for mining subsidence basins is approximately 80%, and that of ResNet50_SVM for mining subsidence basin detection is 83.7%, superior to that of AlexNet_SVM and VGG19_SVM, The accuracy of the improved ResNet50_SVM based on AFSA algorithm is 88.3%, which is better than the unimproved Resnet50_SVM model.…”

“…VGGNet explores the relationship between the depth of a CNN and its performance, and by iteratively stacking small convolutional kernels of 3 × 3 and maximum pooling layers of 2 × 2; VGGNet successfully constructs a CNN with 16 to 19 layers of depth (Blok et al 2021). The network structure is shown in Figure 2.…”