Wheat ear counting using K-means clustering segmentation and convolutional neural network

Abstract

Background: Wheat yield is influenced by the number of ears per unit area, and manual counting has traditionally been used to estimate wheat yield. To realize rapid and accurate wheat ear counting, K-means clustering was used for the automatic segmentation of wheat ear images captured by hand-held devices. The segmented data set was constructed by creating four categories of image labels: non-wheat ear, one wheat ear, two wheat ears, and three wheat ears, which was then was sent into the convolution neural network (CNN) model for training and testing to reduce the complexity of the model. Results: The recognition accuracy of non-wheat, one wheat, two wheat ears, and three wheat ears were 99.8, 97.5, 98.07, and 98.5%, respectively. The model R 2 reached 0.96, the root mean square error (RMSE) was 10.84 ears, the macro F1-score and micro F1-score both achieved 98.47%, and the best performance was observed during late grainfilling stage (R 2 = 0.99, RMSE = 3.24 ears). The model could also be applied to the UAV platform (R 2 = 0.97, RMSE = 9.47 ears). Conclusions: The classification of segmented images as opposed to target recognition not only reduces the workload of manual annotation but also improves significantly the efficiency and accuracy of wheat ear counting, thus meeting the requirements of wheat yield estimation in the field environment.

Wheat Ear Recognition Based on RetinaNet and Transfer Learning

“…During the shooting process, the wheat image is easily affected by the changes in natural light, growth environment, shaking of the shooting equipment, and the unstable focus of the lens. Meanwhile, the obtained image may contain some noise caused by random signals in the process of transmission [ 34 ]. Therefore, the method of data denoising is exploited to remove the noise points in the obtained image and reduce the influence of noise on the recognition results.…”

“…Firstly, the median filtering method with a kernel of 5 is used to remove the noise in the wheat image. The specific denoising process exploits the Python language to call the medianBlur function provided by the OpenCV library, and the parameter ksize is set to 5 [ 34 ].…”

“…Although this work obtained approximately 3000 images, there were not enough to train and validate the model, emphasizing the need for data enhancement. In order to address this issue, programs have been written in Python language to shrink, enlarge, and flip the original image [ 34 ]. In order to simulate the change of light, the HSV (hue, saturation, and value) [ 46 ], color space, and various conversions are exploited, such as linear change of hue, linear change of saturation, and linear change of brightness.…”

“…The current application of drones combined with deep learning technology has greatly promoted the development of precision agriculture. In recent years, some meaningful research [ 7 , 8 , 9 , 15 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ] has emerged. These studies have used RGB (red, green, blue), multispectral, hyperspectral, and thermal infrared data acquired by UAV and CNN to evaluate the phenotypic characteristics of citrus crops [ 38 ], obtain key points of plants/plant leaves [ 39 ], plant stress analysis and plant disease identification [ 40 , 41 ].…”

Occlusion Robust Wheat Ear Counting Algorithm Based on Deep Learning

“…After the image is annotated, the fully convolution network such as Unet (Ronneberger et al, 2015), FCN (Long et al, 2015), etc. is usually trained in way of encoder-decoder (Grbovic et al, 2019;Sadeghi-Tehran et al, 2019;Misra et al, 2020;Xu X. et al, 2020). The trained full convolutional network can segment each wheat ear in the input images and output it in the form of a mask.…”