SlypNet: Spikelet-based yield prediction of wheat using advanced plant phenotyping and computer vision techniques

Abstract

The application of computer vision in agriculture has already contributed immensely to restructuring the existing field practices starting from the sowing to the harvesting. Among the different plant parts, the economic part, the yield, has the highest importance and becomes the ultimate goal for the farming community. It depends on many genetic and environmental factors, so this curiosity about knowing the yield brought several precise pre-harvest prediction methods using different ways. Out of those techniques, non-invasive yield prediction techniques using computer vision have been proved to be the most efficient and trusted platform. This study developed a novel methodology, called SlypNet, using advanced deep learning networks, i.e., Mask R-CNN and U-Net, which can extract various plant morphological features like spike and spikelet from the visual image of the wheat plant and provide a high-throughput yield estimate with great precision. Mask R-CNN outperformed previous networks in spike detection by its precise detection performance with a mean average precision (mAP) of 97.57%, a F1 score of 0.67, and an MCC of 0.91 by overcoming several natural field constraints like overlapping and background interference, variable resolution, and high bushiness of plants. The spikelet detection module’s accuracy and consistency were tested with about 99% validation accuracy of the model and the least error, i.e., a mean square error of 1.3 from a set of typical and complex views of wheat spikes. Spikelet yield cumulatively showed the probable production capability of each plant. Our method presents an integrated deep learning platform of spikelet-based yield prediction comprising spike and spikelet detection, leading to higher precision over the existing methods.

A Comprehensive Review on Deep Learning Assisted Computer Vision Techniques for Smart Greenhouse Agriculture

“…However, challenges persist due to the complex lighting conditions in PFAL environment and similarities between green fruit and their surrounding vegetation. In another study, Maji et al [114] investigated wheat yield estimation through SlypNet, a hybrid deep learning approach that combines Mask R-CNN and U-Net. This approach effectively captures wheat morphological features, attaining a high mAP of 97.57% in spike detection.…”

“…Centralized repositories like PlantVillage offer a valuable start but have limited coverage and annotation complexity. Constructing large-scale greenhouse image datasets with standard formatting and annotations will be critical to benchmark performance of computer vision techniques and fuel advancements ( [93], [113], [114]).…”

Advancements in Utilizing Image-Analysis Technology for Crop-Yield Estimation

“…In terms of wheat-yield calculation, Maji et al [110] developed a second-order deep learning framework called SlypNet for wheat-ear detection, which combined Mask R-CNN and U-Net algorithms to automatically extract rich morphological features from images. It could effectively overcome interference such as leaf overlap and occlusion in peak detection, and the accuracy of the small-ear-detection model validation reached 99%.…”

Segmentation and counting of wheat spike grains based on deep learning and textural feature

“…The results obtained for the same wheat varieties using the same method under different nitrogen fertilizer treatments do not differ significantly, indicating that different nitrogen fertilizer treatments have a small effect on the counting results. Therefore, by applying this method to different varieties of wheat spikes, the accuracy of counting the number of spike grains can be greatly improved, and the automatic counting of spike grains of a single wheat plant with higher accuracy can be achieved [ 34 ].

Fig.

…”