A Weakly Supervised Deep Learning Framework for Sorghum Head Detection and Counting

Abstract

<jats:p>The yield of cereal crops such as sorghum (<jats:italic>Sorghum bicolor</jats:italic> L. Moench) depends on the distribution of crop-heads in varying branching arrangements. Therefore, counting the head number per unit area is critical for plant breeders to correlate with the genotypic variation in a specific breeding field. However, measuring such phenotypic traits manually is an extremely labor-intensive process and suffers from low efficiency and human errors. Moreover, the process is almost infeasible for large-scale breeding plantations or experiments. Machine learning-based approaches like deep convolutional neural network (CNN) based object detectors are promising tools for efficient object detection and counting. However, a significant limitation of such deep learning-based approaches is that they typically require a massive amount of hand-labeled images for training, which is still a tedious process. Here, we propose an active learning inspired weakly supervised deep learning framework for sorghum head detection and counting from UAV-based images. We demonstrate that it is possible to significantly reduce human labeling effort without compromising final model performance (<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1"><mml:mrow><mml:msup><mml:mrow><mml:mi>R</mml:mi></mml:mrow><mml:mrow><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:math> between human count and machine count is 0.88) by using a semitrained CNN model (i.e., trained with limited labeled data) to perform synthetic annotation. In addition, we also visualize key features that the network learns. This improves trustworthiness by enabling users to better understand and trust the decisions that the trained deep learning model makes.</jats:p>

A Deep Learning Semantic Segmentation-Based Approach for Field-Level Sorghum Panicle Counting

“…With the development of deep learning models for point clouds [57,58], compared performance should be achieved from point clouds. Thought not directly comparable, due to the use of different accuracy metrics, the performance achieved here is generally in line to other previous studies that applied image data [34,37,38]. In Olsen et al [34] a 98% counting accuracy and median absolute error (MAE) between 1.88 and 2.66 were achieved for 18 sorghum varieties in the Midwestern United States.…”

“…As highlighted before, this high accuracy came at the expense of significant feature engineering, which we circumvented in this study by applying a deep learning approach. Ghosal et al [38] used 1269 hand-labeled images and achieved an AUC accuracy of 94% by applying an active learning inspired weakly supervised deep learning framework. Similar counting studies on other crops have also reported good performance.…”

“…A similar approach based on learning multiple features for panicle detection is presented in [37]. To reduce the effort spent on manual labeling, Ghosal et al [38] applied a weakly supervised deep learning framework whereby a model was trained on a small training sample but gradually updated through human feedback on object detection results. They demonstrated a significant reduction in human labeling effort without compromising final model performance.…”

Active learning with point supervision for cost-effective panicle detection in cereal crops

“…State-of-the-art convolutional neural networks have been shown to perform well on a wide variety of phenotyping tasks. The applications of CNNs in plant phenotyping include image classification tasks such as plant species identification [1], stress identification [2], object detection arXiv:1910.01789v2 [cs.CV] 15 Oct 2019 and counting tasks such as panicle or spike detection [3,4,5,6], leaf counting [7], fruit detection [8]; as well as pixel-wise segmentation based tasks such as panicle segmentation [9,10] and crop-weed segmentation [11]. We refer the reader to [12] and [13] for a full treatment of deep learning in agriculture and plant phenotyping tasks.…”

Learning from Synthetic Dataset for Crop Seed Instance Segmentation

“…image flipping and rotating) have been the most commonly adopted techniques to compensate for the lack of data. More recently, several reports have highlighted the challenges with incorporating active learning or other approaches that loops the annotation and model training to minimize the labor cost 18–20 .…”