Semi-Self-Supervised Learning for Semantic Segmentation in Images with Dense Patterns

Abstract

Deep learning has shown potential in domains with large-scale annotated datasets. However, manual annotation is expensive, time-consuming, and tedious. Pixel-level annotations are particularly costly for semantic segmentation in images with dense irregular patterns of object instances, such as in plant images. In this work, we propose a method for developing high-performing deep learning models for semantic segmentation of such images utilizing little manual annotation. As a use case, we focus on wheat head segmentation. We synthesize a computationally annotated dataset—using a few annotated images, a short unannotated video clip of a wheat field, and several video clips with no wheat—to train a customized U-Net model. Considering the distribution shift between the synthesized and real images, we apply three domain adaptation steps to gradually bridge the domain gap. Only using two annotated images, we achieved a Dice score of 0.89 on the internal test set. When further evaluated on a diverse external dataset collected from 18 different domains across five countries, this model achieved a Dice score of 0.73. To expose the model to images from different growth stages and environmental conditions, we incorporated two annotated images from each of the 18 domains to further fine-tune the model. This increased the Dice score to 0.91. The result highlights the utility of the proposed approach in the absence of large-annotated datasets. Although our use case is wheat head segmentation, the proposed approach can be extended to other segmentation tasks with similar characteristics of irregularly repeating patterns of object instances.

Semi-Self-Supervised Domain Adaptation: Developing Deep Learning Models with Limited Annotated Data for Wheat Head Segmentation

“…Figure 1 illustrates the three manually annotated images (I η , I ζ , and I τ ) used for synthesizing computationally annotated datasets. Utilizing the methodology presented in our previous work [7], we computationally synthesize three datasets: D η , a set of 8000 images derived from I η ; D η+ζ , a set of 16,000 images derived from I η and I ζ and D ζ+τ , a set of 4000 images derived from I ζ and I τ . Figure 2 illustrates examples of the synthesized images and their corresponding segmentation masks.…”

“…We refer to the dataset resulting from combining D ρ 1 and D ρ 2 as D ρ . We also use two sets of Ψ and Γ as the internal and external test sets, as introduced by Najafian et al [7]. The set Ψ comprises 100 image frames, which were randomly selected from a video clip of a wheat field and have been manually annotated to serve as an internal test set.…”

“…M s j is the ground truth mask for the synthetic image I s j , and Ms j is the model prediction for I s j . We also generate noise-augmented images by applying Gaussian noise to the real image I j over t (1 ≤ t ≤ T) timesteps, as detailed in Equation (7). These noise-augmented images are subsequently routed through the encoder, followed by the image decoder, aiming to reconstruct the original images.…”

“…During the evaluation process, we assessed the model performance using the Dice score and IoU. As a baseline for comparison, we used a model developed in [7].…”

“…Deep learning (DL) methodologies can offer efficient, automated, and data-driven decision-making in agricultural practices. DL has demonstrated significant advancements in visual data analysis across various tasks, including image classification [2,3], object detection [4,5], semantic segmentation [6][7][8], and instance segmentation [9][10][11]. Automated visual monitoring of agricultural fields can aid in the early identification of issues such as pest infestations, diseases, and nutrient deficiencies while optimizing resource utilization.…”