Robust Species Distribution Mapping of Crop Mixtures Using Color Images and Convolutional Neural Networks

Abstract

Crop mixtures are often beneficial in crop rotations to enhance resource utilization and yield stability. While targeted management, dependent on the local species composition, has the potential to increase the crop value, it comes at a higher expense in terms of field surveys. As fine-grained species distribution mapping of within-field variation is typically unfeasible, the potential of targeted management remains an open research area. In this work, we propose a new method for determining the biomass species composition from high resolution color images using a DeepLabv3+ based convolutional neural network. Data collection has been performed at four separate experimental plot trial sites over three growing seasons. The method is thoroughly evaluated by predicting the biomass composition of different grass clover mixtures using only an image of the canopy. With a relative biomass clover content prediction of R2 = 0.91, we present new state-of-the-art results across the largely varying sites. Combining the algorithm with an all terrain vehicle (ATV)-mounted image acquisition system, we demonstrate a feasible method for robust coverage and species distribution mapping of 225 ha of mixed crops at a median capacity of 17 ha per hour at 173 images per hectare.

Towards the fully automated monitoring of ecological communities

“…(1) open-set classification can allow specimens, even those that are not present in training data, to be classified to the lowest possible taxonomic level (Lee et al, 2018); (2) synthetic image datasets can be generated using images of specimens with validated species-level identification (Skovsen et al, 2020) and (3) combination of data from multiple sources or sensors for specieslevel labelling and classification-for example, images may be complemented by DNA sequence data (Badirli et al, 2021).…”

“…Specifically, even with an exceptionally large training dataset, the system will encounter unfamiliar species, including some that are inseparable within high‐resolution imagery. Three emerging approaches will help this challenge to be overcome: (1) open‐set classification can allow specimens, even those that are not present in training data, to be classified to the lowest possible taxonomic level (Lee et al, 2018 ); (2) synthetic image datasets can be generated using images of specimens with validated species‐level identification (Skovsen et al, 2020 ) and (3) combination of data from multiple sources or sensors for species‐level labelling and classification—for example, images may be complemented by DNA sequence data (Badirli et al, 2021 ).…”

Enhancing Aboveground Biomass Prediction through Integration of the SCDR Paradigm into the U-Like Hierarchical Residual Fusion Model

“…In addition to directly predicting biomass values, there are studies that use image data to predict the relative content of a specific plant or crop, such as the percentage occupied, among other metrics [24][25][26]. Most of these studies analyze different grass contents in grasslands, for instance, Skovsen [27] and colleagues utilized convolutional neural networks to determine the biomass species composition of mixed crops from high-resolution color images. Data collection was conducted at four experimental sites over three growing seasons, and the method excelled in predicting the relative biomass content of clover (R 2 = 0.91).…”

High-throughput field crop phenotyping: current status and challenges

“…Huang et al (2018) demonstrated that rice and weeds can be classified based on UAS RGB images using a CNN model, fully convolutional network (FCN) and transfer learning (Jiang and Li 2020). While the current methods are not applicable to complex fields where weeds of various species are intermingled, Skovsen et al (2021) demonstrated that CNN models can classify white clover, red clover and weed from rather complicated canopy images, using synthetic training data which is discussed in the later…”