Optimized Deep Learning Model as a Basis for Fast UAV Mapping of Weed Species in Winter Wheat Crops

Abstract

Weed maps should be available quickly, reliably, and with high detail to be useful for site-specific management in crop protection and to promote more sustainable agriculture by reducing pesticide use. Here, the optimization of a deep residual convolutional neural network (ResNet-18) for the classification of weed and crop plants in UAV imagery is proposed. The target was to reach sufficient performance on an embedded system by maintaining the same features of the ResNet-18 model as a basis for fast UAV mapping. This would enable online recognition and subsequent mapping of weeds during UAV flying operation. Optimization was achieved mainly by avoiding redundant computations that arise when a classification model is applied on overlapping tiles in a larger input image. The model was trained and tested with imagery obtained from a UAV flight campaign at low altitude over a winter wheat field, and classification was performed on species level with the weed species Matricaria chamomilla L., Papaver rhoeas L., Veronica hederifolia L., and Viola arvensis ssp. arvensis observed in that field. The ResNet-18 model with the optimized image-level prediction pipeline reached a performance of 2.2 frames per second with an NVIDIA Jetson AGX Xavier on the full resolution UAV image, which would amount to about 1.78 ha h−1 area output for continuous field mapping. The overall accuracy for determining crop, soil, and weed species was 94%. There were some limitations in the detection of species unknown to the model. When shifting from 16-bit to 32-bit model precision, no improvement in classification accuracy was observed, but a strong decline in speed performance, especially when a higher number of filters was used in the ResNet-18 model. Future work should be directed towards the integration of the mapping process on UAV platforms, guiding UAVs autonomously for mapping purpose, and ensuring the transferability of the models to other crop fields.

How Can Unmanned Aerial Vehicles Be Used for Detecting Weeds in Agricultural Fields?

“…There were some limitations of detecting unknown weed species to the Optimized Deep Learning model in a previous study [55]. When the model precision was increased from 16 to 32 bits, there was no improvement in classification accuracy but a significant decrease in speed performance, especially when a larger number of filters was utilized in the ResNet-18 model.…”

“…One study [53] proposed that crop images could be taken precisely in the early season, so that specifically color-dependent segmentation can be applied to segment weed patches to achieve the higher accuracy of an algorithm. de Camargo et al [55] Sugarcane Developed a framework to identify the defect areas in the sugarcane farms.…”

Identifying and mapping individual medicinal plant Lamiophlomis rotata at high elevations by using unmanned aerial vehicles and deep learning

“…Deep learning (DL) and unmanned aerial vehicles (UAVs) are technical and methodological advancements that strongly contribute to plant identification shifts. Examples include crop yield prediction [ 9 , 10 ] and weed mapping [ 11 , 12 ] in precision agriculture, invasive species identification [ 13 , 14 ], species detection and vegetation classification [ 15 , 16 ] in ecological aspects, and area calculation in cultivated herbal medicine [ 17 , 18 ].…”

Weed Detection by Faster RCNN Model: An Enhanced Anchor Box Approach

“…A study investigated images of UAVs to recognize weeds using a ResNet-based model [20]. Research has been conducted on the identification of weeds using the ResNet model [21]. Ukaegbu et al [22] used a CNN model on a quadcopter to detect broadleaf and grassweeds and to evaluate herbicide spraying.…”