Transformer Neural Network for Weed and Crop Classification of High Resolution UAV Images

Abstract

Monitoring crops and weeds is a major challenge in agriculture and food production today. Weeds compete directly with crops for moisture, nutrients, and sunlight. They therefore have a significant negative impact on crop yield if not sufficiently controlled. Weed detection and mapping is an essential step in weed control. Many existing research studies recognize the importance of remote sensing systems and machine learning algorithms in weed management. Deep learning approaches have shown good performance in many agriculture-related remote sensing tasks, such as plant classification, disease detection, etc. However, despite the success of these approaches, they still face many challenges such as high computation cost, the need of large labelled datasets, intra-class discrimination (in growing phase weeds and crops share many attributes similarity as color, texture, and shape), etc. This paper aims to show that the attention-based deep network is a promising approach to address the forementioned problems, in the context of weeds and crops recognition with drone system. The specific objective of this study was to investigate visual transformers (ViT) and apply them to plant classification in Unmanned Aerial Vehicles (UAV) images. Data were collected using a high-resolution camera mounted on a UAV, which was deployed in beet, parsley and spinach fields. The acquired data were augmented to build larger dataset, since ViT requires large sample sets for better performance, we also adopted the transfer learning strategy. Experiments were set out to assess the effect of training and validation dataset size, as well as the effect of increasing the test set while reducing the training set. The results show that with a small labeled training dataset, the ViT models outperform state-of-the-art models such as EfficientNet and ResNet. The results of this study are promising and show the potential of ViT to be applied to a wide range of remote sensing image analysis tasks.

Landslide Detection Based on ResU-Net with Transformer and CBAM Embedded: Two Examples with Geologically Different Environments

“…In the transformer, assuming that there is a processed sequence different from the CNN that generates the attention map through convolution and pooling [63], the goal of self-attention is to capture the interaction amongst all n entities by encoding each entity in terms of the global contextual information [47,52]. The implementation of the self-attention can be briefly described as a process of mapping queries (Q), keys (K), and values (V) to outputs [46], where queries, keys, and values are generated by projecting the input sequence 1).…”

“…The detailed calculation equation of the attention score can be expressed as follows: Compared to CNN, self-attention increases the receptive field without increasing the computational cost associated with kernel sizes. In addition, self-attention is invariant to permutations and changes in the number of input points, hence it can easily operate on irregular inputs instead of the standard convolution that requires a grid structure [52,64]. In transformers, self-attention is extended to multi-head self-attention by computing h selfattention operations in parallel, in which h is called the head.…”

“…First, the sequence normalized through the layer normalization is inputted into the MSA to calculate the importance weights among all tokens, and the original information is preserved by using the residual structure. Among them, MSA extracts feature information from different subspaces and then concatenates the result from h self-attention head, which enables the model to selectively assign more focus to certain positions of the image, and also helps the model to learn features faster and reduce the training cost [49,52]. Then, the obtained results are sequentially passed through an LN and an MLP, and the same residual structure is introduced to generate the final semantic information extracted by this layer.…”

“…For example, Bazi et al [51] verified that ViT achieved high classification accuracy on four datasets. Reedha et al [52] introduced ViT into vegetation classification with a small dataset, achieving higher F1-score compared to EfficienNet and ResNet. Recently, research by Park and Kim [53] showed that MSA will flatten the loss value, guiding the model to converge along a smooth trajectory.…”

A Systematic Review of Current Trends in Artificial Intelligence for Smart Farming to Enhance Crop Yield

“…Reedha et al [86] explained that controlling weeds and food crops is the main crop production and agriculture. Because basically, weeds take the same nutrients as plants.…”