Object detection using convolutional neural networks and transformer-based models: a review

Abstract

Transformer models are evolving rapidly in standard natural language processing tasks; however, their application is drastically proliferating in computer vision (CV) as well. Transformers are either replacing convolution networks or being used in conjunction with them. This paper aims to differentiate the design of convolutional neural networks (CNNs) built models and models based on transformer, particularly in the domain of object detection. CNNs are designed to capture local spatial patterns through convolutional layers, which is well suited for tasks that involve understanding visual hierarchies and features. However, transformers bring a new paradigm to CV by leveraging self-attention mechanisms, which allows to capture both local and global context in images. Here, we target the various aspects such as basic level of understanding, comparative study, application of attention model, and highlighting tremendous growth along with delivering efficiency are presented effectively for object detection task. The main emphasis of this work is to offer basic understanding of architectures for object detection task and motivates to adopt the same in computer vision tasks. In addition, this paper highlights the evolution of transformer-based models in object detection and their growing importance in the field of computer vision, we also identified the open research direction in the same field.

Identifying rice field weeds from unmanned aerial vehicle remote sensing imagery using deep learning

“…In order to comprehensively evaluate the performance of the model on the rice weed detection task, we conducted comparative experiments on the rice weed dataset, comparing the RMS-DETR model with other classic DETR variants, including Deformable DETR [ 14 ], Anchor DETR [ 40 ], and DAB-DETR [ 41 ]. The experimental results are presented in Table 7 and Fig.…”

“…In recent years, extensive research has been conducted on CNN-based object detectors. These detectors are primarily categorized into two-stage networks and one-stage networks, with the representative models being the R-CNN series and the YOLO series [ 13 , 14 ]. Zhang et al [ 15 ] embedded the CBAM attention mechanism after the pooling layers in the latter part of VGG19, forming the VGG19-CBAM structure as the optimal backbone feature extraction network for the Faster R-CNN model.…”

“…The Transformer-based object detector utilizes the self-attention mechanism to generate contextual representations of input sequences, enabling it to effectively capture long-range dependencies within the input sequences. This is particularly crucial for tasks involving complex structures and long sequences [ 14 ]. In recent years, Transformer-based detectors have made significant progress in performance, thanks to researchers' relentless efforts in accelerating training convergence and reducing optimization challenges [ 17 ].…”

“…In recent years, Transformer-based detectors have made significant progress in performance, thanks to researchers' relentless efforts in accelerating training convergence and reducing optimization challenges [ 17 ]. Zhu et al [ 14 ] pointed out that when Transformer components are initialized, attention modules apply almost identical attention weights to all pixels in the feature map, leading to a longer training time to converge. To address this issue, they proposed a deformable attention module, combining the advantages of deformable convolution sparse spatial sampling and the relationship modeling capability of transformers, to overcome the slow convergence problem in DETR models.…”

Improving Object Detection Accuracy with Self-Training Based on Bi-Directional Pseudo Label Recovery

“…In most machine learning and convolutional neural network (CNN) based detectors [7,8], the success of detection relies on collecting and labeling a large amount of training and testing data in a variety of road environments. If self-driving vehicles are limited to traveling within a specific geographic region, and their training data collection is limited in that region, such detectors may provide good detection accuracy when operated in the same region or in similar environments.…”