Attention-Guided Disentangled Feature Aggregation for Video Object Detection

Abstract

Object detection is a computer vision task that involves localisation and classification of objects in an image. Video data implicitly introduces several challenges, such as blur, occlusion and defocus, making video object detection more challenging in comparison to still image object detection, which is performed on individual and independent images. This paper tackles these challenges by proposing an attention-heavy framework for video object detection that aggregates the disentangled features extracted from individual frames. The proposed framework is a two-stage object detector based on the Faster R-CNN architecture. The disentanglement head integrates scale, spatial and task-aware attention and applies it to the features extracted by the backbone network across all the frames. Subsequently, the aggregation head incorporates temporal attention and improves detection in the target frame by aggregating the features of the support frames. These include the features extracted from the disentanglement network along with the temporal features. We evaluate the proposed framework using the ImageNet VID dataset and achieve a mean Average Precision (mAP) of 49.8 and 52.5 using the backbones of ResNet-50 and ResNet-101, respectively. The improvement in performance over the individual baseline methods validates the efficacy of the proposed approach.

Disentangled Dynamic Deviation Transformer Networks for Multivariate Time Series Anomaly Detection

“…Disentangled representation learning is a well-studied topic that tries to learn the representation of various independent components hidden behind the data. It has been applied to computer vision [ 26 ], recommendation [ 27 ], natural language processing [ 28 ], and other domains. Hamaguchi et al [ 29 ], for example, used the disentanglement technique to detect rare events and proposed a new method for learning disentangled representations from low-cost negative samples in a pair of observations disentangles as variant and invariant factors, respectively, representing mixed information related to trivial events and image content invariant to trivial events.…”

“…In contrast, an effective multiscale scheme enables the model to remain invariant as the scale changes and capture more intrinsic patterns [ 39 ]. The paper [ 40 ] uses k -order polynomials of adjacency matrices to feature aggregate multiscale structural information and learn rich representations by establishing relationships between distant and near neighbors in this way, and similarly in [ 26 , 27 , 29 ].…”

LPO-YOLOv5s: A Lightweight Pouring Robot Object Detection Algorithm

“…Among them, the detection effect of the decoupling head is higher than that of the coupling head in most cases. Muralidhara et al [ 27 ] increased the upper limit of current video object detection methods by introducing DyHead detection heads into faster R-CNN, combining scale, space, and task aware attention, and achieving good results.…”

Salient Target Detection Method of Video Images Based on Convolution Neural Network

“…Video image salient target detection is to simulate human visual perception system, intelligently detect salient targets in video images from semantic level, and finally realize independent analysis and understanding of video image content [5][6][7][8][9][10][11]. Traditional target detection of video images is often used to distinguish the relevant classification of large categories of targets, in the case of complex and diverse image content, it can not capture enough visual cues, which makes it difficult to distinguish small differences between categories [12][13][14][15][16][17][18][19][20][21][22]. To solve this problem, it's impossible to rely on all kinds of artificial image annotation to prompt which areas the detection model needs to extract which target feature information.…”