Learning Tracking Representations via Dual-Branch Fully Transformer Networks

Xie, Fei; Wang, Chunyu; Wang, Guangting; Yang, Wankou; Zeng, Wenjun

doi:10.1109/iccvw54120.2021.00303

Cited by 49 publications

(27 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…STARK [40] employs the original transformer structure by concatenating the template and search region features. Du-alTFR and SwinTrack [38,22] use transformer as the backbone network. In this work, we develop an efficient feature fusion network based on transformer.…”

Section: Transformer In Trackingmentioning

confidence: 99%

Efficient Visual Tracking via Hierarchical Cross-Attention Transformer

Chen¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, target tracking has made great progress in accuracy. This development is mainly attributed to powerful networks (such as transformers) and additional modules (such as online update and refinement modules). However, less attention has been paid to tracking speed. Most state-of-the-art trackers are satisfied with the real-time speed on powerful GPUs. However, practical applications necessitate higher requirements for tracking speed, especially when edge platforms with limited resources are used. In this work, we present an efficient tracking method via a hierarchical cross-attention transformer named HCAT. Our model runs about 195 f ps on GPU, 45 f ps on CPU, and 55 f ps on the edge AI platform of NVidia Jetson AGX Xavier. Experiments show that our HCAT achieves promising results on LaSOT, GOT-10k, TrackingNet, NFS, OTB100, UAV123, and VOT2020. Code and models are available at https://github.com/chenxin-dlut/HCAT.

show abstract

Section: Transformer In Trackingmentioning

confidence: 99%

Efficient Visual Tracking via Hierarchical Cross-Attention Transformer

Chen¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To improve this, our discriminative target-dependent features can greatly lighten the burden for the online DCF. Recent rising Transformer-based methods [5,42,48,52,55] exploit the long-range modelling of Transformer to effectively fuse the features. Thus, they can track robustly without online learning.…”

Section: Related Workmentioning

confidence: 99%

“…The training datasets include the train subsets of LaSOT [13], GOT-10K [19], COCO2017 [26], and TrackingNet [30]. Other settings are the same with [5,48]. Details are in supplement.…”

Section: Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

Correlation-Aware Deep Tracking

Xie¹,

Wang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Robustness and discrimination power are two fundamental requirements in visual object tracking. In most tracking paradigms, we find that the features extracted by the popular Siamese-like networks cannot fully discriminatively model the tracked targets and distractor objects, hindering them from simultaneously meeting these two requirements. While most methods focus on designing robust correlation operations, we propose a novel target-dependent feature network inspired by the self-/cross-attention scheme. In contrast to the Siamese-like feature extraction, our network deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it is able to suppress non-target features, resulting in instancevarying feature extraction. The output features of the search image can be directly used for predicting target locations without extra correlation step. Moreover, our model can be flexibly pre-trained on abundant unpaired images, leading to notably faster convergence than the existing methods. Extensive experiments show our method achieves the stateof-the-art results while running at real-time. Our feature networks also can be applied to existing tracking pipelines seamlessly to raise the tracking performance. Code will be available.

show abstract

“…Efficient Transformers. Though ViT has achieved great success in vision tasks [1,5,11,14,17,26,47,49], it needs huge computational resources to achieve comparable performance to ResNet [16] with a similar model size trained on ImageNet-1K. Therefore, how to build a more efficient Transformer draws researchers' interest.…”

Section: Related Workmentioning

confidence: 99%

Multi-Tailed Vision Transformer for Efficient Inference

Wang¹,

Du²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

Recently, Vision Transformer (ViT) has achieved promising performance in image recognition and gradually serves as a powerful backbone in various vision tasks. To satisfy the sequential input of Transformer, the tail of ViT first splits each image into a sequence of visual tokens with a fixed length. Then the following self-attention layers constructs the global relationship between tokens to produce useful representation for the downstream tasks. Empirically, representing the image with more tokens leads to better performance, yet the quadratic computational complexity of selfattention layer to the number of tokens could seriously influence the efficiency of ViT's inference. For computational reduction, a few pruning methods progressively prune uninformative tokens in the Transformer encoder, while leaving the number of tokens before the Transformer untouched. In fact, fewer tokens as the input for the Transformer encoder can directly reduce the following computational cost. In this spirit, we propose a Multi-Tailed Vision Transformer (MT-ViT) in the paper. MT-ViT adopts multiple tails to produce visual sequences of different lengths for the following Transformer encoder. A tail predictor is introduced to decide which tail is the most efficient for the image to produce accurate prediction. Both modules are optimized in an end-to-end fashion, with the Gumbel-Softmax trick. Experiments on ImageNet-1K demonstrate that MT-ViT can achieve a significant reduction on FLOPs with no degradation of the accuracy and outperform other compared methods in both accuracy and FLOPs.

show abstract

Learning Tracking Representations via Dual-Branch Fully Transformer Networks

Cited by 49 publications

References 39 publications

Efficient Visual Tracking via Hierarchical Cross-Attention Transformer

Efficient Visual Tracking via Hierarchical Cross-Attention Transformer

Correlation-Aware Deep Tracking

Multi-Tailed Vision Transformer for Efficient Inference

Contact Info

Product

Resources

About