SparseTT: Visual Tracking with Sparse Transformers

Fu, Zhaohui; Fu, Zehua; Liu, Qingjie; Cai, Wenrui; Wang, Yunlong

doi:10.48550/arxiv.2205.03776

Cited by 9 publications

(23 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the model architecture, feature extraction, and feature integration techniques, recent deep trackers can be classified as three categories: CNN-based trackers [29,88,89,90,31,91,32,92,93,34,33,94], CNN-Transformer based trackers [46,47,48,49,50,51,52,53,54,55,56,57] and fully-Transformer based trackers [58,59,60,61,62,63,64]. CNN-based trackers rely solely on a CNN architecture for feature extraction and target detection, while CNN-Transformer based trackers and fully-Transformer based trackers partially and fully rely on a Transformer architecture, respectively.…”

Section: Transformer In Single Object Trackingmentioning

confidence: 99%

“…TrDiMP [46], DTT [50], HiFT [51], TransT [47], STARK [48], ToMP [52], CSWinTT [56], AiATrack [57], SiamTPN [53], UTT [54], TrTr [49], CTT [55] Fully-Transformer based Trackers Two-stream Two-stage Trackers DualTFR [58], Swin-Track [59], SparseTT [60] One-stream One-Stage Trackers MixFormer [61], SimTrack [62], OSTrack [63], ProContEXT [64] features are extracted using the ResNet50 [97] model. Then these features are reshaped using a 1 × 1 convolutional layer and fed to the feature fusion network.…”

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

“…Based on the tracking accuracy, SwinTrack out- While the self-attention mechanism of the Transformer captures long-range dependencies, it may not adequately capture target-specific cues, making it susceptible to distraction by background clutter. Fu et al [60] noticed this issue and then proposed a tracker, referred as SparseTT, by using a spare attention module to provide the target-specific attention in the search region features. In this tracker, the Swin Transformer is used to extract the target template and search region features and then fed them to a sparse Transformer.…”

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

“…17. The fully-Transformer based trackers: Mix-Former [61], SparseTT [60], and ProContEXT [64] showed better performance than the CNN-based trackers in many challenging attributes. In particular, almost all of the fully Transformer-based trackers successfully handled the fast motion (FM) and out-of-view (OV) scenarios and outperformed the CNN-based trackers by a large margin, due to their long-range feature capturing capability.…”

Section: Analysis On Otb100 Datasetmentioning

confidence: 99%

“…For the last two years, due to the great success of Transformers in other computer vision tasks, a few transformer based VOT approaches [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64] have been proposed and showed excellent performance on publicly available single object tracking benchmark datasets. They have been proposed as discriminative or Siamese-based trackers and showed outstanding performances than the CNN-based trackers due to their long range feature capturing techniques and high discriminative capabilities.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Transformers in Single Object Tracking: An Experimental Survey

Thangavel¹,

Thanikasalam²,

Ramanan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Single object tracking is a well-known and challenging research topic in computer vision. Over the last two decades, numerous researchers have proposed various algorithms to solve this problem and achieved promising results. Recently, Transformer-based tracking approaches have ushered in a new era in single object tracking due to their superior tracking robustness. Although several survey studies have been conducted to analyze the performance of trackers, there is a need for another survey study after the introduction of Transformers in single object tracking. In this survey, we aim to analyze the literature and performances of Transformer tracking approaches. Therefore, we conduct an in-depth literature analysis of Transformer tracking approaches and evaluate their tracking robustness and computational efficiency on challenging benchmark datasets. In addition, we have measured their performances on different tracking scenarios to find their strength and weaknesses. Our survey provides insights into the underlying principles of Transformer tracking approaches, the challenges they face, and their future directions.

show abstract

Section: Transformer In Single Object Trackingmentioning

confidence: 99%

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

Section: Transformers In Vot Cnn-transformer Based Trackersmentioning

confidence: 99%

Section: Analysis On Otb100 Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Transformers in Single Object Tracking: An Experimental Survey

Thangavel¹,

Thanikasalam²,

Ramanan³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Dual Feature Fusion Tracking With Combined Cross-Correlation and Transformer

Che,

Fu,

Shi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Siamese networks have found applications in various fields, notably object tracking, due to their remarkable speed and accuracy. Siamese tracking networks rely on cross-correlation to obtain the similarity score between the target template and the search region. However, since cross-correlation is a local matching operation, it cannot effectively capture the global context information. While the Transformer for feature fusion can better capture long-range dependencies and obtain more semantic information, more localized edge information is needed to distinguish the target from the background. Cross-correlation fusion and Transformer fusion have their advantages. They can complement each other, so we combine them and propose a dual feature fusion tracker (SiamCT) to obtain the local correlations and global dependencies between the target and the search region. Specifically, we construct two parallel feature fusion paths based on cross-correlation and Transformer. Among them, for cross-correlation fusion, we adopt the more efficient two-dimension pixel-wise cross-correlation (TDPC), which performs correlation operations from both spatial and channel dimensions, and the interaction of multidimensional information helps to realize more accurate feature fusion. Subsequently, the fused features are augmented by coordinate attention (CA) for orientation-dependent positional information. For Transformer fusion, we introduce cos-based linear attention(ClA) to improve Transformer's ability to acquire global context information. Our SiamCT outperforms existing leading methods in GOT-10k, LaSOT, TrackingNet, and OTB100 benchmarks based on extensive experiments.

show abstract

Discriminative correlation filters for visual object tracking with adaptive channel-based sample weights

孙,

junqi,

professor

et al. 2024

Speckle 2023: VIII International Conference on Speckle Metrology

View full text Add to dashboard Cite

In order to improve the accuracy and robustness of correlation filtering algorithm in more challenging scenarios, a target tracking algorithm based on adaptive channel sample weights is proposed in this paper. Firstly, channel attention modules are added to sample branch and search branch respectively to weight the features of each branch respectively, which can effectively improve the feature expression ability. Secondly, according to the channel weights of the two branches, we learn to fuse the weight information, interact the features of the two branches, and suppress the background information. Finally, the sample learning weights are reassigned according to the channel weights of the history frame to make more efficient use of the background information. In this paper, the OTB100 data set is used to verify the effectiveness of the improved algorithm. The area under the curve is increased by 1.6%, which proves the effectiveness of the improved algorithm.

show abstract

SparseTT: Visual Tracking with Sparse Transformers

Cited by 9 publications

References 0 publications

Transformers in Single Object Tracking: An Experimental Survey

Transformers in Single Object Tracking: An Experimental Survey

Dual Feature Fusion Tracking With Combined Cross-Correlation and Transformer

Discriminative correlation filters for visual object tracking with adaptive channel-based sample weights

Contact Info

Product

Resources

About