STMTrack: Template-free Visual Tracking with Space-time Memory Networks

Fu, Zhihong; Liu, Qingjie; Fu, Zehua; Wang, Yunlong

doi:10.1109/cvpr46437.2021.01356

Cited by 254 publications

(98 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this subsection, we compare our TransT and TransT-M methods with five state-of-the-art trackers published in 2021 (ARDiMPsuper [21]), TrDiMP [27], TrSiam [27], STMTrack [46], and SiamGAT [47]), twelve state-of-the-art trackers published in 2020 (SiamR-CNN [48], Ocean [8], KYS [49], DCFST [50], SiamFC++ [7], PrDiMP [14], CGACD [11], SiamAttn [10], MAML [51], D3S [20], SiamCAR [52], and SiamBAN [53]) and six representative trackers presented before (DiMP [12], SiamPRN++ [6], ATOM [5], ECO [26], MDNet [54] and SiamFC [3]) 2 . We report the detailed comparison results on the large-scale LaSOT [43], TrackingNet [44], and GOT-10k [45] datasets in Table 1.…”

Section: Evaluation On Trackingnet Lasot and Got-10k Datasetsmentioning

confidence: 99%

High-Performance Transformer Tracking

Chen¹,

Yan²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Correlation has a critical role in the tracking field, especially in recent popular Siamese-based trackers. The correlation operation is a simple fusion manner to consider the similarity between the template and the search region. However, the correlation operation is a local linear matching process, losing semantic information and falling into local optimum easily, which may be the bottleneck of designing high-accuracy tracking algorithms. In this work, to determine whether a better feature fusion method exists than correlation, a novel attention-based feature fusion network, inspired by Transformer, is presented. This network effectively combines the template and the search region features using attention. Specifically, the proposed method includes an ego-context augment module based on self-attention and a cross-feature augment module based on cross-attention. First, we present a Transformer tracking (named TransT) method based on the Siamese-like feature extraction backbone, the designed attention-based fusion mechanism, and the classification and regression head. Based on the TransT baseline, we further design a segmentation branch to generate an accurate mask. Finally, we propose a stronger version of TransT by extending TransT with a multi-template design and an IoU prediction head, named TransT-M. Experiments show that our TransT and TransT-M methods achieve promising results on seven popular datasets. Code and models are available at https://github.com/chenxin-dlut/TransT-M.

show abstract

Section: Evaluation On Trackingnet Lasot and Got-10k Datasetsmentioning

confidence: 99%

High-Performance Transformer Tracking

Chen¹,

Yan²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…MemTrack [68] reads a residual template from memory and combines it with the initial template to update the representations of targets. STMTrack [16] guides the information retrieval with the current frame and adaptively obtains all useful information as it needs. However, these work focuses on single object tracking (SOT), and does not need to concern about the inter-object associa-…”

Section: Related Workmentioning

confidence: 99%

MeMOT: Multi-Object Tracking with Memory

Cai¹,

Xu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose an online tracking algorithm that performs the object detection and data association under a common framework, capable of linking objects after a long time span. This is realized by preserving a large spatio-temporal memory to store the identity embeddings of the tracked objects, and by adaptively referencing and aggregating useful information from the memory as needed. Our model, called MeMOT, consists of three main modules that are all Transformer-based: 1) Hypothesis Generation that produce object proposals in the current video frame; 2) Memory Encoding that extracts the core information from the memory for each tracked object; and 3) Memory Decoding that solves the object detection and data association tasks simultaneously for multi-object tracking. When evaluated on widely adopted MOT benchmark datasets, MeMOT observes very competitive performance.

show abstract

“…Siamese network is a kind of network that contains more than one identical weight-sharing subnetworks, which can naturally introduce inductive biases for invariance modeling (Bromley et al 1993). It has wide applications including video super-resolution (Zhang, Liu, and Xiong 2019), medical object detection (Chen et al 2020a), visual tracking (Fu et al 2021), etc. Inspired by its success in various visual tasks, researchers have successfully introduced siamese networks into the field of graph learning (Krivosheev et al 2021;Jin et al 2021).…”

Section: Related Workmentioning

confidence: 99%

Siamese Attribute-missing Graph Auto-encoder

Tu¹,

Zhou²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph representation learning (GRL) on attribute-missing graphs, which is a common yet challenging problem, has recently attracted considerable attention. We observe that existing literature: 1) isolates the learning of attribute and structure embedding thus fails to take full advantages of the two types of information; 2) imposes too strict distribution assumption on the latent space variables, leading to less discriminative feature representations. In this paper, based on the idea of introducing intimate information interaction between the two information sources, we propose our Siamese Attribute-missing Graph Auto-encoder (SAGA). Specifically, three strategies have been conducted. First, we entangle the attribute embedding and structure embedding by introducing a siamese network structure to share the parameters learned by both processes, which allows the network training to benefit from more abundant and diverse information. Second, we introduce a K-nearest neighbor (KNN) and structural constraint enhanced learning mechanism to improve the quality of latent features of the missing attributes by filtering unreliable connections. Third, we manually mask the connections on multiple adjacent matrices and force the structural information embedding sub-network to recover the true adjacent matrix, thus enforcing the resulting network to be able to selectively exploit more high-order discriminative features for data completion. Extensive experiments on six benchmark datasets demonstrate the superiority of our SAGA against the state-of-the-art methods.

show abstract

STMTrack: Template-free Visual Tracking with Space-time Memory Networks

Cited by 254 publications

References 44 publications

High-Performance Transformer Tracking

High-Performance Transformer Tracking

MeMOT: Multi-Object Tracking with Memory

Siamese Attribute-missing Graph Auto-encoder

Contact Info

Product

Resources

About