Multi‐template temporal information fusion for Siamese object tracking

Lu, Xiaofeng; Wang, Zhengyang; Wang, Xuan; Hei, Xinhong

doi:10.1049/cvi2.12128

Cited by 4 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tables 5 and 6 report the AUC scores and precision scores compared with SiamFC [2], SiamRPN++, MemTrack [52], CFNet [53], hierarchical feature transformer (HiFT) [54], SiamRPN [6], ADNet [55], SiamCAR, VITAL [56], MTFM [57], and DP‐Siam [58] on the UAV123 and UAV20L datasets. As can be seen from Table 5, our method ranks first in both precision score and AUC score.…”

Section: Methodsmentioning

confidence: 99%

SiamCCF: Siamese visual tracking via cross‐layer calibration fusion

Chen

Huang

Zhu

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

Siamese networks have attracted wide attention in visual tracking due to their competitive accuracy and speed. However, the existing Siamese trackers usually leverage a fixed linear aggregation of feature maps, which does not effectively fuse the different layers of features with attention. Besides, most of Siamese trackers calculate the similarity between the template and the search region through a cross‐correlation operation between the features of the last blocks from the two branches, which might introduce the redundant noise information. In order to solve these problems, this study proposes a novel Siamese visual tracking method via cross‐layer calibration fusion, termed SiamCCF. An attention‐based feature fusion module is employed using local attention and non‐local attention to fuse the features from the deep and shallow layers, so as to capture both local details and high‐level semantic information. Moreover, a cross‐layer calibration module can use the fused features to calibrate the features of the last network blocks and build the cross‐layer long‐range spatial and inter‐channel dependencies around each spatial location. Extensive experiments demonstrate that the proposed method has achieved competitive tracking performance compared with state‐of‐the‐art trackers on challenging benchmarks, including OTB100, OTB2013, UAV123, UAV20L, and LaSOT.

show abstract

Section: Methodsmentioning

confidence: 99%

SiamCCF: Siamese visual tracking via cross‐layer calibration fusion

Chen

Huang

Zhu

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…Visual object tracking is an important research direction in computer vision, and it is widely employed in the fields of autonomous driving, intelligent security, and robot motion [1,2]. In the tracking task, the initial target is first provided and then localised in subsequent frames of the video.…”

Section: Introductionmentioning

confidence: 99%

STFT: Spatial and temporal feature fusion for transformer tracker

Zhang,

Piao,

2023

IET Computer Vision

View full text Add to dashboard Cite

Siamese‐based trackers have demonstrated robust performance in object tracking, while Transformers have achieved widespread success in object detection. Currently, many researchers use a hybrid structure of convolutional neural networks and Transformers to design the backbone network of trackers, aiming to improve performance. However, this approach often underutilises the global feature extraction capability of Transformers. The authors propose a novel Transformer‐based tracker that fuses spatial and temporal features. The tracker consists of a multilayer spatial feature fusion network (MSFFN), a temporal feature fusion network (TFFN), and a prediction head. The MSFFN includes two phases: feature extraction and feature fusion, and both phases are constructed with a Transformer. Compared with the hybrid structure of “CNNs + Transformer,” the proposed method enhances the continuity of feature extraction and the ability of information interaction between features, enabling comprehensive feature extraction. Moreover, to consider the temporal dimension, the authors propose a TFFN for updating the template image. The network utilises the Transformer to fuse the tracking results of multiple frames with the initial frame, allowing the template image to continuously incorporate more information and maintain the accuracy of target features. Extensive experiments show that the tracker STFT achieves state‐of‐the‐art results on multiple benchmarks (OTB100, VOT2018, LaSOT, GOT‐10K, and UAV123). Especially, the tracker STFT achieves remarkable area under the curve score of 0.652 and 0.706 on the LaSOT and OTB100 benchmark respectively.

show abstract