Learning Soft Mask Based Feature Fusion with Channel and Spatial Attention for Robust Visual Object Tracking

Fiaz, Mustansar; Mahmood, Arif; Jung, Soon Ki

doi:10.3390/s20144021

Cited by 11 publications

(4 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The outputs of the

and

layers consist of

and

respectively. The one-stage region proposal network is illustrated in detail in [ 20 ]. It can be employed to obtain the proposals for a visual tracker.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Song et al [ 19 ] performed different kinds of adversarial networks to generate variable samples, which helped to identify richer representation for tracking. Fiaz et al [ 20 ] proposed a soft mask feature fusion mechanism, which can be easily integrated into the conventional Siamese tracking framework to enhance the discriminative capability when distinguish the target from the background. Gordon et al [ 21 ] introduced the real-time recurrent regression networks to combine the multiple appearance features and motion information together, then perform the spatial-temporal fusion to accomplish a tracking network that increases the precision of the tracking results.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Proposal-Based Visual Tracking Using Spatial Cascaded Transformed Region Proposal Network

Zhang

Luo

Fan

2020

Sensors

View full text Add to dashboard Cite

Region proposal network (RPN) based trackers employ the classification and regression block to generate the proposals, the proposal that contains the highest similarity score is formulated to be the groundtruth candidate of next frame. However, region proposal network based trackers cannot make the best of the features from different convolutional layers, and the original loss function cannot alleviate the data imbalance issue of the training procedure. We propose the Spatial Cascaded Transformed RPN to combine the RPN and STN (spatial transformer network) together, in order to successfully obtain the proposals of high quality, which can simultaneously improves the robustness. The STN can transfer the spatial transformed features though different stages, which extends the spatial representation capability of such networks handling complex scenarios such as scale variation and affine transformation. We break the restriction though an easy samples penalization loss (shrinkage loss) instead of smooth L1 function. Moreover, we perform the multi-cue proposals re-ranking to guarantee the accuracy of the proposed tracker. We extensively prove the effectiveness of our proposed method on the ablation studies of the tracking datasets, which include OTB-2015 (Object Tracking Benchmark 2015), VOT-2018 (Visual Object Tracking 2018), LaSOT (Large Scale Single Object Tracking), TrackingNet (A Large-Scale Dataset and Benchmark for Object Tracking in the Wild) and UAV123 (UAV Tracking Dataset).

show abstract

“…The outputs of the

and

layers consist of

and

respectively. The one-stage region proposal network is illustrated in detail in [ 20 ]. It can be employed to obtain the proposals for a visual tracker.…”

Section: Proposed Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Proposal-Based Visual Tracking Using Spatial Cascaded Transformed Region Proposal Network

Zhang

Luo

Fan

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…In the field of image classification, such as end‐to‐end object detection with transformers [16], the modified transformer structure has secured good performance. In the field of target tracking, such as transformer tracking [4] and transformer meets tracker [17–19], the structures of the encoder and decoder have been modified and innovated according to the characteristics of target tracking task, and the algorithms have reached the most advanced level.…”

Section: Related Workmentioning

confidence: 99%

Multi‐template temporal information fusion for Siamese object tracking

Wang

et al. 2022

IET Computer Vision

View full text Add to dashboard Cite

The object tracking algorithm based on Siamese network often extracts the deep feature of the target to be tracked from the first frame of the video sequence as a template, and uses the template for the whole tracking process. Because the manually annotated target in the first frame of video sequence is more accurate, these algorithms often have stable performance. However, it is difficult to adapt to the changing target features only using the target template extracted from the first frame. Inspired by the feature fusion network based on a transformer, this paper proposes a template update module called multi‐template temporary information fusion module (MTFM), which can be trained offline. By fusing multiple target template features on time series, the template can always adapt to the changes of target appearance in the tracking process. In order to train the MTFM, this paper proposes a training method using time series data and Mean Square Error (MSE) as the loss function. This paper uses the MTFM on SiamFC++ tracker, and obtains good experimental results in three challenging datasets, including VOT2016, OTB100 and GOT‐10k. The running speed of the algorithm on graphics processing unit (GPU) is maintained at about 200fps, which exhibits good real‐time performance.

show abstract

“…Table 2 shows the compared trackers and their corresponding accuracy, robustness, EAO score, and tracking speed. We consider GradNet [71], SCS-Siam [72], MemTrack [14], ECOhc [6], SATIN [73], DSiam [11], CSRDCF [74], SiamFC [10], DCFNet [75], DensSiam [76], DSST [66], and SRDCF [61] trackers for the comparison. The top three performed trackers in terms of VOT measure metrics are highlighted using red, green, and blue colors, respectively.…”

Section: Experiments On Vot2017 and Vot2018 Benchmarkmentioning

confidence: 99%

Siamese High-Level Feature Refine Network for Visual Object Tracking

et al. 2020

Self Cite

View full text Add to dashboard Cite

Siamese network-based trackers are broadly applied to solve visual tracking problems due to its balanced performance in terms of speed and accuracy. Tracking desired objects in challenging scenarios is still one of the fundamental concerns during visual tracking. This research paper proposes a feature refined end-to-end tracking framework with real-time tracking speed and considerable performance. The feature refine network has been incorporated to enhance the target feature representation power, utilizing high-level semantic information. Besides, it allows the network to capture the salient information to locate the target and learns to represent the target feature in a more generalized way advancing the overall tracking performance, particularly in the challenging sequences. But, only the feature refine module is unable to handle such challenges because of its less discriminative ability. To overcome this difficulty, we employ an attention module inside the feature refine network that strengths the tracker discrimination ability between the target and background. Furthermore, we conduct extensive experiments to ensure the proposed tracker’s effectiveness using several popular tracking benchmarks, demonstrating that our proposed model achieves state-of-the-art performance over other trackers.

show abstract

Learning Soft Mask Based Feature Fusion with Channel and Spatial Attention for Robust Visual Object Tracking

Cited by 11 publications

References 95 publications

Proposal-Based Visual Tracking Using Spatial Cascaded Transformed Region Proposal Network

Proposal-Based Visual Tracking Using Spatial Cascaded Transformed Region Proposal Network

Multi‐template temporal information fusion for Siamese object tracking

Siamese High-Level Feature Refine Network for Visual Object Tracking

Contact Info

Product

Resources

About