Differential Reinforcement and Global Collaboration Network for RGBT Tracking

Mei, Jiatian; Zhou, Dongming; Alsaedi, Ahmed; Nie, Rencan; He, Kangjian

doi:10.1109/jsen.2023.3244834

Cited by 24 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Offline appearance model without considering the state changes of objects leads to limited discriminative ability. The second one uses the tracking results of each frame to generate new training samples and trains the tracker online during tracking process (Wang, Li, and Tang 2018;Zhai et al 2019;Zhang et al 2018;Mei et al 2023), as shown in Figure 1 (a). However, this update method is prone to tracking drift and neglects the correlation of spatial and temporal information.…”

Section: It Has Been Successfully Deployed In Various Applicationsmentioning

confidence: 99%

“…Despite the strong similarity to VOT, RGBT tracking needs to consider the fusion of complementary information between two modalities, which limits the exploitation of temporal information. Prevailing RGBT trackers usually exploit temporal information through online training, and two classical works are correlation filter-based trackers (Wang, Li, and Tang 2018;Zhai et al 2019) and MDNet-based trackers (Zhang et al 2018;Mei et al 2023;Xiao et al 2022). The former mostly does not require offline training but samples and trains online.…”

Section: Related Work Temporal Information Exploitationmentioning

confidence: 99%

See 1 more Smart Citation

Temporal Adaptive RGBT Tracking with Modality Prompt

Wang,

Liu,

et al. 2024

AAAI

View full text Add to dashboard Cite

RGBT tracking has been widely used in various fields such as robotics, surveillance processing, and autonomous driving. Existing RGBT trackers fully explore the spatial information between the template and the search region and locate the target based on the appearance matching results. However, these RGBT trackers have very limited exploitation of temporal information, either ignoring temporal information or exploiting it through online sampling and training. The former struggles to cope with the object state changes, while the latter neglects the correlation between spatial and temporal information. To alleviate these limitations, we propose a novel Temporal Adaptive RGBT Tracking framework, named as TATrack. TATrack has a spatio-temporal two-stream structure and captures temporal information by an online updated template, where the two-stream structure refers to the multi-modal feature extraction and cross-modal interaction for the initial template and the online update template respectively. TATrack contributes to comprehensively exploit spatio-temporal information and multi-modal information for target localization. In addition, we design a spatio-temporal interaction (STI) mechanism that bridges two branches and enables cross-modal interaction to span longer time scales. Extensive experiments on three popular RGBT tracking benchmarks show that our method achieves state-of-the-art performance, while running at real-time speed.

show abstract

Section: It Has Been Successfully Deployed In Various Applicationsmentioning

confidence: 99%

Section: Related Work Temporal Information Exploitationmentioning

confidence: 99%

Temporal Adaptive RGBT Tracking with Modality Prompt

Wang,

Liu,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Algorithms [32][33] [34] have proved the importance of establishing long-term dependence on cross-modality information. Inspired by this, we adopt two Transformer decoders to carry out bidirectional joint modulation and establish cross-modalities dependency.…”

Section: Cross-modulation Modulementioning

confidence: 99%

Edge-Aware and Relation Mining Network for RGBT Tracking

Xiao,

chen,

Meng

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, the fusion tracking of RGB and thermal infrared images (RGBT) hasreceived most research interest. However, existing RGBT tracking algorithmsbased on deep learning mainly perform modality weight allocation to highlighttarget information, but background clutter may limit the improvement of trackingperformance. To solve this problem, we propose a new edge-aware and rela-tion mining network for RGBT tracking. This network takes advantage of thecomplementary advantages of edge information and deep information to makeup for the defect that deep information is susceptible to background interfer-ence. Meanwhile, a cross-modality relation mining module is designed to explorethe node correlation score to highlight target state information and reduce thebackground noise. Also, Transformer is utilized for bidirectional modulation tocapture cross-modality dependencies. Finally, multi-branch supervised learningand decision-level fusion are performed to improve tracking accuracy and suc-cess rate further. Extensive experiments on three publicly available datasets showthat the proposed algorithm performs excellently.

show abstract