Learning Local-Global Multi-Graph Descriptors for RGB-T Object Tracking

Li, Chenglong; Zhu, Chengli; Zhang, Jian; Luo, Bin; Wu, Xiaohao; Tang, Jin

doi:10.1109/tcsvt.2018.2874312

Cited by 60 publications

(16 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cvejic et al [14] investigates the effect of pixel-level fusion of visible and infrared videos on object tracking performance. After that, the representative works are based on sparse representation [15], [1], [16], [17], manifold ranking [18], [19] and dynamic graph [20], [21]. Early works focus on the sparse representation due to their robustness to noise and outliers.…”

Section: A Traditional Methods For Rgbt Trackingmentioning

confidence: 99%

“…These works, however, rely on the structure-fixed graphs, and the relations among patches are not well explored. To handle this problem, Li et al [20] propose a spatially regularized graph learning to automatically explore the intrinsic relationship of global patches and local patches. Besides, Li et al [21] propose a sparse representation regularized graph learning to explore patch relations in an adaptive manner.…”

Section: A Traditional Methods For Rgbt Trackingmentioning

confidence: 99%

See 1 more Smart Citation

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

Lu,

Li,

Yan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

RGBT tracking has attracted increasing attention since RGB and thermal infrared data have strong complementary advantages, which could make trackers all-day and all-weather work. Existing works usually focus on extracting modality-shared or modality-specific information, but the potentials of these two cues are not well explored and exploited in RGBT tracking. In this paper, we propose a novel multi-adapter network to jointly perform modality-shared, modality-specific and instance-aware target representation learning for RGBT tracking. To this end, we design three kinds of adapters within an end-to-end deep learning framework. In specific, we use the modified VGG-M as the generality adapter to extract the modality-shared target representations. To extract the modality-specific features while reducing the computational complexity, we design a modality adapter, which adds a small block to the generality adapter in each layer and each modality in a parallel manner. Such a design could learn multilevel modality-specific representations with a modest number of parameters as the vast majority of parameters are shared with the generality adapter. We also design instance adapter to capture the appearance properties and temporal variations of a certain target. Moreover, to enhance the shared and specific features, we employ the loss of multiple kernel maximum mean discrepancy to measure the distribution divergence of different modal features and integrate it into each layer for more robust representation learning. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against the state-of-the-art methods.

show abstract

Section: A Traditional Methods For Rgbt Trackingmentioning

confidence: 99%

Section: A Traditional Methods For Rgbt Trackingmentioning

confidence: 99%

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

Lu,

Li,

Yan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Because the fusion of RGB and TIR data more easily achieves all-weather object tracking in the open environment, the researches on RGB-T object tracking methods become more and more popular. From the perspective of data fusion, the RGB-T object tracking framework can be roughly divided into traditional methods [ 38 , 39 ], sparse representation (SR)-based [ 40 , 41 , 42 , 43 , 44 ], graph-based [ 45 , 46 , 47 ], correlation filter-based [ 48 , 49 , 50 , 51 ], and deep learning-based approaches. Earlier studies used manual features to perform the appearance modeling of the target object.…”

Section: Related Workmentioning

confidence: 99%

Channel Exchanging for RGB-T Tracking

Zhao

Zhu

Ren

et al. 2021

Sensors

View full text Add to dashboard Cite

It is difficult to achieve all-weather visual object tracking in an open environment only utilizing single modality data input. Due to the complementarity of RGB and thermal infrared (TIR) data in various complex environments, a more robust object tracking framework can be obtained using video data of these two modalities. The fusion methods of RGB and TIR data are the core elements to determine the performance of the RGB-T object tracking method, and the existing RGB-T trackers have not solved this problem well. In order to solve the current low utilization of information intra single modality in aggregation-based methods and between two modalities in alignment-based methods, we used DiMP as the baseline tracker to design an RGB-T object tracking framework channel exchanging DiMP (CEDiMP) based on channel exchanging. CEDiMP achieves dynamic channel exchanging between sub-networks of different modes hardly adding any parameters during the feature fusion process. The expression ability of the deep features generated by our data fusion method based on channel exchanging is stronger. At the same time, in order to solve the poor generalization ability of the existing RGB-T object tracking methods and the poor ability in the long-term object tracking, more training of CEDiMP on the synthetic dataset LaSOT-RGBT is added. A large number of experiments demonstrate the effectiveness of the proposed model. CEDiMP achieves the best performance on two RGB-T object tracking benchmark datasets, GTOT and RGBT234, and performs outstandingly in the generalization testing.

show abstract

“…More details about the attributes are listed in Table 1. The performance of SiamFT on nineteen sequences are compared with 14 state-of-the-art trackers, including ECO [41], C-COT [42], CN [43], JSR [3], CSK [44], CT [45], L1 [46], MIL [47], RPT [48], STC [49], STRUCK [50], TLD [51], SGT [33], LGMG [36]. In these methods, the JSR, L1, SGT and LGMG methods are designed for RGB-infrared fusion tracking, while the others are originally developed for tracking based on visible images.…”

Section: B Sequences and Compared Trackersmentioning

confidence: 99%

SiamFT: An RGB-Infrared Fusion Tracking Method via Fully Convolutional Siamese Networks

Zhang

Peng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Object tracking based on visible images may fail when the visible images are unreliable, for example when the illumination condition is poor. Infrared images reveal thermal radiation of objects and are insensitive to these factors. Due to the complementary features of visible and infrared images, RGB-infrared fusion tracking has attracted widespread attention recently. In this paper, an RGB-infrared fusion tracking method based on the fully convolutional Siamese Networks, termed as SiamFT, is proposed. Visible and infrared images are firstly processed by two Siamese Networks, namely visible network and infrared network, respectively. Then, convolutional features of visible and infrared template images extracted from two Siamese Networks are concatenated to form fused template image. Convolutional features of visible and infrared search image are fused through the proposed feature fusion network adaptively. In particular, a modality weight computation method based on the response value of Siamese network is proposed to predict the reliability of different images. Cross-relation is then applied to the fused template feature and the fused search feature to produce the final response map, based on which the tracking results can be obtained. Extensive experiments indicate that the proposed SiamFT shows better performance than thestate-of-art fusion tracking algorithms at real-time speed. INDEX TERMS Deep learning, fusion tracking, object tracking, Siamese network.

show abstract

Learning Local-Global Multi-Graph Descriptors for RGB-T Object Tracking

Cited by 60 publications

References 42 publications

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

Channel Exchanging for RGB-T Tracking

SiamFT: An RGB-Infrared Fusion Tracking Method via Fully Convolutional Siamese Networks

Contact Info

Product

Resources

About