Global Tracking via Ensemble of Local Trackers

Zhou, Zikun; Chen, Jianqiu; Pei, Wenjie; Kaige, Mao,; Wang, Hongpeng; He, Zhenyu

doi:10.1109/cvpr52688.2022.00856

Cited by 32 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, the language token has not been attended to by visual features. JointNLT [45] unifies the cross-modality relation and the cross-temporal relation among natural language and image, which can be implemented in both grounding tasks and tracking tasks.…”

Section: Vision-language Object Trackingmentioning

confidence: 99%

Multimodal Features Alignment for Vision–Language Object Tracking

Ye,

Xiao,

Liu

2024

Remote Sensing

View full text Add to dashboard Cite

Vision–language tracking presents a crucial challenge in multimodal object tracking. Integrating language features and visual features can enhance target localization and improve the stability and accuracy of the tracking process. However, most existing fusion models in vision–language trackers simply concatenate visual and linguistic features without considering their semantic relationships. Such methods fail to distinguish the target’s appearance features from the background, particularly when the target changes dramatically. To address these limitations, we introduce an innovative technique known as multimodal features alignment (MFA) for vision–language tracking. In contrast to basic concatenation methods, our approach employs a factorized bilinear pooling method that conducts squeezing and expanding operations to create a unified feature representation from visual and linguistic features. Moreover, we integrate the co-attention mechanism twice to derive varied weights for the search region, ensuring that higher weights are placed on the aligned visual and linguistic features. Subsequently, the fused feature map with diverse distributed weights serves as the search region during the tracking phase, facilitating anchor-free grounding to predict the target’s location. Extensive experiments are conducted on multiple public datasets, and our proposed tracker obtains a success score of 0.654/0.553/0.447 and a precision score of 0.872/0.556/0.513 on OTB-LANG/LaSOT/TNL2K. These results are satisfying compared with those of recent state-of-the-art vision–language trackers.

show abstract

Section: Vision-language Object Trackingmentioning

confidence: 99%

Multimodal Features Alignment for Vision–Language Object Tracking

Ye,

Xiao,

Liu

2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Then, some trackers (Li et al 2022;Wang et al 2021b;Yang et al 2020b) follow this paradigm to design different models to solve the grounding task and tracking task separately. Recently, JointNLT (Zhou et al 2023) performs tracking and grounding using a unified model, which simplifies the overall framework and enables end-to-end optimization. Tracking by language and box specification (NL+BBOX) specifies the target through both the initial bounding box and natural language.…”

Section: Vision-language Trackingmentioning

confidence: 99%

“…2) tracking the target based on the language description and the predicted bounding box. Recently, JointNLT (Zhou et al 2023) proposes a unified network to jointly conduct locating and tracking, enabling…”

Section: Introductionmentioning

confidence: 99%

“…Typically, VLT (Guo et al 2022) designs a ModaMixer to fuse the language feature and visual feature by channel-wise attention. JointNLT (Zhou et al 2023) builds feature interactions between language and vision through self-attention mechanisms. However, they ignore the semantic gap between different modalities, resulting in a tendency for trained visuallanguage trackers to rely on semantic information in language references, which limits their performance in pure bounding box reference setting (Guo et al 2022).…”

Section: Introductionmentioning

confidence: 99%

“…2) Modality-adaptive target localization. Existing trackers design various static box heads to estimate the target state, such as anchor-free head (Ye et al 2022a), corner-based head (Zhou et al 2023), and pointbased head (Ma et al 2023). These heads commonly take the reference-enhanced features of the search region as input, and regress the target box through offline trained parameters.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Unifying Visual and Vision-Language Tracking via Contrastive Learning

Ma,

Tang,

Yang

et al. 2024

AAAI

View full text Add to dashboard Cite

Single object tracking aims to locate the target object in a video sequence according to the state specified by different modal references, including the initial bounding box (BBOX), natural language (NL), or both (NL+BBOX). Due to the gap between different modalities, most existing trackers are designed for single or partial of these reference settings and overspecialize on the specific modality. Differently, we present a unified tracker called UVLTrack, which can simultaneously handle all three reference settings (BBOX, NL, NL+BBOX) with the same parameters. The proposed UVLTrack enjoys several merits. First, we design a modality-unified feature extractor for joint visual and language feature learning and propose a multi-modal contrastive loss to align the visual and language features into a unified semantic space. Second, a modality-adaptive box head is proposed, which makes full use of the target reference to mine ever-changing scenario features dynamically from video contexts and distinguish the target in a contrastive way, enabling robust performance in different reference settings. Extensive experimental results demonstrate that UVLTrack achieves promising performance on seven visual tracking datasets, three vision-language tracking datasets, and three visual grounding datasets. Codes and models will be open-sourced at https://github.com/OpenSpaceAI/UVLTrack.

show abstract

Tracking

Xu,

Lin,

et al. 2023

Synthesis Lectures on Computer Vision

View full text Add to dashboard Cite

Global Tracking via Ensemble of Local Trackers

Cited by 32 publications

References 32 publications

Multimodal Features Alignment for Vision–Language Object Tracking

Multimodal Features Alignment for Vision–Language Object Tracking

Unifying Visual and Vision-Language Tracking via Contrastive Learning

Tracking

Contact Info

Product

Resources

About