Self-supervising Fine-Grained Region Similarities for Large-Scale Image Localization

Ge, Yixiao; Wang, Haibo; Zhu, Feng; Zhao, Rui; Li, Hongsheng

doi:10.1007/978-3-030-58548-8_22

Cited by 107 publications

(119 citation statements)

References 40 publications

Supporting

Mentioning

119

Contrasting

Order By: Relevance

“…It also achieves competitive results on Pitts30k and Robotcar-S2 datasets. Note that when training TransVPR on Pitts30k dataset, we only used the weakly supervised learning strategy in [2], and we can expect further boost of TransVPR performance using the fine-grained supervision proposed by [17]. Taking the average of all datasets, our method surpasses the global feature retrieval based methods by a large margin, and outperforms the two-stage approaches, SP-SuperGlue, DELG and Patch-NetVLAD, with absolute gains of 10.8%, 5.9% and 7.2% on Recall@1 score.…”

Section: Quantitative Resultsmentioning

confidence: 99%

“…We compared TransVPR against several state-of-the-art algorithms, including two VPR methods based on nearestneighbor searching using global image representations: NetVLAD [2] and SFRS [17], and two models which extract both global and patch for two-stage pipeline (i.e., retrieval and re-ranking): Patch-NetVLAD [19] and DELG [6]. For Patch-NetVLAD, we tested both its speed-focused and performance-focused configurations, denoted as Patch-NetVLAD-s and Patch-NetVLAD-p respectively.…”

Section: Compared Methodsmentioning

confidence: 99%

“…Experimental results show that the proposed TransVPR achieves superior performance on VPR benchmark datasets with low computational time and memory requirements. It outperforms the state-of-the-art VPR approaches [2,6,17,19,41] by significant margins (5.8% absolute increase on Recall@1 compared with the best baseline method, DELG [6]).…”

Section: Introductionmentioning

confidence: 93%

See 2 more Smart Citations

TransVPR: Transformer-based place recognition with multi-level attention aggregation

Wang¹,

Shen²,

Zuo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Visual place recognition is a challenging task for applications such as autonomous driving navigation and mobile robot localization. Distracting elements presenting in complex scenes often lead to deviations in the perception of visual place. To address this problem, it is crucial to integrate information from only task-relevant regions into image representations. In this paper, we introduce a novel holistic place recognition model, TransVPR, based on vision Transformers. It benefits from the desirable property of the selfattention operation in Transformers which can naturally aggregate task-relevant features. Attentions from multiple levels of the Transformer, which focus on different regions of interest, are further combined to generate a global image representation. In addition, the output tokens from Transformer layers filtered by the fused attention mask are considered as key-patch descriptors, which are used to perform spatial matching to re-rank the candidates retrieved by the global image features. The whole model allows end-to-end training with a single objective and image-level supervision. TransVPR achieves state-of-the-art performance on several real-world benchmarks while maintaining low computational time and storage requirements.

show abstract

Section: Quantitative Resultsmentioning

confidence: 99%

Section: Compared Methodsmentioning

confidence: 99%

See 1 more Smart Citation

TransVPR: Transformer-based place recognition with multi-level attention aggregation

Wang¹,

Shen²,

Zuo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The task of image retrieval requires searching images of the same content or object from a large-scale database based on their feature similarities, given images of interest. It is a fundamental task for many visual applications, e.g., face recognition (Guillaumin et al, 2009;Cui et al, 2013), person re-identification (Bak & Carr, 2017;Liu et al, 2017), image localization (Arandjelovic et al, 2016;Ge et al, 2020b). Deep learning-based methods improve the accuracy of image retrieval by casting it as metric learning tasks (Gordo et al, 2016;Chen et al, 2018;Brown et al, 2020), architecture design/search tasks (Zhou et al, 2019), or self-/semi-supervised learning tasks (Ge et al, 2020c;a).…”

Section: Modelmentioning

confidence: 99%

Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval

Zhang¹,

Ge²,

Shen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The task of hot-refresh model upgrades of image retrieval systems plays an essential role in the industry but has never been investigated in academia before. Conventional cold-refresh model upgrades can only deploy new models after the gallery is overall backfilled, taking weeks or even months for massive data. In contrast, hot-refresh model upgrades deploy the new model immediately and then gradually improve the retrieval accuracy by backfilling the gallery on-the-fly.Compatible training has made it possible, however, the problem of model regression * with negative flips poses a great challenge to the stable improvement of user experience. We argue that it is mainly due to the fact that new-to-old positive query-gallery pairs may show less similarity than new-to-new negative pairs. To solve the problem, we introduce a Regression-Alleviating Compatible Training (RACT) method to properly constrain the feature compatibility while reducing negative flips. The core is to encourage the new-to-old positive pairs to be more similar than both the new-to-old negative pairs and the new-to-new negative pairs. An efficient uncertainty-based backfilling strategy is further introduced to fasten accuracy improvements. Extensive experiments on large-scale retrieval benchmarks (e.g., Google Landmark) demonstrate that our RACT effectively alleviates the model regression for one more step towards seamless model upgrades § . † Corresponding authors. ‡ Work done when Binjie, Yantao and Yu are at ARC Lab, Tencent PCG. * Throughout this paper, the term model regression indicates the degradation of retrieval performance when the hot-refresh model upgrades proceed.§ The code will be available at https://github.com/binjiezhang/RACT_ICLR2022.

show abstract

“…To alleviate this problem, we resort to a fusion result of global features to make global description more robust. We fuse NetVLAD [1], DELG [3], APGeM [17,21] and OpenIBL [8] features as the final global representation.…”

Section: Fusion Of Multiple Global Featuresmentioning

confidence: 99%

MegLoc: A Robust and Accurate Visual Localization Pipeline

Shuxue¹,

He²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Self-supervising Fine-Grained Region Similarities for Large-Scale Image Localization

Cited by 107 publications

References 40 publications

TransVPR: Transformer-based place recognition with multi-level attention aggregation

TransVPR: Transformer-based place recognition with multi-level attention aggregation

Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval

MegLoc: A Robust and Accurate Visual Localization Pipeline

Contact Info

Product

Resources

About