Learning Transformation-Invariant Local Descriptors With Low-Coupling Binary Codes

Abstract

Despite the great success achieved by prevailing binary local descriptors, they are still suffering from two problems: 1) vulnerable to the geometric transformations; 2) lack of an effective treatment to the highly-correlated bits that are generated by directly applying the scheme of image hashing. To tackle both limitations, we propose an unsupervised Transformationinvariant Binary Local Descriptor learning method (TBLD). Specifically, the transformation invariance of binary local descriptors is ensured by projecting the original patches and their transformed counterparts into an identical high-dimensional feature space and an identical low-dimensional descriptor space simultaneously. Meanwhile, it enforces the dissimilar image patches to have distinctive binary local descriptors. Moreover, to reduce high correlations between bits, we propose a bottom-up learning strategy, termed Adversarial Constraint Module, where low-coupling binary codes are introduced externally to guide the learning of binary local descriptors. With the aid of the Wasserstein loss, the framework is optimized to encourage the distribution of the generated binary local descriptors to mimic that of the introduced low-coupling binary codes, eventually making the former more low-coupling. Experimental results on three benchmark datasets well demonstrate the superiority of the proposed method over the state-of-the-art methods. The project page is available at https://github.com/yoqim/TBLD.

Revisiting Unsupervised Local Descriptor Learning

“…In this section, we compare HybridDesc with several existing approaches including unsupervised deep local descriptors DeepBit (Lin et al 2019), GraphBit (Duan et al 2018), D-GraphBit (Wang et al 2022), TBLD (Miao et al 2021), BLCD (Fan et al 2021), RDRL (Yu et al 2019), supervised deep local descriptors DeepDesc (Simo-Serra et al 2015, DC (Zagoruyko and Komodakis 2015), TFeat (Balntas et al 2016), L2Net (Tian, Fan, and Wu 2017), Hard-Net (Mishchuk et al 2017), DOAP (He, Lu, and Sclaroff 2018), SOSNet (Tian et al 2019), Dynamic Soft Margin (DSM) (Zhang and Rusinkiewicz 2019), DualHard (Wang et al 2020a), HyNet (Tian et al 2020) as well as handcrafted descriptors SIFT (Lowe 2004), BRIEF (Calonder et al 2010), ORB (Rublee et al 2011). The evaluation is conducted on the UBC Phototour (Brown, Hua, and Winder 2011), HPatches (Balntas et al 2017), Heinly (Heinly, Dunn, and Frahm 2012) and W1BS (Mishkin et al 2015) datasets.…”

Mélange fabric image retrieval based on soft similarity learning

“…To reduce the information loss in hash code generation, many researchers introduce quantization loss. [39][40][41] Miao et al 42 proposed a bottom-up learning strategy, termed Adversarial Constraint Module, where low-coupling binary codes are introduced to guide the learning of binary local descriptors.…”

Region-Object Relation-Aware Dense Captioning via Transformer

“…D Ense captioning has gained significant attention from both the engineering and research communities recently. On the one hand, it facilitates important practical applications [1], such as human-robot interaction [2], navigation for the blind, object detection [3] [4] or segmentation [5] and imagetext retrieval [6] [7]. On the other hand, it poses substantial challenges to both computer vision and natural language processing research communities.…”