Tiny Descriptors for Image Retrieval with Unsupervised Triplet Hashing

Abstract

A typical image retrieval pipeline starts with the comparison of global descriptors from a large database to find a short list of candidate matches. A good image descriptor is key to the retrieval pipeline and should reconcile two contradictory requirements: providing recall rates as high as possible and being as compact as possible for fast matching. Following the recent successes of Deep Convolutional Neural Networks (DCNN) for large scale image classification, descriptors extracted from DCNNs are increasingly used in place of the traditional hand crafted descriptors such as Fisher Vectors (FV) with better retrieval performances. Nevertheless, the dimensionality of a typical DCNN descriptor -extracted either from the visual feature pyramid or the fully-connected layers-remains quite high at several thousands of scalar values.In this paper, we propose Unsupervised Triplet Hashing (UTH), a fully unsupervised method to compute extremely compact binary hashes -in the 32-256 bits range-from high-dimensional global descriptors. UTH consists of two successive deep learning steps. First, Stacked Restricted Boltzmann Machines (SRBM), a type of unsupervised deep neural nets, are used to learn binary embedding functions able to bring the descriptor size down to the desired bitrate. SRBMs are typically able to ensure a very high compression rate at the expense of loosing some desirable metric properties of the original DCNN descriptor space. Then, triplet networks, a rank learning scheme based on weight sharing nets is used to fine-tune the binary embedding functions to retain as much as possible of the useful metric properties of the original space. A thorough empirical evaluation conducted on multiple publicly available dataset using DCNN descriptors shows that our method is able to significantly outperform state-of-the-art unsupervised schemes in the target bit range. * Equal contributions from Jie Lin, Olivier Morère and Julie Petta.

Compact Hash Codes for Efficient Visual Descriptors Retrieval in Large Scale Databases

“…number of nearest centroids n for the m-k-meansn 1 method, and Hamming distance threshold (H). The proposed approaches are compared with several state-of-the-art methods, among which the recent UTH method [31]; while some of these methods were originally proposed for engineered features, they have been evaluated on CNN features (results reported from [31]). Retrieval is performed using the coarse-to-fine approach used in [24], where the hash is used to select a candidate list of images and CNN descriptors are used to re-rank this list.…”

“…In this experiment (Table V) we compare the behavior of retrieval performance for different lengths of the hash code (for m-kmeans-t 1 ) and for different values of n nearest neighbors (for [60] 0.381 0.225 -PCAHash [54] 0.528 0.239 -LSH [61] 0.431 0.239 -SKLSH [62] 0.241 0.134 -SH [2] 0.522 0.232 -SRBM [63] 0.516 0.212 -UTH [31] 0.571 0.240 -m-k-means-n 1 (n = m-kmeans-n 1 ). Experiments were made for SIFT1M for different values of recall@R. Fig.…”

“…Lin et al [31] have proposed to use unsupervised two steps hashing of CNN features. In the first step Stacked Restricted Boltzmann Machines learn binary embedding functions, then fine tuning is performed to retain the metric properties of the original feature space.…”

Unsupervised Triplet Hashing for Fast Image Retrieval

“…Hashing, which uses mapping functions to transform a high-dimensional feature vector into a compact and expressive binary codes [1,2,3], has shown significant success for fast image retrieval. In recent years, with the rapid development of Convolutional Neural Network (CNN), several CNNbased hashing methods [4,5,6,7,8,9,10] have been proposed and demonstrated promising results. In particular, unsupervised hashing learning has recently received increasing attention because it does not require labeled training data thus making the methods widely applicable.…”

“…In particular, unsupervised hashing learning has recently received increasing attention because it does not require labeled training data thus making the methods widely applicable. The earliest studies use stacked Restricted Boltzmann Machines (RBMs) to encode binary codes [8,9] for unsupervised hashing. However, RBMs are complex and require pre-training, which are not efficient for practical applications.…”