Rotation and translation covariant match kernels for image retrieval

Abstract

International audienceMost image encodings achieve orientation invariance by aligning the patches to their dominant orientations and translation invariance by completely ignoring patch position or by max-pooling. Albeit successful, such choices introduce too much invariance because they do not guarantee that the patches are rotated or translated consistently. In this paper, we propose a geometric-aware aggregation strategy, which jointly encodes the local descriptors together with their patch dominant angle or location. The geometric attributes are encoded in a continuous manner by leveraging explicit feature maps. Our technique is compatible with generic match kernel formulation and can be employed along with several popular encoding methods, in particular Bag-of-Words, VLAD and the Fisher vector. The method is further combined with an efficient monomial embedding to provide a codebook-free method aggregating local descriptors into a single vector representation. Invariance is achieved by efficient similarity estimation of multiple rotations or translations, offered by a simple trigonometric polynomial. This strategy is effective for image search, as shown by experiments performed on standard benchmarks for image and particular object retrieval, namely Holidays and Oxford buildings

Explicit Spatial Encoding for Deep Local Descriptors

“…The kernel K (or the feature map f ) is constructed to approximate the Von Mises kernel [51]. We propose encoding function g xy :…”

Understanding and Improving Kernel Local Descriptors

“…As non-linear kernel for scalars we use the normalized Von Mises probability density function 1 , which is used for image [61] and patch [16] representations. It is parametrized by κ controlling the shape of the kernel, where lower κ corresponds to wider kernel, i.e.…”

“…The reader is encouraged to read prior work for details on these feature maps [63,18], which are previously used in various contexts [61,16].…”

“…In this work, we choose to work with a particular family of hand-crafted descriptors, the so called kernel descriptors [13,12,61]. They provide a quite flexible framework for matching sets, patches in our case, by encoding different properties of the set elements, pixels in our case.…”

Multiple-Kernel Local-Patch Descriptor

“…Feature maps. As non-linear kernel for scalars we use the normalized Von Mises probability density function 1 , which is used for image [31] and patch [9] representation. It is parametrized by κ controlling the shape of the kernel, where lower κ corresponds to wider kernel.…”