Kernelizing Spatially Consistent Visual Matches for Fine-Grained Classification

Spatially Localized Visual Dictionary Learning

“…Its principle is to progressively compress the spatially localized vocabulary Z by recursively eliminating the less discriminant atoms. Each recursion includes 3 main steps: (i) the computation of the SNN representations [10] of the images in the training set X (based on the current spatially localized vocabulary Z (t) ), (ii) the learning of a multi-class support vector machines on top of the computed SNN representations and (iii), the elimination of the less discriminant spatial atoms Z (t) j in Z (t) . These 3 steps are repeated T times.…”

“…Each image in X is supposed to be described by a set of d-dimensional spatially localized local features X = {x}. To map these local features onto the N spatial atoms Zj of the vocabulary Z, we use the Shared Nearest Neighbor embedding method introduced in [10] and from which we can derive the following explicit embedding function:…”

“…is a ranking function returning the rank of z according to its L2 distance to x. The function ϕ(r) is a rank-based activation function that is decreasing with r and that is close to zero when the rank r becomes sufficiently high (see [10] for more details). Intuitively, each component Φj(X) of the high-dimensional representation Φ(X) quantifies how likely it is that the image X contains the same visual pattern than the one depicted by the spatial atom Zj.…”

“…Therefore, an approximate nearest neighbor search method is used and only the approximate K nearest neighbors of each x are considered in the SNN embedding. In [10], Leveau et al propose a variant of the base SNN scheme for embedding rigid geometry constraints. This is done by multiplying the activation function rx(z) by a geometry consistency indicator function δx(z) equals to one if z is an inlier of a RANSAC-like algorithm estimating the best affine transformation between the image X containing x and the spatial atom Zj containing z (and zero otherwise).…”

“…Therefore, we propose a supervised method for learning a compact vocabulary of discriminant and spatially localized visual patches to be used as atoms of a highly interpretable image representation. To do so, we introduce a recursive visual patch selection technique built on top of a recently introduced embedding scheme called Shared Nearest Neighbor match kernel [10]. The interesting property of such embedding is that it explicitly maps the visual content of a given image onto a potentially huge set of visual patches.…”