Learning to Assign Orientations to Feature Points

Abstract

We show how to train a Convolutional Neural Network to assign a canonical orientation to feature points given an image patch centered on the feature point. Our method improves feature point matching upon the state-of-the art and can be used in conjunction with any existing rotation sensitive descriptors. To avoid the tedious and almost impossible task of finding a target orientation to learn, we propose to use Siamese networks which implicitly find the optimal orientations during training. We also propose a new type of activation function for Neural Networks that generalizes the popular ReLU, maxout, and PReLU activation functions. This novel activation performs better for our task. We validate the effectiveness of our method extensively with four existing datasets, including two non-planar datasets, as well as our own dataset. We show that we outperform the state-of-the-art without the need of retraining for each dataset.

Rethinking the sGLOH Descriptor

“…Linear Discriminant Analysis Hashing (LDAHash) [47] defines thresholds on SIFT linear projections, while Boosted Gradient Map (BGM) [48] and RDFs [17] threshold on the patch gradient map, parameters are learned from training data. Recently, Learned Arrangements of Three Patch Codes (LATCH) [49] compares learned sub-patch triplets, while in [18], [19] CNNs are trained respectively to assign the reference orientation and to define a full descriptor. Binary Online Learned Descriptor (BOLD) [40] defines a binary mask so that only the descriptor vector elements minimizing the intra-class variance on affine warps of the original patch are used in the matching.…”

“…These include the well known SIFT, that is considered as reference, LIOP, MIOP and MROGH, that represent the state of the art for rotational invariant descriptors, and RFDs, that are among the best binary 1. http://cvg.dsi.unifi.it/ descriptors. Additionally, the descriptor proposed in [18], here referred to as DeepDesc, and the SIFT coupled with the orientation estimation described in [19], both based on CNNs, were also included in the evaluation as interesting emerging approaches.…”

“…In order to gain further insight into the descriptor robustness, another evaluation in the case of planar scenes was done using the recent Viewpoint dataset [19] (see Fig. 3b), containing 5 planar scenes where images are subject to different incremental viewpoint and scale changes.…”

“…VLFeat was also used to get patches and orientation estimation for DeepDesc, since the DeepDesc implementation works only on already normalized patches. The CNN-based orientation estimation proposed in [19] was coupled with VL SIFT and denoted by the superscript "Ã" (more in detail, we used the EdgeFoci/SIFT with random rotation learned CNN).…”

“…These include the popular and well investigated SIFT descriptor, the rotational invariant LIOP, MIOP and MROGH descriptors, the recent learning-based binary Receptive Fields Descriptors (RFDs) [17] and two emerging approaches based on convolutional neural networks (CNNs) [18], [19]. Beside a standard evaluation in the case of planar scenes on the Oxford dataset [8] and the recent Viewpoint dataset [19], descriptors behavior is analyzed on four non-planar scene datasets using respectively the approximated overlap error [11], [20], structured light 3D data [21], epipolar constraints between triplets of calibrated views [22] and patches extracted using SfM [23]. Furthemore, descriptor properties in the case of object retrieval tests [10], [17] are also investigated through the ZuBuD [24] and Kentucky [25] datasets.…”