A Grassmannian Graph Approach to Affine Invariant Feature Matching

Abstract

In this work, we present a novel and practical approach to address one of the longstanding problems in computer vision: 2D and 3D affine invariant feature matching. Our Grassmannian Graph (GrassGraph) framework employs a two stage procedure that is capable of robustly recovering correspondences between two unorganized, affinely related feature (point) sets. The first stage maps the feature sets to an affine invariant Grassmannian representation, where the features are mapped into the same subspace. It turns out that coordinate representations extracted from the Grassmannian differ by an arbitrary orthonormal matrix. In the second stage, by approximating the Laplace-Beltrami operator (LBO) on these coordinates, this extra orthonormal factor is nullified, providing true affine-invariant coordinates which we then utilize to recover correspondences via simple nearest neighbor relations. The resulting GrassGraph algorithm is empirically shown to work well in non-ideal scenarios with noise, outliers, and occlusions. Our validation benchmarks use an unprecedented 440,000+ experimental trials performed on 2D and 3D datasets, with a variety of parameter settings and competing methods. State-of-the-art performance in the majority of these extensive evaluations confirm the utility of our method.

Associating Landmarks from SLAM’s Visual Structure

“…BIRCH operates through tree-based partitioning and is provided with a target total, where as DBSCAN generates a widely varying number of clusters. GrassGraph [13] requires a consistent total number, so we must randomly add (within the same bounds) or subtract some when using it. We ensure 120 landmark points with regular clustering and 20 with semantic clustering, based on the average number typically found in the first 100 frames of sequence 00.…”

“…To find associations between sets of landmarks from different visits we make use of GrassGraph [13], a SoTA method for finding associations between matching sets of points when their relative transformation is unknown. In return we receive a partial set of landmark-to-landmark associations and a recovered alignment transformation matrix.…”

“…The result are coordinates for each point in a new space that is invariant to affine transformations. [13] overcome a lingering rotational ambiguity from the use of Singular Value Decomposition by using eigenvectors to describe each point, derived from a graph of the distances between the new point coordinates. GrassGraph is meant for small sets of keypoints (runtime grows with the cube of the number of points) and so our conversion of messy 10,000+ point pointclouds into fewer landmarks is critical to feasibly using it for association.…”

“…We measure the performance of GrassGraph [13] when associating landmarks to quantify what effects make the task difficult. The performance of GrassGraph itself is measured in terms of the number of associations it recovers, as well as the accuracy of the estimated aligning transformations.…”

“…Our contribution is the exploration of a new approach to integrate SLAM-recovered structure into visual place recognition, in hopes of overcoming the difficulties of this previous approach. To overcome the problem of coverage encountered by [11], we associate individual landmarks extracted from imitation scans using SoTA graph association methods [13]. The goal is that that full spatial coverage should not be required, instead associating landmarks detected in the area of overlap.…”