Physical Scale Keypoints: Matching and Registration for Combined Intensity/Range Images

Abstract

We present a new framework for detecting, describing, and matching keypoints in combined range-intensity data, resulting in what we call physical scale keypoints. We first produce an image mesh by backprojecting associated 2D intensity images onto the 3D range data. We detect and describe keypoints on the image mesh using an analogue of the SIFT algorithm for images with two key modifications: the process is made insensitive to viewpoint and structural discontinuities using a novel bilinear filter, and a physical scale space is constructed that exploits the reliable range measurements. Keypoints are matched between scans only when their physical scales agree, avoiding many potential false matches. Finally, the matches are rank-ordered using a new quality measure and supplied to a registration algorithm that refines each match into a rigid transformation for the entire scan pair. We report experimental results on keypoint detection and matching and range scan registration and verification in a set of difficult real-world scan pairs, showing that the new physical scale keypoints are demonstrably better than a competing approach based on backprojected SIFT keypoints.

Using retinex for point selection in 3D shape registration

“…The detected key points are useful in the context of registration, and with direct application of ICP variants, they can produce accurate registration results with significantly improved computational efficiency. Future research will investigate the similarity between the shapes defined by these key points and the original complete point sets, how the detected key points can be applied for the generation of levels of detail for efficient data transmission, rendering and visualization, and how the detected points can be incorporated into feature extraction and matching methods [7,23] for applications such as registration.…”

“…These values are smoothed over the whole image, and points with local maxima larger than a threshold are selected as key points. In [23], given an intensity image associated with a range map, the image mesh is generated and smoothed using a multiple scale bilateral filter, then the gradient at each vertex is estimated using the Laplace-Beltrami operator (LBO). Points with locally extremal gradients are filtered by thresholding their LBO response and suppressed by a non-maximal scheme to finally detect key points.…”

Utilization of Scale in Keypoint Detection and Feature Description

“…SIFT successfully achieved this task using image intensity and became one of the most widely used algorithms. Developing scale-space applications on 3D surfaces has lead to the diffusion of geometric quantities like mean curvature [57,58,59] and normal vectors [60,61], sensor intensities [62], physical point locations [63,64,65], arbitrary surface signal [59,66,67] and a mixture of range and intensity data [68]. This section presents an overview of various methods that are most aligned to our proposed scale-space.…”

“…ISOC [16]) is interpolated to account for various factors such as reference frame orientation uncertainty, partial occlusions, or sensor noise [54]. It has long been the standard of feature histogram construction to interpolate all components across neighboring bins [74,88,66,68,139,140]. Our first modification to OUR-CVFH is to…”

“…Multi-scale frameworks are generalization of scale-space operators that offer desirably properties and computationally efficiency, but do not necessary completely satisfy the scale-space axioms. There are many examples of multi-scale frameworks on 3D meshes [146,68,147,148]. These methods still examine the object at various scales to find scale-defined clusters or keypoints, but often pre-define the scales of interest.…”