RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform

Abstract

Traditional feature matching methods such as scale-invariant feature transform (SIFT) usually use image intensity or gradient information to detect and describe feature points; however, both intensity and gradient are sensitive to nonlinear radiation distortions (NRD). To solve the problem, this paper proposes a novel feature matching algorithm that is robust to large NRD. The proposed method is called radiation-invariant feature transform (RIFT). There are three main contributions in RIFT: first, RIFT uses phase congruency (PC) instead of image intensity for feature point detection. RIFT considers both the number and repeatability of feature points, and detects both corner points and edge points on the PC map. Second, RIFT originally proposes a maximum index map (MIM) for feature description. MIM is constructed from the log-Gabor convolution sequence and is much more robust to NRD than traditional gradient map. Thus, RIFT not only largely improves the stability of feature detection, but also overcomes the limitation of gradient information for feature description. Third, RIFT analyzes the inherent influence of rotations on the values of MIM, and realizes rotation invariance. We use six different types of multi-model image datasets to evaluate RIFT, including optical-optical, infrared-optical, synthetic aperture radar (SAR)-optical, depth-optical, map-optical, and day-night datasets. Experimental results show that RIFT is much more superior to SIFT and SAR-SIFT. To the best of our knowledge, RIFT is the first feature matching algorithm that can achieve good performance on all the above-mentioned types of multi-model images. The source code of RIFT and multi-modal remote sensing image datasets are made public 1 .Index Terms-multi-modal image matching, nonlinear radiation distortions (NRD), feature matching, maximum index map (MIM), phase congruency (PC).

Generation of a Benchmark Dataset Using Historical Photographs for an Automated Evaluation of Different Feature Matching Methods

“…For the first and second test the standard implementations of ORB, SURF and MSER in OpenCV were used. The third test used the implementation of RIFT in Matlab (Li et al, 2018). The results are presented without outlier removal using brute force matching, outlier removal using a symmetry test and as a third approach outlier removal using Fundamental Matrix calculation with the random sample consensus (RANSAC) (Fischler and Bolles, 1981).…”

“…The third method used is called RIFT. The radiation-invariant feature transform is chosen because of its invariance to nonlinear radiation distortions (NRD) (Li et al, 2018) and the use of edge features in addition to corner features. Both effects can support the feature detection in historical images.…”

“…Independent maps for each orientation of a 2D log-Gabor filter are created and used for the detection of corner features as well as edge features. In the following, those features are described by a 216-dimensional feature vector calculated through a maximum index map based on a log-Gabor convolution sequence (Li et al, 2018). RIFT is currently not scale invariant and so it should perform bad on large scale-changes.…”

Multi Sensor Image Matching using Super Symmetric Affinity Tensors based HyperGraph Matching

“…Merely, the matching performance cannot be improved by suggestively enhancing the feature identifier, descriptor merely, or matching criterion, however, intended on improving the complete matching efficiency via enhancing the strategy and certain phases in the workflow To compare multimodal images, in [16] suggested developing phase congruency as a simplification of gradient data. The method presented in [17] outspreads this usage of phase congruency to generate a radiation-invariant characteristics transformation, i.e., lesser vulnerable to nonlinear radiation misrepresentations. The usage of a Siamese CNN framework and to figure comparative change amongst SAR and optical image patches were proposed by Merkle et al [18], with the aim of enlightening geo-localization accurateness of optical images.…”

Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks

“…Many local features are developed from the SIFT, such as the Affine-SIFT [6], the UC (Uniform Competency-based) features [7], and the PSO-SIFT [8]. Some studies establish feature descriptors by structure attributes instead of intensities, examples include the HOPC [9], the RIFT [10], the MSPC [11], and the MPFT [12]. Some works attempt to increase the percentage of correct correspondences through the Markov random field [13] and the Gaussian field [14] The combination of the feature-based and the area-based methods has been studied [15,16].…”