3MRS: An Effective Coarse-to-Fine Matching Method for Multimodal Remote Sensing Imagery

Abstract

The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical–optical, optical–infrared, optical–depth, optical–map, optical–SAR, and day–night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available.

Multisource remote sensing image matching based on expanded phase consistency

“…In addition to spatial domain information, images can be described using frequency domain information such as phase information, which is highly stable and applicable, does not change because of variations in information such as the illumination and scale of the image, and can extract significant structural features in images with large nonlinear radiometric distortions 32 – 36 …”

“…In addition to spatial domain information, images can be described using frequency domain information such as phase information, which is highly stable and applicable, does not change because of variations in information such as the illumination and scale of the image, and can extract significant structural features in images with large nonlinear radiometric distortions. [32][33][34][35][36] Anisotropic filtering is a type of nonlinear filtering that can better preserve image edge information, facilitate image structure feature extraction, and increase the richness of feature points to a certain extent. The image-scale spatial nonlinear diffusion method proposed in the literature achieves the precise localization of edge regions at each scale through successive iterative calculations.…”

Feature Matching for Remote-Sensing Image Registration via Neighborhood Topological and Affine Consistency

“…Image registration is widely used in computer vision [18,19], pattern recognition [20][21][22], medical imaging [23,24], and especially remote-sensing image analysis [25][26][27][28]. From some literature reviews, we can find a detailed introduction on image registration methods [3,29].…”

Edge Consistency Feature Extraction Method for Multi-Source Image Registration

“…Yao et al proposed a histogram of absolute phase consistency gradients (HAPCG) algorithm, which extended the phase consistency model, established absolute phase consistency directional gradients, and built the HAPCG descriptors [17] achieving robust matching between different source images with large illumination and contrast difference. Fan et al proposed a 3MRS method based on a 2D phase consistency model to construct a new template feature based on Log-Gabor convolutional image sequences and used 3D phase correlation as a similarity metric [18]. In general, although these phasecongruency-based registration methods demonstrate excellent performance in multi-source image matching, there are still two problems for remote sensing images registration: (1) the interference of noise and texture on feature extraction cannot be avoided; (2) the computation of phase congruency involves Log-Gabor transforms at multiple scales and directions, which is heavy in computation and results in the extension of feature detection time.…”