Sfoc: A Novel Multi-Directional and Multi-Scale Structural Descriptor for Multimodal Remote Sensing Image Matching

Abstract

Abstract. Accurate matching of multimodal remote sensing (RS) images (e.g., optical, infrared, LiDAR, SAR, and rasterized maps) is still an ongoing challenge because of nonlinear radiometric differences (NRD) between these images. Considering that structural properties are preserved between multimodal images, this paper proposes a robust matching method based on multi-directional and multi-scale structural features, which consist of two critical steps. Firstly, a novel structural descriptor named the Steerable Filters of first- and second-Order Channels (SFOC) is constructed to address severe NRD, which combines the first- and second-order gradient information by using the steerable filters to depict multi-directional and multi-scale structural features of images. Meanwhile, SFOC is further enhanced by performing the dilated Gaussian convolutions with different dilated rates on it, which can capture multi-level context structural features and improve the ability to resist noise. Then, a fast similarity measure, called Fast Normalized Cross-Correlation (Fast-NCCSFOC), is established to detect correspondences by a template matching scheme, which employs the Fast Fourier Transform (FFT) technique and the integral image to improve the matching efficiency. The performance of the proposed SFOC has been evaluated with many different kinds of multimodal RS images, and experimental results show its superior matching performance compared with the state-of-the-art methods.

Fast Double-Channel Aggregated Feature Transform for Matching Planetary Remote Sensing Images

“…In addition, in order to address the problems of rotation and scales in matching, the methods often use geographic reference coordinates for initial calibration to improve the matching accuracy [22]. This method of initial calibration based on geographic reference coordinates is widely used in matching optical-SAR images [8,22,23]. The key for area-based methods is establishing reliable similarity measures from the spatial or frequency domain instead of finding corresponding features from two images.…”

“…Two independent experiments are designed and can be shown in TABLE I. The other parameters in this paper are set according to the relevant research [29,51,23].…”

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

“…GAN-based models have demonstrated their potential in cloud removal for remote sensing applications in several studies [96]- [98]. Recently, attention has also been directed toward denoising SAR images [8], [9], [100] overcoming the sensitivity of gradient-based descriptors to noise. While [8] takes advantage of a PCA-based approach to filter out noise, [100] proposes a structural descriptor SFOC with dilated Gaussian convolution to resist noisy signals.…”

“…Recently, attention has also been directed toward denoising SAR images [8], [9], [100] overcoming the sensitivity of gradient-based descriptors to noise. While [8] takes advantage of a PCA-based approach to filter out noise, [100] proposes a structural descriptor SFOC with dilated Gaussian convolution to resist noisy signals. On the other hand, [9] integrated denoising into a fusion network using a hierarchical spatial-spectral structure in which residual blocks with channel attention mechanism (RBCA) is an important component.…”

Multi-Orientation Edge-Based Satellite Image Matching Method for Optical and Sar Images

“…However, the availability of training samples limits the generalization capacity in different application scenarios (Zhang et al, 2023). Template-based matching is also known as area-based image matching, which needs to determine the similarity between the template from the referenced image and the matched image in the spatial or frequency domain (Zhu et al, 2022). However, traditional template image matching usually relied on the intensity of images which has been found to be hardly applied to the optical and SAR image matching research due to the nonlinear radiance distortion (Ye et al, 2017).…”