An evaluation of conventional and deep learning‐based image‐matching methods on diverse datasets

Abstract

Image matching plays an important role in photogrammetry, computer vision and remote sensing. Modern deep learning‐based methods have been proposed for image matching; however, whether they will surpass and take the place of the conventional handcrafted methods in the remote sensing field still remains unclear. A comprehensive evaluation on stereo remote sensing images is also lacking. This paper comprehensively evaluates the performance of conventional and deep learning‐based image‐matching methods by dividing the matching process into feature point extraction, description and similarity measure on various datasets, including images captured from close‐range indoor and outdoor scenarios, unmanned aerial vehicles (UAVs) and satellite platforms. Different combinations of the three steps are evaluated. The experimental results reveal that, first, the performance of the different combinations varies between individual datasets, and it is difficult to determine the best combination. Second, by using more comprehensive indicators on all of the datasets, that is, the average rank and absolute rank, the combination of scale‐invariant feature transform (SIFT), ContextDesc and the nearest neighbour distance ratio (NNDR), and also the original SIFT, achieve the best results, and are recommended for use in remote sensing. Third, the deep learning‐based Sub‐SuperPoint extractor obtains a good performance, and is second only to SIFT. The learning based ContextDesc descriptor is as effective as the SIFT descriptor, and the learning based SuperGlue matcher is not as stable as NNDR, but leads to a few top‐performing combinations. Finally, the handcrafted methods are generally faster than the deep learning‐based methods, but the efficiency of the latter is acceptable. We conclude that although a full deep learning‐based method/combination has not yet beaten the conventional methods, there is still much room for improvement with the deep learning‐based methods because large‐scale aerial and satellite training datasets remain to be constructed, and specific methods for remote sensing images remain to be developed. The performance of the different combinations of feature extractor, descriptor and similarity measure varies between individual datasets. The combination of SIFT, ContextDesc and NNDR, and also the original SIFT, achieve the best results when using more comprehensive indicators on all the datasets. For extractor, the learning based Sub‐SuperPoint is second only to SIFT; for descriptor, learning‐based ContextDesc is as effective as the SIFT descriptor; and for matcher, learning‐based SuperGlue is not as stable as NNDR.

Robust Fusion of Multi-Source Images for Accurate 3D Reconstruction of Complex Urban Scenes

“…The unevenly distributed tie points require manual adjustments. Although learning-based image matching methods have shown promising performance, recent studies have demonstrated that these methods do not have obvious advantages over handcrafted ones in conventional 3D reconstruction tasks [39][40][41][42]. Directly applying these learning-based methods to match cross-platform images and reconstruct 3D models of complex urban scenes remains challenging.…”

Improvement of the spaceborne synthetic aperture radar stereo positioning accuracy without ground control points

“…The stereo positioning of remote sensing images involves calculating ground point coordinates using a space intersection method based on a geometric positioning model of remote sensing images and the homologous points of stereo image pairs (Albanwan & Qin, 2022; Deng et al., 2019; Ji et al., 2023; Luo, Qiu, Peng, et al., 2022). Remote sensing satellites typically include optical, synthetic aperture radar (SAR) and laser altimetry satellites.…”

Mono‐MVS: textureless‐aware multi‐view stereo assisted by monocular prediction

“…The active type includes techniques like lidar, structured light and RGBD cameras. The passive type also referred to as image‐based 3D reconstruction (Ji et al., 2023; Remondino & El‐Hakim, 2006), involves a sequential workflow of image retrieval, feature extraction and matching, sparse reconstruction and dense reconstruction using photographs captured from multiple viewpoints (Griwodz et al., 2021). The primary goal of this process is to recover the 3D model of the scene by matching corresponding points and determining the geometric relationships between multiple views.…”