Lattice-Point Mutually Guided Ground-to-Aerial Feature Matching for Urban Scene Images

Abstract

Ground-to-aerial feature matching bridges information from cross-view images, which enables optimized urban applications, e.g., pixel-level geolocating and complete urban 3D reconstruction. However, urban ground and aerial images typically suffer from drastic changes in viewpoint, scale, and illumination, together with repetitive patterns. Thus, direct matching of local features between ground and aerial images is particularly difficult because of the low similarity of local descriptors and high ambiguity in true-false match discrimination. For this challenging task, we propose a novel lattice-point mutually guided matching (LPMG) method in this paper. We specifically address two key issues: 1) reducing descriptor variance and 2) enhancing true-false match discriminability. The former is solved by recovering the geometry and appearance of the underlying image region in 3D through automatic view rectification on ground and aerial images. The latter is circumvented by replacing the conventional mismatch removal with an LPMG strategy. In this strategy, the topology structure of repeated façade elements (i.e., lattice), and the high reliable point matching seeds, are first extracted from the rectified ground and aerial images. Then, the point matching seeds guide the self-similar lattice tiles from two views to be precisely aligned, thereby estimating an accurate transformation model from lattice tile correspondences. Finally, the estimated model powerfully supervises the differentiation of true and false matches from the entire putative match set. Extensive experiments conducted on several datasets show that our method can obtain a considerable number of nearly pure correct matches from urban ground and aerial images, significantly outperforming those existing methods.

Learning Dense Consistent Features for Aerial-to-Ground Structure-From-Motion

“…Wu et al [9] corrected the images by projecting them onto virtual planes generated from dense point cloud data. Zheng et al [10] first extracted the building façade structures in the aerial and ground images using the local consistency of features, then verifies images based on the transform-invariant low-rank texture [25], and finally achieves aerial-ground image matching by a mutually supervised manner between extracted façade grid structures and matched seeds. However, the viewpoint rectification approach suffers from the following two problems: 1) even if the viewpoint of both images is successfully verified, the information difference between the aerial and ground images is not handled, and the further scale change brought by the viewpoint rectification will exacerbate the difficulty of feature matching and 2) real scenes are often complex, and it is difficult to guarantee the extraction of planar structures, especially when the building facades are nonplanar.…”

“…In order to address the problem of 2-D/3-D feature-based registration difficulties between aerial and ground images or models, some researchers have conducted pioneering studies using viewpoint rectification [9], [10] or rendering/synthesis [1], [2], [11] to improve feature matching performance and achieve registration of aerial and ground models. Viewpoint rectification works focus on identifying view-independent planar structures (such as ground and building facades) in the scene to correct the aerial and ground images to a normalized viewpoint, thus reducing the differences in viewpoint between the aerial and ground This work is licensed under a Creative Commons Attribution 4.0 License.…”