An automatic workflow for orientation of historical images with large radiometric and geometric differences

Maiwald, Ferdinand; Maas, Hans‐Gerd

doi:10.1111/phor.12363

Cited by 15 publications

(20 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section intends to give a (nonexhaustive) overview of methods for image retrieval and feature matching in the photogrammetric context of calculating the camera position and orientation of historical images. For a summary of 3D building reconstruction approaches based on the works of [9,10] using historical images refer to [8].…”

Section: Related Workmentioning

confidence: 99%

“…While there is little research focusing on the evaluation of feature matching methods using exclusively historical images [8] a lot of methods are tested on difficult benchmark datasets, e.g., the Photo Tourism dataset [21] and the Aachen Day-Night dataset [22]. With an increasing use of neural networks for feature detection, matching, and scene reconstruction an overview of well-performing, recently published methods using diverse network architectures is given in this section.…”

Section: Feature Detection and Matchingmentioning

confidence: 99%

“…A focus lies especially on the methods D2-net [38] and R2D2 [39], which are used in the photogrammetric workflow and evaluation (see Section 4) of this contribution. D2-Net is a winner of the Image Matching Challenge of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019) and already produced good results in recent research on historical images [8]. The method jointly detects and describes stable feature matches using a CNN which extracts a set of feature maps for every single image.…”

Section: Feature Detection and Matchingmentioning

confidence: 99%

“…Additionally, the distance to the second-closest neighbor has to be smaller than a certain ratio threshold [38] recommend a threshold of 0.90 for the off-the-shelf descriptors and 0.95 for the fine-tuned ones. The application on historical images has shown that this threshold is critical for retrieving robust results [8]. Considering these findings, the threshold was slightly modified using 0.96 for the single-scale and 0.97 for the multiscale approach.…”

Section: Feature Detection and Matchingmentioning

confidence: 99%

“…The retrieval result is directly used as input for an automatic six degrees-of-freedom (6-DoF) pose estimation workflow, i.e., calculation of the camera position and camera orientation up to scale. Especially the automatic detection of distinctive features and their matching are the most difficult tasks when calculating the image pose [8]. This is because the images were mostly made by different people at different times in no photogrammetric context, e.g., a convergent image block with significant overlap between the images.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods

Maiwald

Lehmann

Lazariv

2021

IJGI

Self Cite

View full text Add to dashboard Cite

The idea of virtual time machines in digital environments like hand-held virtual reality or four-dimensional (4D) geographic information systems requires an accurate positioning and orientation of urban historical images. The browsing of large repositories to retrieve historical images and their subsequent precise pose estimation is still a manual and time-consuming process in the field of Cultural Heritage. This contribution presents an end-to-end pipeline from finding relevant images with utilization of content-based image retrieval to photogrammetric pose estimation of large historical terrestrial image datasets. Image retrieval as well as pose estimation are challenging tasks and are subjects of current research. Thereby, research has a strong focus on contemporary images but the methods are not considered for a use on historical image material. The first part of the pipeline comprises the precise selection of many relevant historical images based on a few example images (so called query images) by using content-based image retrieval. Therefore, two different retrieval approaches based on convolutional neural networks (CNN) are tested, evaluated, and compared with conventional metadata search in repositories. Results show that image retrieval approaches outperform the metadata search and are a valuable strategy for finding images of interest. The second part of the pipeline uses techniques of photogrammetry to derive the camera position and orientation of the historical images identified by the image retrieval. Multiple feature matching methods are used on four different datasets, the scene is reconstructed in the Structure-from-Motion software COLMAP, and all experiments are evaluated on a newly generated historical benchmark dataset. A large number of oriented images, as well as low error measures for most of the datasets, show that the workflow can be successfully applied. Finally, the combination of a CNN-based image retrieval and the feature matching methods SuperGlue and DISK show very promising results to realize a fully automated workflow. Such an automated workflow of selection and pose estimation of historical terrestrial images enables the creation of large-scale 4D models.

show abstract

Section: Related Workmentioning

confidence: 99%