A Novel Method for the Deblurring of Photogrammetric Images Using Conditional Generative Adversarial Networks

Abstract

The visual data acquisition from small unmanned aerial vehicles (UAVs) may encounter a situation in which blur appears on the images. Image blurring caused by camera motion during exposure significantly impacts the images interpretation quality and consequently the quality of photogrammetric products. On blurred images, it is difficult to visually locate ground control points, and the number of identified feature points decreases rapidly together with an increasing blur kernel. The nature of blur can be non-uniform, which makes it hard to forecast for traditional deblurring methods. Due to the above, the author of this publication concluded that the neural methods developed in recent years were able to eliminate blur on UAV images with an unpredictable or highly variable blur nature. In this research, a new, rapid method based on generative adversarial networks (GANs) was applied for deblurring. A data set for neural network training was developed based on real aerial images collected over the last few years. More than 20 full sets of photogrammetric products were developed, including point clouds, orthoimages and digital surface models. The sets were generated from both blurred and deblurred images using the presented method. The results presented in the publication show that the method for improving blurred photo quality significantly contributed to an improvement in the general quality of typical photogrammetric products. The geometric accuracy of the products generated from deblurred photos was maintained despite the rising blur kernel. The quality of textures and input photos was increased. This research proves that the developed method based on neural networks can be used for deblur, even in highly blurred images, and it significantly increases the final geometric quality of the photogrammetric products. In practical cases, it will be possible to implement an additional feature in the photogrammetric software, which will eliminate unwanted blur and allow one to use almost all blurred images in the modelling process.

Histogram Adjustment of Images for Improving Photogrammetric Reconstruction

“…Some methods have already been used to improve photo quality in the photogrammetric reconstruction pipeline. Many of these are associated with the deblurring of images [ 27 , 28 , 29 ]. The literature also includes studies on the use of using polarizing filters or High Dynamic Range (HDR) images, [ 25 ] decolorization [ 30 ] or novel techniques for the conversion of the color signal into a grayscale [ 31 ].…”

“…Another result of evaluating the impact of image processing for photogrammetry were presented by Feng et al [ 33 ]. Presently, neural networks, especially the ones based on deep models, are used to improve photogrammetric quality [ 29 , 34 , 35 , 36 ].…”

Polymodal Method of Improving the Quality of Photogrammetric Images and Models

“…A factor is defined as a certain process activity impacting quality. The quality of the photogrammetric process can be built on three main pillars (Figure 1) [6]:…”

“…Neural networks, especially the ones based on deep models, are widely used also in photogrammetry. Such cases include, for example, the following methods of improving photogrammetric processing quality [6,7,[35][36][37][38][39].…”

Observability Analysis and Navigation Filter Optimization of High-Orbit Satellite Navigation System Based on GNSS

“…The position and velocity estimation of the users can be obtained based on GNSS by measuring the pseudo-range and Doppler frequency shift between the users and the navigation satellites. It has the advantages of comprehensive navigation information, high navigation accuracy, and simple measurement equipment [4], and has been widely applied in reality [5][6][7][8][9][10]. The navigation of the high-orbit satellites based on GNSS is one of the important research areas in the high-orbit satellite navigation technology.…”