Blind Geometric Distortion Correction on Images Through Deep Learning

Abstract

We propose the first general framework to automatically correct different types of geometric distortion in a single input image. Our proposed method employs convolutional neural networks (CNNs) trained by using a large synthetic distortion dataset to predict the displacement field between distorted images and corrected images. A model fitting method uses the CNN output to estimate the distortion parameters, achieving a more accurate prediction. The final corrected image is generated based on the predicted flow using an efficient, high-quality resampling method. Experimental results demonstrate that our algorithm outperforms traditional correction methods, and allows for interesting applications such as distortion transfer, distortion exaggeration, and co-occurring distortion correction. Multi-model distortion estimationThe GeoNetS network is only able to capture a specific distortion type with a distortion model at a time. For a new type, the entire network has to be retrained. Furthermore, the distortion type and model can be unknown in some cases. In view of these limitations, we designed a second network for multi-model distortion estimation. However, since the distortion model and the parameters ρ β can vary drastically across types, it is impossible to train a multimodel network with the model constraints. We train a network to regress the distortion flow without model constraints and at the same time classify the distortion type. The network is illustrated in Figure 3. The multi-model network N parameterized by θ is jointly trained for two tasks. The first task estimates the distortion flow, learning the mapping from the image domain I to the flow domain F. The second task classifies the distortion type, learning the mapping from image domain I to type domain T .Architecture The entire network adopts an encoderdecoder structure, which includes an encoder part, a decoder part, and a classification part. The input image is fed

Blind First-Order Perspective Distortion Correction Using Parallel Convolutional Neural Networks

“…Blind distortion correction is an ill-posed problem. Therefore, learning-based methods using only a single distorted image are being pursued [ 10 , 11 , 12 , 42 , 43 , 44 , 45 , 46 ]. Deep learning for correcting documents were proposed recently [ 12 , 44 , 45 , 46 ] which implements convolutional neural networks, encoder-decoders, and U-net-based architectures [ 47 ].…”

“…Work on correcting portrait images used an encoder-decoder architecture [ 10 ]. The encoder-decoder architecture proposed by Li et al [ 11 ] aims to correct real-world images by predicting the distortion flow and further refining the correction by iterative resampling, which is a predecessor of our work. Instead of using a multi-model network for predicting the distortion flow, we used multiple convolutional neural networks (CNN) that run in parallel to predict the transformation matrix.…”

“…Instead of using a multi-model network for predicting the distortion flow, we used multiple convolutional neural networks (CNN) that run in parallel to predict the transformation matrix. Our network is trained purely for correcting perspective distortions, unlike the work of Li et al [ 11 ] that correct a wide range of distortion types, such as barrel and pincushion distortions. Furthermore, our results outperforms the method of Li et al [ 11 ], which occasionally generates incorrect rectification of images even on the dataset they have used for training (Places-365 dataset [ 48 ]).…”

“…Our network is trained purely for correcting perspective distortions, unlike the work of Li et al [ 11 ] that correct a wide range of distortion types, such as barrel and pincushion distortions. Furthermore, our results outperforms the method of Li et al [ 11 ], which occasionally generates incorrect rectification of images even on the dataset they have used for training (Places-365 dataset [ 48 ]). To some extent, our network properly generalizes to this dataset despite being trained on KITTI [ 49 ] images.…”

“…Since each network only produces a certain element pair in , it provides a more straightforward approach by simply applying a transformation to correct the image, unlike generating corrected images using GAN or encoder-decoder architectures, which are more difficult to train and prone to instability such as mode collapse [ 8 , 9 ]. While our method requires multiple networks to correct an image, this approach results in a smaller computational footprint because each CNN has a fewer number of hidden layers compared to other architectures [ 10 , 11 , 12 ] involving deep networks.…”