Confidence Guided Network For Atmospheric Turbulence Mitigation

Abstract

Atmospheric turbulence can adversely affect the quality of images or videos captured by long range imaging systems. Turbulence causes both geometric and blur distortions in images which in turn results in poor performance of the subsequent computer vision algorithms like recognition and detection. Existing methods for atmospheric turbulence mitigation use registration and deconvolution schemes to remove degradations. In this paper, we present a deep learning-based solution in which Effective Nearest Neighbors (ENN) based method is used for registration and an uncertainty-based network is used for restoration. We perform qualitative and quantitative comparisons using synthetic and real-world datasets to show the significance of our work.

Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

“…While it can effectively remove the geometric distortions induced by atmospheric turbulence, its computational cost is comparable to conventional methods, as it needs to repeat the training step for each input image. There are also several works that focus on specific types of images, such as face restoration [36,23,14]. They are usually based on a simplified assumption on atmospheric turbulence where they assume the blur to be spatially invariant.…”

“…If there is only G, the problem is a simple geometric unwrapping. Generic deep-learning models such as [36,23] adopt network architectures for classical restoration problems based on conventional CNNs, which are developed for one type of distortion. Effective, their models treat the problem as…”

“…A more feasible approach here is to leverage a powerful turbulence simulator to synthesize the turbulence effects. Turbulence simulation in the context of deep learning has been reported in [36,23,14]. Their model generates the geometric distortions by repeatedly smoothing a set of random spikes, and the blur is assumed to be spatially invariant Gaussian [13].…”

“…Image processing algorithms for mitigating the atmospheric turbulence effect have been studied for decades [1,21,12,14,40,36,10,20,23,18]. However, many of them have limitations that prohibit them from being launched to practical systems: 1) Many of the existing algorithms [1,21,12,40,1,10] are based on the principle of lucky imaging that requires multiple input frames.…”

“…2) The conventional algorithms are often computationally expensive, making them unsuitable for processing large-scale datasets to meet the need of the latest computer vision systems. 3) Existing deep learning solutions [36,23,14] are not utilizing the physics of the turbulence. Many of them are also tailored to recovering faces instead of generic scenes.…”