Blind Restoration of Images Distorted by Atmospheric Turbulence Based on Deep Transfer Learning

Abstract

Removing space-time varying blur and geometric distortions simultaneously from an image is a challenging task. Recent methods (including physical-based methods or learning-based methods) commonly default the turbulence-degraded operator as a fixed convolution operator. Obviously, the assumption does not hold in practice. According to the situation that the real turbulence distorted operator has double uncertainty in space and time dimensions, this paper reports a novel deep transfer learning (DTL) network framework to address this problem. Concretely, the training process of the proposed approach contains two stages. In the first stage, the GoPro Dataset was used to pre-train the Network D1 and freeze the bottom weight parameters of the model; in the second stage, a small amount of the Hot-Air Dataset was employed for finetuning the last two layers of the network. Furthermore, residual fast Fourier transform with convolution block (Res FFT-Conv Block) was introduced to integrate both low-frequency and high-frequency residual information. Subsequently, extensive experiments were carried out with multiple real-world degraded datasets by implementing the proposed method and four existing state-of-the-art methods. In contrast, the proposed method demonstrates a significant improvement over the four reported methods in terms of alleviating the blur and distortions, as well as improving the visual quality.

High-Resolution Image Processing of Probe-Based Confocal Laser Endomicroscopy Based on Multistage Neural Networks and Cross-Channel Attention Module

“…Deep convolutional neural networks (CNNs) are a new type of network and have the ability to automatically effectively learn the nonlinear relationship between input and output [8][9][10]. Recent studies have shown that deep convolutional neural networks (CNNs) are widely used in image restoration tasks [11][12][13][14][15] in computer vision. For instance, Liu et al [16] proposed a novel approach that merged multilevel image restoration with the pix2pix generative adversarial network architecture within the lensless imaging sphere, which greatly improved image recovery quality in lensless systems.…”

Automated Nighttime Cloud Detection Using Keograms When Aurora Is Present

“…In this work, detecting the presence of a filtering function due to atmospheric scattering is the focus, rather than defining the precise form of the function. The concept of atmospheric filtering is mentioned by Guo et al (2022) who investigated neural network-based restoration of images distorted by atmospheric turbulence. We do not need to go so far as to restore images blurred by clouds in a large multi-year database of auroral imagery, but we can leverage the effect of clouds on keograms to determine their presence.…”