Motion Deblurring in the Wild

Abstract

We propose a deep learning approach to remove motion blur from a single image captured in the wild, i.e., in an uncontrolled setting. Thus, we consider motion blur degradations that are due to both camera and object motion, and by occlusion and coming into view of objects. In this scenario, a modelbased approach would require a very large set of parameters, whose fitting is a challenge on its own. Hence, we take a data-driven approach and design both a novel convolutional neural network architecture and a dataset for blurry images with ground truth. The network produces directly the sharp image as output and is built into three pyramid stages, which allow to remove blur gradually from a small amount, at the lowest scale, to the full amount, at the scale of the input image. To obtain corresponding blurry and sharp image pairs, we use videos from a high frame-rate video camera. For each small video clip we select the central frame as the sharp image and use the frame average as the corresponding blurred image. Finally, to ensure that the averaging process is a sufficient approximation to real blurry images we estimate optical flow and select frames with pixel displacements smaller than a pixel. We demonstrate state of the art performance on datasets with both synthetic and real images.

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

“…There is no easy method to obtain image pairs of corresponding sharp and blurred images for training.A typical approach to obtain image pairs for training is to use a high frame-rate camera to simulate blur using average of sharp frames from video [27,25]. It allows to create realistic blurred images but limits the image space only to scenes present in taken videos and makes it complicated to scale the dataset.…”

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

“…-that well describes what we have done to DeblurGAN, too; although we consider DeblurGAN-v2 a non-incremental upgrade of DeblurGAN, with significant performance & efficiency improvements. explored to restore a clean image from the blurry input directly, e.g., [33,35]. The latest work by Tao et al [45] extended the Multi-Scale CNN from [33] to a Scale-Recurrent CNN for blind image deblurring, with impressive results.…”

Spatio-Temporal Filter Adaptive Network for Video Deblurring

“…However, the predicted kernels are line-shaped which are inaccurate in some scenarios, and time-consuming conventional non-blind deblurring [46] is generally required to restore the sharp image. More recently, many end-to-end CNN models [38,44,17,23,26] have also been proposed for image deblurring. To obtain a large receptive field for handling the large blur, the multi-scale strategy is used in [38,23].…”