Joint Estimation of Camera Pose, Depth, Deblurring, and Super-Resolution from a Blurred Image Sequence

Abstract

The conventional methods for estimating camera poses and scene structures from severely blurry or low resolution images often result in failure. The off-the-shelf deblurring or super-resolution methods may show visually pleasing results. However, applying each technique independently before matching is generally unprofitable because this naïve series of procedures ignores the consistency between images. In this paper, we propose a pioneering unified framework that solves four problems simultaneously, namely, dense depth reconstruction, camera pose estimation, superresolution, and deblurring. By reflecting a physical imaging process, we formulate a cost minimization problem and solve it using an alternating optimization technique. The experimental results on both synthetic and real videos show high-quality depth maps derived from severely degraded images that contrast the failures of naïve multi-view stereo methods. Our proposed method also produces outstanding deblurred and super-resolved images unlike the independent application or combination of conventional video deblurring, super-resolution methods.

Self-Supervised Linear Motion Deblurring

“…Optimization-based methods rely on the blur formation model to recover the latent sharp image by minimizing an energy function [2], [4], [16], [18], [28], [37], [42], e.g., using Gaussian [2], [16], [37] or Poisson [27], [31] likelihood functions in the context of maximum-a-posteriori (MAP) estimation. Depending on the number of input blurry images, additional terms can be formulated by warping the other image(s) to a reference image using either dense flow or by combining relative camera poses with dense depth maps [14], [22]. Due to the nonlinear and ill-posed (in the case of a single image) nature of the problem, prior information on either the motion blur kernel or the latent sharp image must be used to constrain the solution space [2], [4], [16], [16], [28], [37], [37].…”

Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation

“…In [8], a new method is proposed to simultaneously estimate optical flow and tackle the case of general blur by minimization a single non-convex energy function. Park et al [33] estimate camera poses and scene structures from severely blurred images and deblurring by using the motion information.…”

Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More

“…The approaches of [7,25] handle depth estimation with other vision tasks like segmentation and surface normal prediction. The work of [36], on the other hand, proposes a multi-task prediction and distillation network (PAD-Net) structure for joint depth estimation and scene parsing in aim to improve both tasks.…”