Stereo Video Deblurring

Abstract

Abstract. Videos acquired in low-light conditions often exhibit motion blur, which depends on the motion of the objects relative to the camera. This is not only visually unpleasing, but can hamper further processing. With this paper we are the first to show how the availability of stereo video can aid the challenging video deblurring task. We leverage 3D scene flow, which can be estimated robustly even under adverse conditions. We go beyond simply determining the object motion in two ways: First, we show how a piecewise rigid 3D scene flow representation allows to induce accurate blur kernels via local homographies. Second, we exploit the estimated motion boundaries of the 3D scene flow to mitigate ringing artifacts using an iterative weighting scheme. Being aware of 3D object motion, our approach can deal robustly with an arbitrary number of independently moving objects. We demonstrate its benefit over state-of-the-art video deblurring using quantitative and qualitative experiments on rendered scenes and real videos.

Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation

“…(a) Original Blurred image (b) Kim and Lee [8] (c) Sellent et al [9] (d) Ours Compared with these monocular and stereo deblurring methods, our method achieves more accurate blur kernel estimation.…”

“…Image deblurring (even under stereo configuration) is generally an ill-posed problem, thus certain assumptions or additional constraints are required to regularize the solution space. Numerous methods have been proposed to address the problem [8], [9], [10], [20], [13], [5], [18], [21], [22]. As per the system configuration, the methods can be roughly categorized into two groups: monocular based approaches and binocular or multi-view based approaches.…”

“…Xu et al [35] inferred depth from two blurred images captured by a stereo camera and proposed a hierarchical estimation framework to remove motion blur caused by the in-plane translation. Sellent et al [9] proposed a video deblurring technique based on a stereo video, where 3D scene flow is estimated from the blurred images using a piecewise rigid 3D scene representation. Along the same line, Ren et al [21] proposed an algorithm where accurate semantic segmentation is known.…”

“…Early deblurring methods mainly focus on the blur caused by camera shake in constant depth or static scenes with moving objects [13], [14]. In this work, we focus on a more generalized motion blur Liyuan (a) Blurred image (b) Initial segmentation (c) Kim and Lee [8] (d) Our flow (e) Kim and Lee [8] (f) Sellent et al [9] (g) Pan et al [10] (h) Ground-truth (i) Our segmentation (j) Our deblurred result (e) Deblurring results by [8]. (f) Stereo deblurring results by [9] which uses [11] to estimate scene flow.…”

“…For a scenario where both camera motion and multiple moving objects exist, the blur kernel is, in principle, defined for each pixel individually. Recently, several researchers have studied to handle the blurred images with spatially-variant blur [8], [9], [10] which uses accurate motion estimation to model the blur kernel. The phenomenon around motion and blur can be viewed as a chicken-egg problem: effective motion blur removal requires accurate motion estimation.…”