Light field image editing by 4D patch synthesis

Abstract

This paper presents a patch-based synthesis framework for lightfield image editing. The core of the proposed method builds upon a patch-based optimization approach. The main contribution of the paper is to extend the versatile patch-based image editing framework to 4D lightfield images and enable many editing applications for them. Specifically, the paper introduces a novel 4D lightfield patch consistency measure for avoiding synthesis of inconsistent patches into the edited lightfield images. Combining with a joint 4D patch search, our method is able to maintain the correlation among views and render a consistent interpretation of the scene. The proposed method offers patch-based solutions to a wide variety of lightfield image editing problems, including inpainting, retargeting and reshuffling.

Light Field Inpainting Propagation via Low Rank Matrix Completion

“…Nevertheless, a comparison between these patch-based methods and our approach can still be led by evaluating the consistency between the inpainted views. Section III of the supplementary materials shows that inconsistencies appear in the results of [18] and [23] (e.g flickering of texture details between views in [18], texture not following the geometry of the object in [23]), while our method does not produce such artifacts.…”

“…Moreover, since the patches are selected from the central view, it is not robust to color and light intensity differences between the views. Chen et al [18] and Zhang et al [20] also proposed more advanced patch-based methods taking advantage of the 4D structure of the light field. Unlike our method, they cannot be used to propagate a view inpainted with an arbitrary technique to the rest of the light field.…”

“…A different approach is used in [18], where the spatial consistency is ensured by minimizing a 4D patch consistency measure extending the bidirectional similarity measure of [19]. A depth estimation of each view of the light field is required to evaluate this measure.…”

Epipolar Plane Diffusion: An Efficient Approach for Light Field Editing

“…For the other views, instead of searching a best matching patch in the known region of the view to inpaint, the patch is searched in the first inpainted view in order to ensure a better consistency across views. A 4D patch-based method is also considered in [9], where consistency is ensured by minimizing a 4D patch bi-directional similarity measure. In the variational framework proposed by [15], various inverse problems are solved, including inpainting.…”

Fast Light Field Inpainting Propagation Using Angular Warping and Color-Guided Disparity Interpolation

“…The authors in [3] extend 2D patch-based methods to 4D patches and ensure consistency by minimizing a 4D patch bi-directional similarity measure. This method progresses patch per patch in a greedy fashion and suffers from a high computational complexity.…”