Robust Guided Image Filtering Using Nonconvex Potentials

Abstract

Filtering images using a guidance signal, a process called guided or joint image filtering, has been used in various tasks in computer vision and computational photography, particularly for noise reduction and joint upsampling. This uses an additional guidance signal as a structure prior, and transfers the structure of the guidance signal to an input image, restoring noisy or altered image structure. The main drawbacks of such a data-dependent framework are that it does not consider structural differences between guidance and input images, and that it is not robust to outliers. We propose a novel SD (for static/dynamic) filter to address these problems in a unified framework, and jointly leverage structural information from guidance and input images. Guided image filtering is formulated as a nonconvex optimization problem, which is solved by the majorize-minimization algorithm. The proposed algorithm converges quickly while guaranteeing a local minimum. The SD filter effectively controls the underlying image structure at different scales, and can handle a variety of types of data from different sensors. It is robust to outliers and other artifacts such as gradient reversal and global intensity shift, and has good edge-preserving smoothing properties. We demonstrate the flexibility and effectiveness of the proposed SD filter in a variety of applications, including depth upsampling, scale-space filtering, texture removal, flash/non-flash denoising, and RGB/NIR denoising.

Robust semi-automatic 2D-to-3D image conversion via residual-driven optimization

“…β is set to 100 for RGBZ datasets and 50 for Middlebury datasets. Results of the proposed method are compared to the state-of-theart: RW [17], hybrid GC and RW (HGR) [18], nonlocal RW (NRW) [22], optimization (OPT) [24], OCP [31], SDF [33], and 1 [34]. Note that OCP originally aims for interactive segmentation, and this paper applies it to 2D-to-3D conversion by replacing the confidence in Formula (3) with the aggregation of the OCPs in a local neighborhood.…”

“…Recently, Ham et al [33] proposed a static dynamic filter (SDF) to reduce artifacts caused by structural differences between guidance and input signals. Although SDF [33] can handle differences in structure, it is not robust to outliers introduced by cross-boundary scribbles.…”

“…Although SDF [33] can handle differences in structure, it is not robust to outliers introduced by cross-boundary scribbles. Yuan et al [34] proposed an 1 optimization method to remove user erroneous scribbles.…”

Fast Color-guided Depth Denoising for RGB-D Images by Graph Filtering

“…Color-guided methods for depth image denoising in [6]- [8] formulate the problem as an optimization problem and utilize the guidelines from the color images by modifying the regularization. However, these methods are likely to be intractable and time-consuming to implement.…”

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

“…where we denote by N (p) a set of neighbors (defined on a discrete regular grid) near the position p. The filter kernel W is a function of the guidance image g (Park et al 2011;Ferstl et al 2013;Kopf et al 2007;He et al 2013), the target image f itself (Zhang et al 2014;Tomasi and Manduchi 1998), or both (Li et al 2016;Ham et al 2018), normalized so that q∈N (p) W pq (f, g) = 1.…”