Domain transform for edge-aware image and video processing

Abstract

a) Photograph (b) Edge-aware smoothing (c) Detail enhancement (d) Stylization (e) Recoloring (f) Pencil drawing (g) Depth-of-field Figure 1: A variety of effects illustrating the versatility of our domain transform and edge-preserving filters applied to the photograph in (a). AbstractWe present a new approach for performing high-quality edgepreserving filtering of images and videos in real time. Our solution is based on a transform that defines an isometry between curves on the 2D image manifold in 5D and the real line. This transform preserves the geodesic distance between points on these curves, adaptively warping the input signal so that 1D edge-preserving filtering can be efficiently performed in linear time. We demonstrate three realizations of 1D edge-preserving filters, show how to produce high-quality 2D edge-preserving filters by iterating 1D-filtering operations, and empirically analyze the convergence of this process. Our approach has several desirable features: the use of 1D operations leads to considerable speedups over existing techniques and potential memory savings; its computational cost is not affected by the choice of the filter parameters; and it is the first edge-preserving filter to work on color images at arbitrary scales in real time, without resorting to subsampling or quantization. We demonstrate the versatility of our domain transform and edge-preserving filters on several real-time image and video processing tasks including edgepreserving filtering, depth-of-field effects, stylization, recoloring, colorization, detail enhancement, and tone mapping.

Fast Edge-Aware Processing via First Order Proximal Approximation

“…(a) Input (b) BLF [1] (c) NCF [11] (d) TV [20] (e) Extrema [27] (f) GF [28] (g) WLS [16] (h) L0 [21] (i) Proposed [21]). The proposed method produces high-quality smoothing while being flexible and computationally efficient.…”

“…As can be seen, the proposed smoothing method produces comparable smoothing quality at lower computational cost. Note that the results presented here are produced (a) Input (b) Fast BLF [7] (c) RF [11] (d) AdaptM [12] (e) Proposed (1 iteration with the original Local Laplacian Filter implementation that is computationally very demanding. The Fast Local Laplacian Filters method [18] that aims at making the Local Laplacian Filter faster is only an approximation and does not produce comparable smoothing quality.…”

“…A naive implementation of this filter is computationally demanding as it operates in a high-dimension space. Many researchers tried to boost this filter or at least simulate bilaterallike results by either using linear interpolation [3], optimized data structures such as the bilateral grid [4], reformulation and downsampling [5], [6], constant time spatial filters decomposition [7], Gaussian KDtrees [8], Supports Vector Machines regression [9], recursive implementation [10], dimensionality reduction [11], adaptive manifolds [12], among others. The bilateral filter is the building block of a wide range of applications such as HDR tone-mapping [3], nonphotorealistic rendering [13], upsampling [14] and non-blind image deconvolution [15].…”

“…The method was mainly used for denoising and its use for computer graphics applications was limited. The method of Farbman et (c) BLF [1] (d) Domain Transform [11] (e) WLS [16] (f) Proposed Fig. 1: Image smoothing of the noisy input (a) with various edge-aware filters (image from [16]).…”

“…The bilateral filter (c) is not able to smooth large-scale details and introduces strong edge blurring. The domain transform method [11], which is a fast method based on local filtering, is also not able either to correctly smooth the image. Similar to the bilateral filter, the result (d) contains large-scale noise and blurred edges.…”