Foveated self-similarity in nonlocal image filtering

Abstract

Nonlocal image …lters suppress noise and other distortions by searching for similar patches at di¤erent locations within the image, thus exploiting the self-similarity present in natural images. This similarity is typically assessed by a windowed distance of the patches pixels. Inspired by the human visual system, we introduce a patch foveation operator and measure patch similarity through a foveated distance, where each patch is blurred with spatially variant point-spread functions having standard deviation increasing with the spatial distance from the patch center. In this way, we install a di¤erent form of self-similarity in images: the foveated self-similarity.We consider the Nonlocal Means algorithm (NL-means) for the removal of additive white Gaussian noise as a simple prototype of nonlocal image …ltering and derive an explicit construction of its corresponding foveation operator, thus yielding the Foveated NL-means algorithm.Our analysis and experimental study show that, to the purpose of image denoising, the foveated self-similarity can be a far more e¤ective regularity assumption than the conventional windowed self-similarity. In the comparison with NL-means, the proposed foveated algorithm achieves a substantial improvement in denoising quality, according to both objective criteria and visual appearance, particularly due to better contrast and sharpness. Moreover, foveation is introduced at a negligible cost in terms of computational complexity.

Epigraphical projection and proximal tools for solving constrained convex optimization problems

“…n). In the present work, we consider the foveated selfsimilarity measure recently introduced in [89], due to its better performance in denoising. This approach can be derived from (42) by setting F ℓ (resp.…”

Anisotropically foveated nonlocal image denoising

“…In [1], patch foveation was proposed as an alternative to windowing in nonlocal imaging. Patch foveation is performed through a foveation operator, which consists in a spatially variant blur where the point-spread functions (PSFs) have bandwidth decreasing with the spatial distance from the patch center.…”

“…In [1] we presented an explicit construction of a foveation operator yielding a foveated distance that, in terms of expectation under zero mean, i.i.d., Gaussian noise, is guaranteed to be equivalent to the distance induced by a given arbitrary windowing kernel. However, in presence of structured differences, such as those arising in the vicinity of edges, the windowed and foveated distances are fundamentally distinct, with the latter providing stronger response.…”

“…The Foveated NL-means [1], which modifies the classical nonlocal means denoising filter (NL-means) [6] by computing the averaging weights based on the foveated patch distance instead of the conventional windowed patch distance, leads to a consistent improvement in the quality of the restored images according to both objective criteria and visual appearance, particularly due to better contrast and sharpness.…”

Affine Invariant Self-similarity for Exemplar-based Inpainting

“…In that work the idea of self-similarity is exploited for direct and non-parametric sampling of the desired texture. The self-similarity prior is one of the most influential ideas underlying the recent progress in image processing and has been effectively used for different image processing and computer vision tasks, such as denoising and other inverse problems (Foi and Boracchi, 2012;Buades et al, 2005;Gilboa and Osher, 2008;Peyré, 2009;Pizarro et al, 2010). It has also found its application to inpainting: the value of each target pixel x in the inpainting domain can be sampled from the known part of the image or even from a vast database of images (Hays and Efros, 2007).…”