Patch Complexity, Finite Pixel Correlations and Optimal Denoising

Abstract

Abstract. Image restoration tasks are ill-posed problems, typically solved with priors. Since the optimal prior is the exact unknown density of natural images, actual priors are only approximate and typically restricted to small patches. This raises several questions: How much may we hope to improve current restoration results with future sophisticated algorithms? And more fundamentally, even with perfect knowledge of natural image statistics, what is the inherent ambiguity of the problem? In addition, since most current methods are limited to finite support patches or kernels, what is the relation between the patch complexity of natural images, patch size, and restoration errors? Focusing on image denoising, we make several contributions. First, in light of computational constraints, we study the relation between denoising gain and sample size requirements in a non parametric approach. We present a law of diminishing return, namely that with increasing patch size, rare patches not only require a much larger dataset, but also gain little from it. This result suggests novel adaptive variable-sized patch schemes for denoising. Second, we study absolute denoising limits, regardless of the algorithm used, and the converge rate to them as a function of patch size. Scale invariance of natural images plays a key role here and implies both a strictly positive lower bound on denoising and a power law convergence. Extrapolating this parametric law gives a ballpark estimate of the best achievable denoising, suggesting that some improvement, although modest, is still possible.

Learning How to Combine Internal and External Denoising Methods

“…Tab. 3 compares our results against recently estimated bounds for image denoising [9]. Our proposed method that combines BM3D and MLP gets much closer to the bounds (last row).…”

“…Instead we propose a learning based approach using a neural network, that automatically combines denoising results from an internal and from an external method. This approach outperforms both other combination methods and state-of-the-art stand-alone image denoising methods, hereby further closing the gap to the theoretically achievable performance limits of denoising [9]. Our denoising results can be replicated with a publicly available toolbox 1 .…”

“…This has naturally led to the question if there are inherent limits to how well it is possible to denoise, and if so, whether current methods are approaching these limits. Even though the approaches taken are different [5,9], the consensus is that current methods are indeed not far away from theoretical limits, especially at lower noise levels. Contributions: In this paper, we will study the performance of internal and external methods across an image database and patch-wise across single images.…”

Mapping-Based Image Diffusion

“…Rather than computing new pixel-centres from matched neighbourhoods as in NLM, BM3D considers patches and adaptively denoises a group of local neighbourhoods. In this context we mention Levin et al [44] who studied image denoising quality w.r.t. patch-size and patch-complexity in view of natural image statistics, they conclude: while the restoration of homogeneous regions is straightforward by increasing the sample size, an increase of patch-size for textured regions has limited impact on the restoration quality as pixels in these regions are weakly correlated.…”

Performance Limits for Computational Photography

“…However, all the above works, do not analyze the performance of CI systems when a signal prior is used for demultiplexing. A few recent papers have analyzed the fundamental performance limits of image denoising in the presence of image priors [2,10]. A similar approach was used by Mitra et al [13] to extend this analysis to general framework for analyzing computational imaging systems.…”