Joint Demosaicing and Denoising via Learned Nonparametric Random Fields

Abstract

Abstract-We introduce a machine learning approach to demosaicing, the reconstruction of color images from incomplete color filter array samples. There are two challenges to overcome by a demosaicing method: first, it needs to model and respect the statistics of natural images in order to reconstruct natural looking images; second, it needs to be able to perform well in the presence of noise. To facilitate an objective assessment of current methods we introduce a public ground truth data set of natural images suitable for research in image demosaicing and denoising. We then use this large data set to develop a machine learning approach to demosaicing. Our proposed method addresses both demosaicing challenges by learning a statistical model of images and noise from hundreds of natural images. The resulting model performs simultaneous demosaicing and denoising. We show that the machine learning approach has a number of benefits: 1. the model is trained to directly optimize a user-specified performance measure such as peak signal-to-noise ratio (PSNR) or structural similarity (SSIM), 2. we can handle novel color filter array layouts by retraining the model on such layouts, 3. it outperforms the previous state-of-the-art, in some setups by 0.7dB PSNR, faithfully reconstructing edges, textures, and smooth areas. Our results demonstrate that in demosaicing and related imaging applications, discriminatively trained machine learning models have the potential for peak performance at comparatively low engineering effort.

Lossless Compression of Color Filter Array Mosaic Images With Visualization via JPEG 2000

“…Several demosaicking algorithms have been recently proposed [4], [6]- [8], [10]- [14] and very detailed reviews of the state of the art can be found in the literature [8], [37]. An alternative approach to demosaicking was described in [23].…”

“…For example, some authors employ a pipeline in which interpolation occurs after white balancing [10], while some software tools merge the green components after removing the chromatic aberrations [16].…”

“…8h) are due to the more sophisticated interpolation algorithm employed by the latter [2], which removes chromatic aberrations (due to the camera) and produces sharper edges. To the best of our knowledge, existing chromatic aberration techniques are not reversible [4]- [10], and thus, not admitted by the framework of the JPEG 2000 standard. Despite the fact that not all chromatic aberrations are removed by the proposed system, we emphasize again that the system provides automatic developing and rendering of mosaic images using only a standard viewer.…”

“…These simulated mosaic images were obtained by downsampling fully developed 8-bit sRGB images [20]- [22], [26]- [28], [35], [36], [45]. This approach is reasonable, but not ideal since the development process described in Section II cannot be inverted perfectly [10], [30]. Thus, simulated mosaic images may contain statistically significant differences from real mosaic images.…”

Extended Shearlet HMT Model-Based Image Denoising Using BKF Distribution

“…Although seemingly very different, they all share the same property: to keep the meaningful edges and remove less meaningful ones. The existing image denoising work can be roughly divided into Nonlocal Methods [3][4][5], Random Fields [6][7][8], Bilateral Filtering [9][10][11], Anisotropic Diffusion [12,13], and Statistical Model [15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32]. In addition, many authors have developed image denoising algorithms based on support vector machine (SVM) classification [14].…”