Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework

Chen, Jie; Hou, Junhui; Chau, Lap-Pui

doi:10.1109/lsp.2018.2861212

Cited by 49 publications

(13 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further evaluate the preservation of the light field parallax structure quantitatively, we compared the light field parallax edge precision-reall (PR) curves (Chen, Hou, and Chau 2018) of the angularly super-resolved light fields, and Fig. 6 shows the results.…”

Section: Resultsmentioning

confidence: 99%

Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

Jin

Hou

Yuan

et al. 2020

AAAI

Self Cite

101

View full text Add to dashboard Cite

The acquisition of light field images with high angular resolution is costly. Although many methods have been proposed to improve the angular resolution of a sparsely-sampled light field, they always focus on the light field with a small baseline, which is captured by a consumer light field camera. By making full use of the intrinsic geometry information of light fields, in this paper we propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our model consists of two learnable modules and a physically-based module. Specifically, it includes a depth estimation module for explicitly modeling the scene geometry, a physically-based warping for novel views synthesis, and a light field blending module specifically designed for light field reconstruction. Moreover, we introduce a novel loss function to promote the preservation of the light field parallax structure. Experimental results over various light field datasets including large baseline light field images demonstrate the significant superiority of our method when compared with state-of-the-art ones, i.e., our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48×. In addition, our method preserves the light field parallax structure better.

show abstract

Section: Resultsmentioning

confidence: 99%

Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

Jin

Hou

Yuan

et al. 2020

AAAI

Self Cite

101

View full text Add to dashboard Cite

show abstract

“…Single view NLM [8] BM3D [9,10] SGN [11] Real image denoising [12] Universal denoising networks [13] FFDNet [14] Video VBM3D [15] VBM5D [16] With optical flow estimation [17] Frame-to-frame training [18] SALT [19] Multi-view NLM with PS [20] With occlusion handling [21] MVCNN [22] LFBM5D [24,25] APA [26] An epipolar plane image (EPI) is a typical means of expressing the structure of multiviews. In an EPI, the relative motion between cameras or objects is expressed as lines that have different slopes depending on the depth.…”

Section: Target Applications Denoising Algorithmsmentioning

confidence: 99%

“…LFBM5D has also been applied to remove artifacts in super resolution studies of LF images [ 25 ]. Furthermore, Chen et al used a CNN to remove noise from LF images based on an anisotropic parallax analysis (APA) [ 26 ]. For real-time LF denoising, 4-dimensional (4D) linear and shift-invariant hyper fan filters have been proposed and implemented in hardware [ 27 ].…”

Section: Introductionmentioning

confidence: 99%

Improved Light Field Compression Efficiency through BM3D-Based Denoising Using Inter-View Correlation

Jin

Rhee

2021

Sensors

View full text Add to dashboard Cite

Multi-view or light field images have recently gained much attraction from academic and commercial fields to create breakthroughs that go beyond simple video-watching experiences. Immersive virtual reality is an important example. High image quality is essential in systems with a near-eye display device. The compression efficiency is also critical because a large amount of multi-view data needs to be stored and transferred. However, noise can be easily generated during image capturing, and these noisy images severely deteriorate both the quality of experience and the compression efficiency. Therefore, denoising is a prerequisite to produce multi-view-based image contents. In this paper, the structural characteristics of linear multi-view images are fully utilized to increase the denoising speed and performance as well as to improve the compression efficiency. Assuming the sequential processes of denoising and compression, multi-view geometry-based denoising is performed keeping the temporal correlation among views. Experimental results show the proposed scheme significantly improves the compression efficiency of denoised views up to 76.05%, maintaining good denoising quality compared to the popular conventional denoise algorithms.

show abstract

“…Recently, the application of a deep neural network to multi-view denoising has attracted researchers’ attention. Chen et al [39] proposed a light field denoising framework based on anisotropic parallax analysis. In this work, two convolutional neural networks (CNN) will jointly predict parallax information and restore non-Lambertian variations to each view.…”

Section: Related Workmentioning

confidence: 99%

“…This method, though simple and convenient, does not exploit the redundant image information that exists in multiple views capturing the same scene. Numerous studies [32,33,34,35,36,37,38,39,40,41,42,43,44] have demonstrated that inter-view redundant information is essential for recovering the original image details without creating undesirable over-smoothing artifacts that are common in single image denoising. Therefore, we believe that the denoising performance of CNN model can be further enhanced if inter-view information, in addition to intra-view information, is taken into consideration.…”

Section: The Proposed Denoising Networkmentioning

confidence: 99%

Multi-View Image Denoising Using Convolutional Neural Network

Zhou

Jiang

2019

Sensors

View full text Add to dashboard Cite

In this paper, we propose a novel multi-view image denoising algorithm based on convolutional neural network (MVCNN). Multi-view images are arranged into 3D focus image stacks (3DFIS) according to different disparities. The MVCNN is trained to process each 3DFIS and generate a denoised image stack that contains the recovered image information for regions of particular disparities. The denoised image stacks are then fused together to produce a denoised target view image using the estimated disparity map. Different from conventional multi-view denoising approaches that group similar patches first and then perform denoising on those patches, our CNN-based algorithm saves the effort of exhaustive patch searching and greatly reduces the computational time. In the proposed MVCNN, residual learning and batch normalization strategies are also used to enhance the denoising performance and accelerate the training process. Compared with the state-of-the-art single image and multi-view denoising algorithms, experiments show that the proposed CNN-based algorithm is a highly effective and efficient method in Gaussian denoising of multi-view images.

show abstract

Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework

Cited by 49 publications

References 28 publications

Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

Improved Light Field Compression Efficiency through BM3D-Based Denoising Using Inter-View Correlation

Multi-View Image Denoising Using Convolutional Neural Network

Contact Info

Product

Resources

About