Real-Time Light Field Denoising Using a Novel Linear 4-D Hyperfan Filter

Premaratne, Sanduni U.; Liyanage, Namalka; Edussooriya, Chamira U. S.; Wijenayake, Chamith

doi:10.1109/tcsi.2020.2981387

Cited by 17 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, 3D LF considers HPO and incurs the problem of insufficient light rays in the vertical direction. 3D/4D LF is the most basic form and is used in various applications, such as depth estimations [24], [27], super-resolution [23], [25], [26], and denoising [28], [29].…”

Section: A 3d/4d Light Fieldmentioning

confidence: 99%

Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

2023

View full text Add to dashboard Cite

Conventional captured-image-based virtual reality (VR) systems hardly support free movement of the viewpoint, greatly reducing the user's sense of reality. A light field (LF), which generates a novel view at a free viewpoint through a combination of light rays, is a suitable approach for increasing the degree of freedom (DoF) in captured-image-based VR. Previous LF-based VR systems have increased LF dimensions or used various acquisition layouts to obtain as many light rays as possible and thereby expand the renderable view range. However, these attempts are costly and time-consuming. This paper enables 5-DoF outcomes from a low-dimensional LF by increasing the field-of-view during light ray acquisition. The method starts with the simple idea that if a three-dimensional (3D) LF is constructed with 360 • images that can capture all directions, the renderable range can be widened in a cost-efficient manner. However, using light rays with a large incident angle in a 3D LF can cause visual artifacts. To cope with this problem, this paper initially defines and analyzes the visual artifacts. While these errors were simply ignored in the past, they become particularly noticeable when using light rays with large incident angles. Thus, a careful analysis of the errors and their impacts is required. Additionally, this paper proposes two schemes to deal with visual artifacts based on the error analysis: a horizontal LF and vertical depth (HLF-VD) view generation scheme that complements the structural limitation of 3D LFs and can address all types of errors, and a hybrid 3D LF connection scheme that effectively reduces connection errors by controlling the connection position of neighboring 3D LFs. The proposed system is implemented and verified in various real-world scenarios, and the experimental results show that the user's movement is naturally reflected at a level similar to that of high-dimensional LF, with minimal visual artifacts. INDEX TERMSFree viewpoint, Image-based rendering, Light field, View navigation, Virtual reality I. INTRODUCTIONA few years ago, virtual reality (VR) technology gained significant attention. However, problems such as insufficient technical maturity, lack of VR content, and motion sickness drastically reduced interest in VR. Nevertheless, with the onset of the COVID-19 pandemic, VR technology is experiencing a resurgence. The pandemic has changed the way people interact, and to avoid direct contact, telecommut-ing, video conferencing, and video education have become increasingly popular. VR technology can provide an allencompassing solution in the COVID-19 era as it allows people to work, meet, and play in a virtual space separated from the real world. However, the level of VR technology is still not as advanced as people expect, and there are numerous challenges that need to be addressed.Computer graphic-based virtual reality (CG-based VR)

show abstract

Section: A 3d/4d Light Fieldmentioning

confidence: 99%

Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

2023

View full text Add to dashboard Cite

show abstract

“…They employed a 4-D linear filter having a hyperfanshaped passband to achieve volumetric refocusing. Dansereau et al [17] and Premaratne et al [18] employed similar 4-D linear hyperfan filters for LF denoising. In [19], Premaratne et al proposed a 4-D sparse finite-extent impulse response (FIR) filter having a hyperfan-shaped passband, designed using the windowing technique and hard thresholding, for volumetric S. S. Jayaweera and C. U. S. Edussooriya are with the Department of Electronic and Telecommunication Engineering, University of Moratuwa, Moratuwa 10400, Sri Lanka (e-mails: sakilaj@uom.lk; chamira@uom.lk).…”

Section: Introductionmentioning

confidence: 99%

Multi-Volumetric Refocusing of Light Fields

Jayaweera

Edussooriya

Wijenayake

et al. 2021

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

Geometric information of scenes available with fourdimensional (4-D) light fields (LFs) pave the way for postcapture refocusing. Light filed refocusing methods proposed so far have been limited to a single planar or a volumetric region of a scene. In this paper, we demonstrate simultaneous refocusing of multiple volumetric regions in LFs. To this end, we employ a 4-D sparse finite-extent impulse response (FIR) filter consisting of multiple hyperfan-shaped passbands. We design the 4-D sparse FIR filter as an optimal filter in the least-squares sense. Experimental results confirm that the proposed filter provides 64% average reduction in computational complexity with negligible degradation in the fidelity of multi-volumetric refocused LFs compared to a 4-D nonsparse FIR filter.

show abstract

“…Furthermore, Chen et al used a CNN to remove noise from LF images based on an anisotropic parallax analysis (APA) [ 26 ]. For real-time LF denoising, 4-dimensional (4D) linear and shift-invariant hyper fan filters have been proposed and implemented in hardware [ 27 ]. However, the correlation among the denoised multi-view sequences is not maintained because compression was not considered.…”

Section: Introductionmentioning

confidence: 99%

Improved Light Field Compression Efficiency through BM3D-Based Denoising Using Inter-View Correlation

Jin

Rhee

2021

Sensors

View full text Add to dashboard Cite

Multi-view or light field images have recently gained much attraction from academic and commercial fields to create breakthroughs that go beyond simple video-watching experiences. Immersive virtual reality is an important example. High image quality is essential in systems with a near-eye display device. The compression efficiency is also critical because a large amount of multi-view data needs to be stored and transferred. However, noise can be easily generated during image capturing, and these noisy images severely deteriorate both the quality of experience and the compression efficiency. Therefore, denoising is a prerequisite to produce multi-view-based image contents. In this paper, the structural characteristics of linear multi-view images are fully utilized to increase the denoising speed and performance as well as to improve the compression efficiency. Assuming the sequential processes of denoising and compression, multi-view geometry-based denoising is performed keeping the temporal correlation among views. Experimental results show the proposed scheme significantly improves the compression efficiency of denoised views up to 76.05%, maintaining good denoising quality compared to the popular conventional denoise algorithms.

show abstract

Real-Time Light Field Denoising Using a Novel Linear 4-D Hyperfan Filter

Cited by 17 publications

References 44 publications

Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

Multi-Volumetric Refocusing of Light Fields

Improved Light Field Compression Efficiency through BM3D-Based Denoising Using Inter-View Correlation

Contact Info

Product

Resources

About