Subjective Evaluation of Light Field Image Compression Methods based on View Synthesis

Abstract

Light field (LF) images provide rich visual information enabling amazing applications, from post-capture image processing to immersive applications. However, this rich information requires significant storage and bandwidth capabilities thus urgently raises the question of their compression. Many studies have investigated the compression of LF images using both spatial and angular redundancies existing in the LF images. Recently, interesting LF compression approaches based on view synthesis technique have been proposed. In these approaches, only sparse samples of LF views are encoded and transmitted, while the other views are synthesized at decoder side. Different techniques have been proposed to synthesize the dropped views. In this paper, we describe subjective quality evaluation of two recent compression methods based on view synthesis and comparing them to two pseudo-video sequence based coding approaches. Results show that view synthesis based approaches provide higher visual quality than the naive LF coding approaches. In addition, the database as well as subjective scores are publicly available to help designing new objective metrics or can be used as a benchmark for future development of LF coding methods. 1

The Good News, the Bad News, and the Ugly Truth: A Review on the 3D Interaction of Light Field Displays

“…This statement is also applicable to other types of visualization technologies, as well as audio. In the context of light field visualization, subjective quality evaluation commonly accompanies research efforts related to compression [81][82][83][84][85][86][87][88]. Other topics addressing light field QoE include, but are definitely not limited to, objective quality assessment (i.e., the prediction or estimation of subjectively perceived quality through metrics and models) [89][90][91][92][93][94]; datasets for objective and subjective assessment [95][96][97][98][99][100][101][102][103]; reconstruction, subsampling, interpolation, and view synthesis [104][105][106][107][108][109][110]; spatial and angular resolution [111][112][113][114]; methodology and viewing conditions [115][116][117][118][119][120]; human factors and content impact [121][122][123][124]; and renderi...…”

On the use-case-specific quality degradations of light-field visualization

“…4 While such characteristics of content and display are directly studied -e.g., by the research efforts of Kovacs et al 5,6 -yet the majority of the scientific literature on light field visualization approaches the topic of quality degradation via compression. Kovacs et al 7 investigated H.264/MVC, Adhikarla et al, 8 Ahar et al, 9 Bakir et al, 10 Paudyal et al, 11 Perra et al, 12 Recio et al, 13 Shi et al 14 and Tian et al 15 involved H.265/HEVC. The utilization of JPEG and JPEG 2000 is also frequent in the literature, like in the work of Shan et al 16 and in several of the previously listed scientific publications using HEVC.…”

“…The utilization of JPEG and JPEG 2000 is also frequent in the literature, like in the work of Shan et al 16 and in several of the previously listed scientific publications using HEVC. Further degradations include Gaussian noise (also quite common, e.g., Adhikarla et al 8 ), VP9 coding (e.g., Viola et al 17 ), WAC coding (e.g., Ahar et al 9 ), VVC coding (e.g., Bakir et al 10 ), LFTC coding (e.g., Palma et al 18 ), encryption (Wen et al 19 ), watermarking (e.g., Paudyal et al 20 ), light field reconstruction (e.g., Kara et al 21 ) and view interpolation (e.g., Cserkaszky et al 22 ).…”

Beyond Perceptual Thresholds and Personal Preference: Towards Novel Research Questions and Methodologies of Quality of Experience Studies on Light Field Visualization

“…Over the recent years, numerous works on light field visualization have been published. Adhikarla et al [1] performed a preference test regarding multiple levels of degradation, Bakir et al [2] examined how view-synthesis-based light field compression affects the perceived quality; Battisti et al [3] investigated the artefacts that may appear on views synthesized by depth-image-based rendering (DIBR); Carballeira et al [4] studied the suitable levels of view density; Palma et al [5] and Perra et al [6] subjectively assessed different codecs and bitrates; Paudyal et al [7] addressed the perceivable results of wavelet-based watermarking; Recio et al [8] particularly focused on view transition; Shi et al [9] evaluated the performance of the conventional objective quality metrics; and Wen et al [10] tested encrypted light field contents. These scientific efforts have a couple of things in common.…”