NIQSV+: A No-Reference Synthesized View Quality Assessment Metric

Abstract

Benefiting from multi-view video plus depth and depth-image-based-rendering technologies, only limited views of a real 3-D scene need to be captured, compressed, and transmitted. However, the quality assessment of synthesized views is very challenging, since some new types of distortions, which are inherently different from the texture coding errors, are inevitably produced by view synthesis and depth map compression, and the corresponding original views (reference views) are usually not available. Thus the full-reference quality metrics cannot be used for synthesized views. In this paper, we propose a novel no-reference image quality assessment method for 3-D synthesized views (called NIQSV+). This blind metric can evaluate the quality of synthesized views by measuring the typical synthesis distortions: blurry regions, black holes, and stretching, with access to neither the reference image nor the depth map. To evaluate the performance of the proposed method, we compare it with four full-reference 3-D (synthesized view dedicated) metrics, five full-reference 2-D metrics, and three no-reference 2-D metrics. In terms of their correlations with subjective scores, our experimental results show that the proposed no-reference metric approaches the best of the state-of-the-art full reference and no-reference 3-D metrics; and outperforms the widely used no-reference and full-reference 2-D metrics significantly. In terms of its approximation of human ranking, the proposed metric achieves the best performance in the experimental test.

A Benchmark of DIBR Synthesized View Quality Assessment Metrics on a New Database for Immersive Media Applications

“…They exploit simple opening and closing operations to remove the synthesis distortions, and then the quality is obtained through the comparison between the synthesized image and the processed synthesized image. • NIQSV+: Improved NIQSV proposed by Tian et al in [66].…”

Deep Blind Synthesized Image Quality Assessment with Contextual Multi-Level Feature Pooling

“…Current blind synthesized image quality metrics still follow the traditional way, but paying extra attention on geometric distortions. For instance, Tian [14] assumed that holes regions appear differently before and after morphological operations. The distortion degrees can thus be estimated by extracting and pooling the differential map features.…”

“…We benchmarked four NR-IQA metrics (BRISQUE [8], NIQE [9], Kang [10], and MFIQA [13]) designed for traditional 2D image, and four DIBR-related metrics (3DSwiM [21] and SDRD [22] as FR-IQAs, NIQSV+ [14] and APT [15] as NR-IQAs). For the CNN-based metrics, we adopted the same training and testing on our DIBR image dataset.…”

Perceptual Representations of Structural Information in Images: Application to Quality Assessment of Synthesized View in FTV Scenario

“…In [14], NIQSV was proposed by hypothesizing that high-quality images are consist of flat areas separated by edges. It is then extended to NIQSV+ [15] by considering the existence of the disoccluded regions. Recently, a novel no reference quality metric for synthesized images namely APT was proposed in [16], where the auto-regression (AR) based local image description is employed.…”

“…The overall performance results are shown in [7] 0.6864 0.4842 0.6125 MP-PSNRr [10] 0.6954 0.4784 0.6606 MW-PSNRr [10] 0.6637 0.4921 0.6293 CT-IQM [11] 0.6809 0.6626 0.4877 BF-M [1] 0.6980 0.5885 0.4768 EM-IQM [12] 0.7430 0.6726 0.4455 ST-IQM [22] 0.8217 0.7710 0.3929 LoGs [13] 0.8256 0.7812 0.3601 Proposed 0.9023 0.8448 0.2870 NO Reference Metric (NR) NIQSV [14] 0.6346 0.5146 0.6167 NIQSV+ [15] 0.7114 0.4679 0.6668 APT [16] 0.7307 0.7140 0.4622 CSC-NRM [25] 0.8302 0.7827 0.3233…”