On the Improvement of 2D Quality Assessment Metrics for Omnidirectional Images

Sendjasni, Abderrezzaq; Larabi, Mohamed–Chaker; Cheikh, Faouzi Alaya

doi:10.2352/issn.2470-1173.2020.9.iqsp-287

Cited by 5 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can notice that traditional 2D models and their extended versions have significantly lower performance compared to 360-degree models. Therefore, they are not well suited for this type of image as already demonstrated in benchmark studies [5]. SSP-BOIQA slightly improves the correlation with subjective MOS compared to SSIM that measures the structural similarity according to the HVS characteristics.…”

Section: Performance Comparisonmentioning

confidence: 93%

“…As these models do not account for perceptual aspects, they fail in predicting the visual quality accurately. Besides, well-performing 2D metrics are not suitable for 360-degree images as they neither account for spherical characteristics nor for the specific exploration of the scene made by observers [5]. These limitations push towards the design of specific IQA models accounting for the perceptual peculiarities of 360-degree images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Perceptually-Weighted Cnn For 360-Degree Image Quality Assessment Using Visual Scan-Path And Jnd

Sendjasni¹,

Larabi²,

Cheikh³

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Image quality assessment of immersive content and more specifically 360-degree one is still in its infancy. There are many challenges regarding sphere vs. projected representation, human visual system (HVS) properties in a 360-degree environment, etc. In this paper, we propose the use of CNNs to design a no reference model to predict visual quality of 360-degree images. Instead of feeding the CNN with ERPs, visually important viewports are extracted based on visual scan-path prediction and given to a multi-channel CNN using DenseNet-121. Moreover, information about visual fixations and just noticeable difference are used to account for the HVS properties and make the network closer to human judgment. The scan-path is also used to create multiple instances of the database so as to perform a robust generalization analysis and compensate for the lack of databases.

show abstract

Section: Performance Comparisonmentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

Perceptually-Weighted Cnn For 360-Degree Image Quality Assessment Using Visual Scan-Path And Jnd

Sendjasni¹,

Larabi²,

Cheikh³

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Still, these metrics do not account for the non-uniform sampling density at pixel locations from the sphere to plane projection [1]. And, their performances are lacking in terms of correlation with subjective quality scores as shown in [2]. Thus, having metrics dedicated to omnidirectional images becomes of major importance in order to meet the challenges related to this type of content.…”

Section: Introductionmentioning

confidence: 99%

“…One major challenge when dealing with omnidirectional images is the lack of reliable and representative datasets and mean opinion scores that would allow deep learning models to show their full potential. The construction of such databases require important efforts in terms of scenes acquisition, device calibration, paradigm defini-tion, subjective testing and data analysis [2]. To cope with this lack, the use of pre-trained models seems a good alternative.…”

Section: Introductionmentioning

confidence: 99%

Convolutional Neural Networks for Omnidirectional Image Quality Assessment: Pre-Trained or Re-Trained?

Sendjasni¹,

Larabi²,

Cheikh³

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

The use of convolutional neural networks (CNN) for image quality assessment (IQA) becomes many researcher's focus. Various pretrained models are fine-tuned and used for this task. In this paper, we conduct a benchmark study of seven state-of-the-art pre-trained models for IQA of omnidirectional images. To this end, we first train these models using an omnidirectional database and compare their performance with the pre-trained versions. Then, we compare the use of viewports versus equirectangular (ERP) images as inputs to the models. Finally, for the viewports-based models, we explore the impact of the input number of viewports on the models' performance. Experimental results demonstrated the performance gain of the re-trained CNNs compared to their pre-trained versions. Also, the viewports-based approach outperformed the ERP-based one independently of the number of selected views.

show abstract

“…Still, these metrics do not account for the non-uniform sampling density at pixel locations from the sphere to plane projection [4]. And, their performances are lacking in terms of correlation with subjective quality scores as shown in [5]. Furthermore, the most projection format used is the equirectangular (ERP) one.…”

Section: Introductionmentioning

confidence: 99%

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Sendjasni¹,

Larabi²,

Cheikh³

2021

lim

Self Cite

View full text Add to dashboard Cite

360-degree Image quality assessment (IQA) is facing the major challenge of lack of ground-truth databases. This problem is accentuated for deep learning based approaches where the performances are as good as the available data. In this context, only two databases are used to train and validate deep learning-based IQA models. To compensate this lack, a dataaugmentation technique is investigated in this paper. We use visual scan-path to increase the learning examples from existing training data. Multiple scan-paths are predicted to account for the diversity of human observers. These scan-paths are then used to select viewports from the spherical representation. The results of the data-augmentation training scheme showed an improvement over not using it. We also try to answer the question of using the MOS obtained for the 360-degree image as the quality anchor for the whole set of extracted viewports in comparison to 2D blind quality metrics. The comparison showed the superiority of using the MOS when adopting a patch-based learning.

show abstract

On the Improvement of 2D Quality Assessment Metrics for Omnidirectional Images

Abstract: Fast track article for IS&T International Symposium on Electronic Imaging 2020: Image Quality and System Performance proceedings.

Cited by 5 publications

References 19 publications

Perceptually-Weighted Cnn For 360-Degree Image Quality Assessment Using Visual Scan-Path And Jnd

Perceptually-Weighted Cnn For 360-Degree Image Quality Assessment Using Visual Scan-Path And Jnd

Convolutional Neural Networks for Omnidirectional Image Quality Assessment: Pre-Trained or Re-Trained?

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Contact Info

Product

Resources

About