Convolutional Neural Networks for Omnidirectional Image Quality Assessment: Pre-Trained or Re-Trained?

Sendjasni, Abderrezzaq; Larabi, Mohamed–Chaker; Cheikh, Faouzi Alaya

doi:10.1109/icip42928.2021.9506192

Cited by 5 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We chose the ResNet-50 with its pre-trained weights obtained on the ImageNet [12] database as it is the most common and widely used. Furthermore, this choice is also motivated based on conclusions of a previous comparative study [27] for which it ranked the best compared to VGG-16/19, ResNet-18/34, and Inception-V3 models.…”

Section: Architecture Of the Modelmentioning

confidence: 99%

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Sendjasni¹,

Larabi²,

Cheikh³

2021

lim

Self Cite

View full text Add to dashboard Cite

360-degree Image quality assessment (IQA) is facing the major challenge of lack of ground-truth databases. This problem is accentuated for deep learning based approaches where the performances are as good as the available data. In this context, only two databases are used to train and validate deep learning-based IQA models. To compensate this lack, a dataaugmentation technique is investigated in this paper. We use visual scan-path to increase the learning examples from existing training data. Multiple scan-paths are predicted to account for the diversity of human observers. These scan-paths are then used to select viewports from the spherical representation. The results of the data-augmentation training scheme showed an improvement over not using it. We also try to answer the question of using the MOS obtained for the 360-degree image as the quality anchor for the whole set of extracted viewports in comparison to 2D blind quality metrics. The comparison showed the superiority of using the MOS when adopting a patch-based learning.

show abstract

Section: Architecture Of the Modelmentioning

confidence: 99%

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Sendjasni¹,

Larabi²,

Cheikh³

2021

lim

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the extraction of these patches, the use of radial content (i.e. from the sphere) is highly recommended compared to the projected one [11,12]. This way, the geometric distortion induced by the sphere-to-plane projection can be avoided.…”

Section: Introductionmentioning

confidence: 99%

Patch-based CNN model for 360 image quality assessment with adaptive pooling strategies

Sendjasni¹,

Larabi²,

Cheikh³

2022

View full text Add to dashboard Cite

Patch-based training for 360-degree images allows to significantly reduce the complexity compared to multichannel models while maintaining good performances. Differently from multichannel models where multi neural networks are trained in parallel to predict the score of the whole 360-degree image, a pooling stage is required to map local qualities to the global one. This step is often neglected by using a simple arithmetic mean, which does not account for (i) the non-uniformity distribution of quality and (ii) the variability among local qualities. In this paper, we analyze several pooling strategies, including basic statistic methods and adaptive pooling ones. Additionally, we propose a pooling strategy based on scene exploration behavior relying on visual scan-path. The performance analysis showed the benefit of using adaptive pooling over arithmetic mean, as well as the incorporation of perceptual properties during the pooling stage. Besides, the comparison with state-of-the-art multichannel models asserts the effectiveness of patch-based training compared to multichannel models.

show abstract

Perceptual Quality Assessment of Omnidirectional Images: A Benchmark and Computational Model

Liu,

Yan,

Huang

et al. 2024

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Compared with traditional 2D images, omnidirectional images (also referred to as 360° images) have more complicated perceptual characteristics due to the particularities of imaging and display. How humans perceive omnidirectional images in an immersive environment and form the immersive quality of experience are important problems. Thus, it is crucial to measure the quality of omnidirectional images under different viewing conditions, which suffer from realistic distortions. In this paper, we build a large-scale subjective assessment database for omnidirectional images and carry out a comprehensive psychophysical experiment to study the relationships between different factors (viewing conditions and viewing behaviors) and the perceptual quality of omnidirectional images. In addition, we collect both subjective ratings and head movement data. A thorough analysis of the collected subjective data is also provided, where we make several interesting findings. Moreover, with the proposed database, we propose a novel transformer-based omnidirectional image quality assessment model. To be consistent with the human viewing process, viewing conditions and behaviors are naturally incorporated into the proposed model. Specifically, the proposed model mainly consists of three parts: viewport sequence generation, multi-scale feature extraction, and perceptual quality prediction. Extensive experimental results conducted on the proposed database demonstrate the effectiveness of the proposed method over existing image quality assessment methods.

show abstract

Convolutional Neural Networks for Omnidirectional Image Quality Assessment: Pre-Trained or Re-Trained?

Cited by 5 publications

References 18 publications

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Patch-based CNN model for 360 image quality assessment with adaptive pooling strategies

Perceptual Quality Assessment of Omnidirectional Images: A Benchmark and Computational Model

Contact Info

Product

Resources

About