Convolutional Neural Networks for Omnidirectional Image Quality Assessment: A Benchmark

Sendjasni, Abderrezzaq; Larabi, Mohamed–Chaker; Cheikh, Faouzi Alaya

doi:10.1109/tcsvt.2022.3181235

Cited by 7 publications

(6 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For 360-degree quality assessment, CNN-based models often adopt the multichannel paradigm, in which multiple CNNs are stacked together to calculate the quality score from selected viewports. However, this architecture can be computationally complex [3] and may not fully consider the nonuniform contribution of each viewport to the overall quality. To address these issues, endto-end multichannel models that learn the relationship among inputs and the importance of each viewport have been proposed.…”

Section: Rendered Viewportmentioning

confidence: 99%

“…The model first extracts viewports on the spherical content of the 360-degree images based on visual scan-path predictions to reproduce the actual viewed content and avoid severe geometric distortions. Then, motivated by the effectiveness of well known pre-trained CNN models and the observations found in [3], we use DenseNet-121 [5] to extract visual features from the selected viewports and predict their visual quality. To adaptively learn the contribution of each viewport to the overall quality score, a viewport weight estimation is incorporated by fusing learned JND features and visual scan-path attributes such as fixation duration and fixation order.…”

Section: Rendered Viewportmentioning

confidence: 99%

“…The end-to-end model is trained to predict a single quality score based on several inputs. However, a major drawback of this approach is the number of channels or CNNs, which can significantly increase the computational complexity and may affect the robustness of the model [3]. To mitigate this drawback, some techniques such as weight sharing across channels can be helpful.…”

Section: Learning-based Modelsmentioning

confidence: 99%

“…It is widely acknowledged that 360-degree images in the projected (plane representation) format are subject to geometric distortions caused by the projection process [3,31]. Moreover, the projected content does not correspond to the actual item viewed by users using HMDs.…”

Section: Data Preparationmentioning

confidence: 99%

“…The first part is the visual feature extractor (VFE) where a pre-trained CNN model is used to encode visual features from the input viewports (VPs). In this work, the DenseNet-121 model [5] is selected based on our previous research on CNNs for 360-degree IQA [3]. The study involved a comparison of several pre-trained models across different architectures and paradigms, encompassing various CNN architectures and image representations.…”

Section: Local Quality Predictormentioning

confidence: 99%

See 4 more Smart Citations

PW-360IQA: Perceptually-Weighted Multichannel CNN for Blind 360-Degree Image Quality Assessment

Sendjasni

Larabi

2023

Sensors

Self Cite

View full text Add to dashboard Cite

Image quality assessment of 360-degree images is still in its early stages, especially when it comes to solutions that rely on machine learning. There are many challenges to be addressed related to training strategies and model architecture. In this paper, we propose a perceptually weighted multichannel convolutional neural network (CNN) using a weight-sharing strategy for 360-degree IQA (PW-360IQA). Our approach involves extracting visually important viewports based on several visual scan-path predictions, which are then fed to a multichannel CNN using DenseNet-121 as the backbone. In addition, we account for users’ exploration behavior and human visual system (HVS) properties by using information regarding visual trajectory and distortion probability maps. The inter-observer variability is integrated by leveraging different visual scan-paths to enrich the training data. PW-360IQA is designed to learn the local quality of each viewport and its contribution to the overall quality. We validate our model on two publicly available datasets, CVIQ and OIQA, and demonstrate that it performs robustly. Furthermore, the adopted strategy considerably decreases the complexity when compared to the state-of-the-art, enabling the model to attain comparable, if not better, results while requiring less computational complexity.

show abstract

Section: Rendered Viewportmentioning

confidence: 99%

Section: Rendered Viewportmentioning

confidence: 99%

Section: Learning-based Modelsmentioning

confidence: 99%

Section: Data Preparationmentioning

confidence: 99%

Section: Local Quality Predictormentioning

confidence: 99%

See 3 more Smart Citations