Non-reference Quality Assessment Model using Deep learning for Omnidirectional Images

Truong, Tung; Tran, Huyen T. T.; Thang, Truong Cong

doi:10.1109/icawst.2019.8923442

Cited by 15 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This makes the training data rich and somehow sufficient for training CNN models. Based on this idea, Truong et al [17] used equirectangular projection (ERP) content to evaluate the quality of 360-degree images, where patches of 64 × 64 are sampled according to a latitude-based strategy. During validation, an equator-biased average pooling of patches' scores is applied to estimate the overall quality.…”

Section: Patch-based Modelsmentioning

confidence: 99%

PW-360IQA: Perceptually-Weighted Multichannel CNN for Blind 360-Degree Image Quality Assessment

Sendjasni

Larabi

2023

Sensors

View full text Add to dashboard Cite

Image quality assessment of 360-degree images is still in its early stages, especially when it comes to solutions that rely on machine learning. There are many challenges to be addressed related to training strategies and model architecture. In this paper, we propose a perceptually weighted multichannel convolutional neural network (CNN) using a weight-sharing strategy for 360-degree IQA (PW-360IQA). Our approach involves extracting visually important viewports based on several visual scan-path predictions, which are then fed to a multichannel CNN using DenseNet-121 as the backbone. In addition, we account for users’ exploration behavior and human visual system (HVS) properties by using information regarding visual trajectory and distortion probability maps. The inter-observer variability is integrated by leveraging different visual scan-paths to enrich the training data. PW-360IQA is designed to learn the local quality of each viewport and its contribution to the overall quality. We validate our model on two publicly available datasets, CVIQ and OIQA, and demonstrate that it performs robustly. Furthermore, the adopted strategy considerably decreases the complexity when compared to the state-of-the-art, enabling the model to attain comparable, if not better, results while requiring less computational complexity.

show abstract

Section: Patch-based Modelsmentioning

confidence: 99%

PW-360IQA: Perceptually-Weighted Multichannel CNN for Blind 360-Degree Image Quality Assessment

Sendjasni

Larabi

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Furthermore, the just-noticeable difference map is used to account for perceptual characteristics of the HVS along with features produced from scan-paths in order to estimate the weights of each viewport. Differently from the previous mentioned works, in [18] a patch-based approach is adopted rather than a mutli-channel one. The patches are 64 × 64 and extracted from the ERP with a focus on the equatorial region.…”

Section: Introductionmentioning

confidence: 99%

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Sendjasni¹,

Larabi²,

Cheikh³

2021

lim

View full text Add to dashboard Cite

360-degree Image quality assessment (IQA) is facing the major challenge of lack of ground-truth databases. This problem is accentuated for deep learning based approaches where the performances are as good as the available data. In this context, only two databases are used to train and validate deep learning-based IQA models. To compensate this lack, a dataaugmentation technique is investigated in this paper. We use visual scan-path to increase the learning examples from existing training data. Multiple scan-paths are predicted to account for the diversity of human observers. These scan-paths are then used to select viewports from the spherical representation. The results of the data-augmentation training scheme showed an improvement over not using it. We also try to answer the question of using the MOS obtained for the 360-degree image as the quality anchor for the whole set of extracted viewports in comparison to 2D blind quality metrics. The comparison showed the superiority of using the MOS when adopting a patch-based learning.

show abstract

“…Recently, the great success of convolutional neural network (CNN) [16,17,18,19] boosts the IQA performance significantly [20,21,22,23]. Although many works have been proposed for FR/NR IQA on 2D images, there still exists limited research on FR/NR IQA for ODIs or omnidirectional videos (ODVs) [24,25,26,27]. Specifically, in [27], patch sampling and quality score pooling strategies are proposed for NR-IQA based on ODI equator-bias technique.…”

Section: Introductionmentioning

confidence: 99%

“…Although many works have been proposed for FR/NR IQA on 2D images, there still exists limited research on FR/NR IQA for ODIs or omnidirectional videos (ODVs) [24,25,26,27]. Specifically, in [27], patch sampling and quality score pooling strategies are proposed for NR-IQA based on ODI equator-bias technique. Considering the projection distortion of ODI, MC360IQA [24] develops a viewport-based multi-channel CNN for NR-IQA, via projecting ODI into six equal cube faces.…”

Section: Introductionmentioning

confidence: 99%

“…Although the above methods take human behavior or equator-bias characteristic of ODIs/ODVs into account for IQA, the pre-processing tasks, i.e., viewport prediction, patch sampling, pay much more computational expense compared with the main task of IQA. Besides, the viewport extraction [24] and patch sampling [27] strategies are both designed in hand-crafted manner, which may not hold for all distortion types. Thus, these issues make the IQA performance unsatisfactory and lead to large computational complexity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatial Attention-based Non-reference Perceptual Quality Prediction Network for Omnidirectional Images

Yang¹,

Xu²,

Deng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Due to the strong correlation between visual attention and perceptual quality, many methods attempt to use human saliency information for image quality assessment. Although this mechanism can get good performance, the networks require human saliency labels, which is not easily accessible for omnidirectional images (ODI). To alleviate this issue, we propose a spatial attention-based perceptual quality prediction network for non-reference quality assessment on ODIs (SAP-net). To drive our SAP-net, we establish a large-scale IQA dataset of ODIs (IQA-ODI), which is composed of subjective scores of 200 subjects on 1,080 ODIs. In IQA-ODI, there are 120 high quality ODIs as reference, and 960 ODIs with impairments in both JPEG compression and map projection. Without any human saliency labels, our network can adaptively estimate human perceptual quality on impaired ODIs through a self-attention manner, which significantly promotes the prediction performance of quality scores. Moreover, our method greatly reduces the computational complexity in quality assessment task on ODIs. Extensive experiments validate that our network outperforms 9 state-of-theart methods for quality assessment on ODIs. The dataset and code have been available on https://github.com/ yanglixiaoshen/SAP-Net.

show abstract

Non-reference Quality Assessment Model using Deep learning for Omnidirectional Images

Cited by 15 publications

References 9 publications

PW-360IQA: Perceptually-Weighted Multichannel CNN for Blind 360-Degree Image Quality Assessment

PW-360IQA: Perceptually-Weighted Multichannel CNN for Blind 360-Degree Image Quality Assessment

Visual Scan-Path based Data-Augmentation for CNN-based 360-degree Image Quality Assessment

Spatial Attention-based Non-reference Perceptual Quality Prediction Network for Omnidirectional Images

Contact Info

Product

Resources

About