Perceptual Quality Assessment of Omnidirectional Images

Abstract

Omnidirectional images, also called 360◦images, have attracted extensive attention in recent years, due to the rapid development of virtual reality (VR) technologies. During omnidirectional image processing including capture, transmission, consumption, and so on, measuring the perceptual quality of omnidirectional images is highly desired, since it plays a great role in guaranteeing the immersive quality of experience (IQoE). In this paper, we conduct a comprehensive study on the perceptual quality of omnidirectional images from both subjective and objective perspectives. Specifically, we construct the largest so far subjective omnidirectional image quality database, where we consider several key influential elements, i.e., realistic non-uniform distortion, viewing condition, and viewing behavior, from the user view. In addition to subjective quality scores, we also record head and eye movement data. Besides, we make the first attempt by using the proposed database to train a convolutional neural network (CNN) for blind omnidirectional image quality assessment. To be consistent with the human viewing behavior in the VR device, we extract viewports from each omnidirectional image and incorporate the user viewing conditions naturally in the proposed model. The proposed model is composed of two parts, including a multi-scale CNN-based feature extraction module and a perceptual quality prediction module. The feature extraction module is used to incorporate the multi-scale features, and the perceptual quality prediction module is designed to regress them to perceived quality scores. The experimental results on our database verify that the proposed model achieves the competing performance compared with the state-of-the-art methods.

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

“…Inspired by this observation, we define our training loss function based on the viewports of the omnidirectional image, reflecting how an omnidirectional image is viewed (Sui et al 2021;Fang et al 2022). Specifically, we first adopt rectilinear projections (Ye, Alshina, and Boyce 2017) to map the recovered HR image in ERP format back to the 3D sphere format and then sample 14 viewports uniformly distributed over the sphere for each omnidirectional image 2 , which cover all spherical content.…”

View-Aware Salient Object Detection for $360^{\circ }$ Omnidirectional Image

“…Recently, some researchers [2], [7], [8], [30]- [42] turn attention to saliency studies in 360 • panoramic scenarios. While most datasets only provide either eye-fixation groundtruth data for saliency prediction or bounding box groundtruth for object detection, which can promote salient object detection but is not enough for accurate pixel-wise salient object segmentation in panoramic scenarios.…”

Blind Multimodal Quality Assessment: A Brief Survey and A Case Study of Low-light Images

“…Existing deep-learned image quality representation [18] mainly relies on the extraction and integration of features in the spatial domain. F img is usually designed as a stack of multiple layers of DNN blocks.…”