An Audio-Visual Quality Assessment Methodology in Virtual Reality Environment

Zhang, Bo; Yan, Zhaoyu; Wang, Jing; Luo, Yiyu; Yang, Shu; Fei, Zesong

doi:10.1109/icmew.2018.8551522

Cited by 10 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above subjective quality assessment methods, originally proposed for conventional videos, have also been used for assessing the quality of 360 • videos, digital games, and 3D graphics on HMDs. For example, subjective quality assessment of 360 • videos using the ACR method can be found in [10][11][12][13][14][15][16][17], the ACR-HR method is applied in [14,18], the DSIS method is used in [19,20], the SAMVIQ method is chosen in [9], and the SAMPVIQ method is used in [9,21].…”

Section: Related Workmentioning

confidence: 99%

On the Effect of Standing and Seated Viewing of 360° Videos on Subjective Quality Assessment: A Pilot Study

Elwardy

Zepernick

2021

Computers

View full text Add to dashboard Cite

Due to the advances in head-mounted displays (HMDs), hardware and software technologies, and mobile connectivity, virtual reality (VR) applications such as viewing 360∘ videos on HMDs have seen an increased interest in a wide range of consumer and vertical markets. Quality assessment of digital media systems and services related to immersive visual stimuli has been one of the challenging problems of multimedia signal processing. Specifically, subjective quality assessment of 360∘ videos presented on HMDs is needed to obtain a ground truth on the visual quality as perceived by humans. Standardized test methodologies to assess the subjective quality of 360∘ videos on HMDs are currently not as developed as for conventional videos and are subject to further study. In addition, subjective tests related to quality assessment of 360∘ videos are commonly conducted with participants seated on a chair but neglect other options of consumption such as standing viewing. In this paper, we compare the effect that standing and seated viewing of 360∘ videos on an HMD has on subjective quality assessment. A pilot study was conducted to obtain psychophysical and psychophysiological data that covers explicit and implicit responses of the participants to the shown 360∘ video stimuli with different quality levels. The statistical analysis of the data gathered in the pilot study is reported in terms of average rating times, mean opinion scores, standard deviation of opinion scores, head movements, pupil diameter, galvanic skin response (GSR), and simulator sickness scores. The results indicate that the average rating times consumed for 360∘ video quality assessment are similar for standing and seated viewing. Further, the participants showed higher resolving power among different 360∘ video quality levels and were more confident about the given opinion scores for seated viewing. On the other hand, a larger scene exploration of 360∘ videos was observed for standing viewing which appears to distract from the quality assessment task. A slightly higher pupil dilation was recorded for standing viewing which suggests a slightly more immersed experience compared to seated viewing. GSR data indicate a lower degree of emotional arousal in seated viewing which seems to allow the participants to better conduct the quality assessment task. Similarly, simulator sickness symptoms are kept significantly lower when seated. The pilot study also contributes to a holistic view of subjective quality assessment and provides indicative ground truth that can guide the design of large-scale subjective tests.

show abstract

Section: Related Workmentioning

confidence: 99%

On the Effect of Standing and Seated Viewing of 360° Videos on Subjective Quality Assessment: A Pilot Study

Elwardy

Zepernick

2021

Computers

View full text Add to dashboard Cite

show abstract

“…However, the effect of user movements on the QoE of omnidirectional video is still largely unexplored, and should be investigated further. Another interesting issue, which is explored in [148], is the impact of audio degradation on omnidirectional video QoE: the authors use a neural network to combine the effects of video and audio impairment, training it on a subjective assessment dataset.…”

Section: Dynamic Factors In Video Qoementioning

confidence: 99%

A Survey on 360-Degree Video: Coding, Quality of Experience and Streaming

Chiariotti¹

2021

Preprint

View full text Add to dashboard Cite

The commercialization of Virtual Reality (VR) headsets has made immersive and 360 • video streaming the subject of intense interest in the industry and research communities. While the basic principles of video streaming are the same, immersive video presents a set of specific challenges that need to be addressed. In this survey, we present the latest developments in the relevant literature on four of the most important ones: (i) omnidirectional video coding and compression, (ii) subjective and objective Quality of Experience (QoE) and the factors that can affect it, (iii) saliency measurement and Field of View (FoV) prediction, and (iv) the adaptive streaming of immersive 360 • videos.The final objective of the survey is to provide an overview of the research on all the elements of an immersive video streaming system, giving the reader an understanding of their interplay and performance.

show abstract

“…Although developed for 2D and 3D video systems, the aforementioned subjective quality assessment methods have also been used for evaluating the quality of immersive media such as 360 • videos, digital games, and 3D graphics shown in a VR environment. In particular, quality assessment of 360 • videos on HMDs using the ACR method can be found in, e.g., [11]- [16], applications of the ACR-HR method are contained in [12], [17], the DSIS method is used in [18], [19], the SAMVIQ method is applied in [10], and the SAMPVIQ method is used in [10], [20].…”

Section: A Related Workmentioning

confidence: 99%

Comparison of ACR Methods for 360° Video Quality Assessment Subject to Participants’ Experience with Immersive Media

Elwardy

Zepernick

et al. 2020

2020 14th International Conference on Signal Processing and Communication Systems (ICSPCS)

View full text Add to dashboard Cite

Watching 360 • videos on head-mounted displays (HMDs) has become more popular in recent years. As such, subjective quality assessment of 360 • videos on HMDs is needed to obtain a ground truth on the quality as perceived by users. Due to the lack of standardized methods, the absolute category rating (ACR) with hidden reference (ACR-HR) method for conventional videos has also been used for subjective quality assessment of 360 • videos on HMDs. A modified ACR (M-ACR) method tailored for assessing 360 • videos on HMDs has recently been proposed. In this paper, we compare the ACR-HR and M-ACR with hidden reference (M-ACR-HR) method regarding subjective quality assessment of 360 • videos on HMDs. The comparison is conducted subject to participants' experience with watching immersive media on HMDs. The findings include: 1) Average rating times to cast opinion scores is lower for the ACR-HR method irrespective of participants' experience, 2) Mean opinion scores are similar for both methods irrespective of participants' experience, 3) The M-ACR-HR method is more reliable, especially with experts and participants with no experience, 4) Discomfort is significantly lower for the ACR-HR method, especially, for participants with no experience.

show abstract

An Audio-Visual Quality Assessment Methodology in Virtual Reality Environment

Cited by 10 publications

References 13 publications

On the Effect of Standing and Seated Viewing of 360° Videos on Subjective Quality Assessment: A Pilot Study

On the Effect of Standing and Seated Viewing of 360° Videos on Subjective Quality Assessment: A Pilot Study

A Survey on 360-Degree Video: Coding, Quality of Experience and Streaming

Comparison of ACR Methods for 360° Video Quality Assessment Subject to Participants’ Experience with Immersive Media

Contact Info

Product

Resources

About