Subjective Test Dataset and Meta-data-based Models for 360° Streaming Video Quality

Fremerey, Stephan; Göring, Steve; Rao, Rakesh Rao Ramachandra; Huang, Rachel; Raake, Alexander

doi:10.1109/mmsp48831.2020.9287065

Cited by 13 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the Joint Video Exploration Team (JVET) created datasets with sequences (in uncompressed format) of 10 seconds for research on 360°video encoding [87], [88]. Also, several databases can be found in the literature with associated ratings of visual quality [31], [32], [89], [90], presence [31], [64], valence and arousal [91], and simulator sickness [31], [64], [90], [92]. In addition, many of these datasets also include data related to head movements [90], [92]- [97] and eye movements [63] of the participants.…”

Section: E Datasets Of 360°videosmentioning

confidence: 99%

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Gutiérrez¹,

Pérez

Orduna

et al. 2022

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Recently an impressive development in immersive technologies, such as Augmented Reality (AR), Virtual Reality (VR) and 360°video, has been witnessed. However, methods for quality assessment have not been keeping up. This paper studies quality assessment of 360°video from the cross-lab tests (involving ten laboratories and more than 300 participants) carried out by the Immersive Media Group (IMG) of the Video Quality Experts Group (VQEG). These tests were addressed to assess and validate subjective evaluation methodologies for 360°video. Audiovisual quality, simulator sickness symptoms, and exploration behavior were evaluated with short (from 10 seconds to 30 seconds) 360°sequences. The following factors' influences were also analyzed: assessment methodology, sequence Manuscript received

show abstract

Section: E Datasets Of 360°videosmentioning

confidence: 99%

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Gutiérrez¹,

Pérez

Orduna

et al. 2022

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many quality-estimation models have been proposed in 2D [20,21,[36][37][38][39][40][41] and VR videos [42][43][44][45][46][47].…”

Section: Quality-estimation Modelsmentioning

confidence: 99%

“…To estimate VR video quality, several studies have been conducted. Fremerey et al [42] described a qualityestimation model using bitrate, framerate, and resolution and found that it estimated quality with high accuracy. Machine learning methods are also used to estimate VR videos.…”

Section: Quality-estimation Modelsmentioning

confidence: 99%

“…Anwar et al [43] used QP and Kim et al [44] used resolution and other parameters related to quality of video for input to machine learning and proposed quality-estimation models with machine learning. These papers [42][43][44] show qualityestimation models for VR videos that are not tile-based VR and are encoded uniformly, so they did not aim to estimate the tile-based VR video or consider the degradation caused by low-resolution tiles. Li et al [45] showed the qualityestimation model of tile-based VR over a wireless network with bitrate, resolution, and other network parameters (i.e., stalling or bitrate level switching).…”

Section: Quality-estimation Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Bitstream-Quality-Estimation Model for Tile-Based VR Video Streaming Services

Koike

Urata

Yamagishi

2022

IEICE Trans. Commun.

View full text Add to dashboard Cite

Tile-based virtual reality (VR) video consists of highresolution tiles that are displayed in accordance with the users' viewing directions and a low-resolution tile that is the entire VR video and displayed when users change their viewing directions. Whether users perceive quality degradation when watching tile-based VR video depends on high-resolution tile size, the quality of high-and low-resolution tiles, and network condition. The display time of low-resolution tile (hereafter delay) affects users' perceived quality because longer delay makes users watch the low-resolution tiles longer. Since these degradations of low-resolution tiles markedly affect users' perceived quality, these points have to be considered in the qualityestimation model. Therefore, we propose a bitstream-quality-estimation model for tile-based VR video streaming services and investigate the effect of bitstream parameters and delay on tile-based VR video quality. Subjective experiments on several videos of different quality and a comparison between other video quality-estimation models were conducted. In this paper, we prove that the proposed model can improve the quality-estimation accuracy by using the high-and low-resolution tiles' quantization parameters, resolution, framerate, and delay. Subjective experimental results show that the proposed model can estimate the quality of tile-based VR video more accurately than other video quality-estimation models.

show abstract

“…Using these models we expect to be able to produce a statistical-based perceptual quality model by utilizing techniques such as curve fitting [3], metadata-based approached [5], and by incorporating viewport information for adaptive streaming application [6]. In [7], a machine learning-based QoE model have been proposed by converting continuous scores into a dichotomous score in order to build a logistic regression model.…”

Section: Introductionmentioning

confidence: 99%

Perceptual Evaluation of 360 Audiovisual Quality and Machine Learning Predictions

Fela¹,

Zacharov²,

Forchhammer³

2021

Preprint

View full text Add to dashboard Cite

In an earlier study, we gathered perceptual evaluations of the audio, video, and audiovisual quality for 360 audiovisual content. This paper investigates perceived audiovisual quality prediction based on objective quality metrics and subjective scores of 360 video and spatial audio content. Thirteen objective video quality metrics and three objective audio quality metrics were evaluated for five stimuli for each coding parameter. Four regression-based machine learning models were trained and tested here, i.e., multiple linear regression, decision tree, random forest, and support vector machine. Each model was constructed using a combination of audio and video quality metrics and two cross-validation methods (k-Fold and Leave-One-Out) were investigated and produced 312 predictive models. The results indicate that the model based on the evaluation of VMAF and AMBIQUAL is better than other combinations of audio-video quality metric. In this study, support vector machine provides higher performance using k-Fold (PCC = 0.909, SROCC = 0.914, and RMSE = 0.416). These results can provide insights for the design of multimedia quality metrics and the development of predictive models for audiovisual omnidirectional media.

show abstract

Subjective Test Dataset and Meta-data-based Models for 360° Streaming Video Quality

Cited by 13 publications

References 21 publications

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Bitstream-Quality-Estimation Model for Tile-Based VR Video Streaming Services

Perceptual Evaluation of 360 Audiovisual Quality and Machine Learning Predictions

Contact Info

Product

Resources

About