Perceived 3D TV Transmission Quality Assessment: Multi-Laboratory Results Using Absolute Category Rating on Quality of Experience Scale

Abstract

International audienceInspired by the rapidly increasing popularity of 3D movies, there is an industrial push for 3DTV services to the home. One important factor for the success and acceptance by the viewers is a positive quality of experience (QoE) of the new service when delivered. The questions of how to efficiently deliver 3DTV service to the home, and how to evaluate the visual quality perceived by end users are a recent research focus. We have investigated users' experience of stereoscopic 3D video quality by preparing two subjective assessment datasets. The first dataset aimed at the evaluation of efficient transmission in the transmission error free case, while the second focused on error concealment. A total of three subjective assessments, two for first dataset and one for the second, were performed using the Absolute Category Rating with Hidden unimpaired Reference video (ACR-HR) method. The experimental setup allows to show that the ACR-HR subjective method provides repeatable results across labs and across conditions for video quality. It was also verified that MVC is more efficient than H.264 simulcast coding. Furthermore it was discovered that based on the same level of quality of experience spatial down-sampling may lead to better bitrate efficiency while temporal down-sampling is not acceptable. When network impairments occur, traditional error 2D concealment methods need to be reinvestigated as they were outperformed by displaying the same view for both eyes (switching to 2D presentation)

3D Video

“…Traditional approaches to evaluate QoE by means of single stimulus methodologies fail to capture the multi-dimensionality of 3D video. They seem to be mainly evaluating one component, namely pictorial quality, as can be seen in [33,55]. In these studies, it can be observed that 3D QoE is not rated higher than 2D QoE.…”

Subjective quality assessment of multichannel audio accompanied with video in representative broadcasting genres

“…Many research efforts are currently being oriented to the development of new technologies devoted to immersive communication, virtual reality and interactive media [20,36]. As a result, providing high audiovisual quality through the combination of emerging audio and video formats is a major objective in audiovisual research and future broadcasting applications [61,63,37].…”

Analysis of quality of experience in 3D video systems

“…It is also worth noting that it is possible to include undistorted sequences as test clips to allow a better analysis of the results, which is usually called ACR with hidden references. Moreover, this method has been used in various VQEG [206,207] projects and in many tests with 3D video, as in [212].…”

“…Moreover, this effect mainly affects zones of the images containing moving elements, and it could propagate to several frames (until a new I-frame is correctly received), due to the use of reference frames in the encoding process. Although these effects already cause severe degradations of the QoE in 2D video, their impact in 3D content may be even more annoying, since they could cause binocular rivalry when the same region in both stereo views is affected differently [57,212]. This effect is deeply investigated in the following section 5.3.…”

“…As an example, it is worth noting the analysis of different transmission errors and scenarios carried out by Strohmeier et al [180]. In addition, Wang et al [212] performed a multi-laboratory experiment investigating the impact on the QoE of various transmission events with simulcast and MVC 3D video. Furthermore, another study was proposed by Yasakethu…”