Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments

Abstract

Abstract-The ever growing bandwidth in access networks, in combination with IPTV and Video on Demand (VoD) offerings, opens up unlimited possibilities to the users. The operators can no longer compete solely on the number of channels or content and increasingly make High Definition channels and Quality of Experience (QoE) a service differentiator. Currently the most reliable way of assessing and measuring QoE is conducting subjective experiments, where human observers evaluate a series of short video sequences, using one of the international standardized subjective quality assessment methodologies. Unfortunately, since these subjective experiments need to be conducted in controlled environments and pose limitations on the sequences and overall experiment duration they cannot be used for reallife QoE assessment of IPTV and VoD services. In this article, we propose a novel subjective quality assessment methodology based on full length movies. Our methodology enables audiovisual quality assessment in the same environments and under the same conditions users typically watch television. Using our new methodology we conducted subjective experiments and compared the outcome with the results from a subjective test conducted using a standardized method. Our findings indicate significant differences in terms of impairment visibility and tolerance and highlight the importance of real-life QoE assessment.

Quality of Experience of adaptive video streaming: Investigation in service parameters and subjective quality assessment methodology

“…Therefore, longer test sequences may be more appropriate to study these cases. Also, as presented in [11], traditional testing methods may not accurately predict the perceptual quality, because the relative impact of impairment types would change with the setting of subjective test. This means, it is not clear if the perceptual quality of adaptation event solely evaluated using the ACR method would be the same as actual scenario when the adaptaion is occurred during a longer sequence.…”

State-of-the-Practice in Evaluation of Quality of Experience in Real-Life Environments

“…The quality of the content was not degraded in any way. Contrary to the conclusions reported in [43,44,46,47], Strohmeier et al state that there were no real differences between the results of real-life and laboratory testing.…”

“…Staelens et al in [46] and [47] report on the results of the evaluation of QoE of full-length movies, whose quality was degraded in several segments by introducing packet loss and using video coded with lower bit rates. After receiving the copies of a DVD video, the test subjects were asked just to watch the movie as they would normally do (e.g.…”

Quality Assessment of User-Generated Video Using Camera Motion

“…Moreover, the traditional QA methods are usually performed on broadcast video which has been professionally preproduced [8]. Compared to broadcast video such as news and sports video, UGV is usually of lower quality, due to the uncontrolled capturing conditions and various types of capture devices.…”