Subjective evaluation of conversational multimedia quality in IP networks

Brauer, F.; Ehsan, Muhammad; Kubin, Gernot

doi:10.1109/mmsp.2008.4665196

Cited by 10 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As these tests have limited ecological validity, active studies have been proposed, where participants actually interact through the system. The majority of these works has been conducted in two party scenarios [29], [20], [30], [31], [19], and employing the Lego® building blocks [17] task. It should be noted that most of these studies use relatively low resolution video (640x480px [31]) and encoding bitrates (maximum 2Mbit [31]).…”

Section: A Impact Of System Factors On Videoconferencing Qoe and Intmentioning

confidence: 99%

See 1 more Smart Citation

Towards Individual QoE for Multiparty Videoconferencing

Schmitt

Redi

Bulterman

et al. 2018

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-Video-conferencing is becoming an essential part in everyday life. The visual channel allows for interactions which were not possible over audio-only communication systems such as the telephone. However, being a de-facto over-the-top service, the quality of the delivered video-conferencing experience is subject to variations, dependent on network conditions. Video-conferencing systems adapt to network conditions by changing for example encoding bitrate of the video. For this adaptation not to hamper the benefits related to the presence of a video channel in the communication, it needs to be optimized according to a measure of the Quality of Experience (QoE) as perceived by the user. The latter is highly dependent on the ongoing interaction and individual preferences, which have hardly been investigated so far. In this paper, we focus on the impact video quality has on conversations that revolve around objects that are presented over the video channel. To this end we conducted an empirical study where groups of 4 people collaboratively build a Lego® model over a video-conferencing system. We examine the requirements for such a task by showing when the interaction, measured by visual and auditory cues, changes depending on the encoding bitrate and loss. We then explore the impact that prior experience with the technology and affective state have on QoE of participants. We use these factors to construct predictive models which double the accuracy compared to a model based on the system factors alone. We conclude with a discussion of how these factors could be applied in real world scenarios.Index Terms-Multi-Party video conferencing, Quality of Experience, Over-the-top, subjective quality, quality metrics, user study

show abstract

Section: A Impact Of System Factors On Videoconferencing Qoe and Intmentioning

confidence: 99%

“…[33]) , or simulations (e.g. [34]) but there is only one study which investigated the effect in video-conferencing with an interactive subjective test [20] with a relatively dated setup (CIF, 15fps, theora codec).…”

Section: A Impact Of System Factors On Videoconferencing Qoe and Intmentioning

confidence: 99%

Towards Individual QoE for Multiparty Videoconferencing

Schmitt

Redi

Bulterman

et al. 2018

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…The building blocks task has been used in previous studies to evaluate video quality in video-conferencing [4][9] [8]. However the setup and assessed conditions are too different for a direct comparison…”

Section: Discussionmentioning

confidence: 99%

“…Interactive studies, especially for multiparty scenarios, have been mostly looking at the effect of delay on QoE [4][5] [2], or at different stream encodings in combination with different dynamic layouts [6]. The impact of encoding and loss rate on quality has been studied only in two-party scenarios [7][8] [9].…”

Section: Introductionmentioning

confidence: 99%

“…Popular video delivery services encode their 720p videos within 2Mbps and 4Mbps 3 . A multiparty conference, with such bitrates, easily maxes out the typical broadband connections available at households 4 . It is therefore important to establish (1) what is the minimal encoding quality for 720p video to still deliver a satisfactory experience; (2) what is the maximum bitrate above which no significant QoE improvement can be achieved and (3) the extent to which packet loss, typically occurring in situations like a home WLAN, impacts on QoE, also depending on the encoding bitrate.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

1Mbps is enough: Video quality and individual idiosyncrasies in multiparty HD video-conferencing

Schmitt

Redi

César

et al. 2016

2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX)

View full text Add to dashboard Cite

Abstract-Most video platforms deliver HD video in high bitrate encoding. Modern video-conferencing systems are capable of handling HD streams, but using multiparty conferencing, average internet connections in the home are on their bandwidth limit. For properly managing the encoding bitrate in videoconferencing, we must know what is the minimum bitrate requirement to provide users an acceptable experience, and what is the bitrate level after which QoE saturates?. Most available subjective studies in this area used rather dated technologies. We report on a multiparty study on video quality with HD resolution. We tested different encoding bitrates (256kbs, 1024kbs and 4096kbs) and packet loss rates (0, 0.5%) in groups of 4 participants with a scenario based on the ITU building blocks task. We discuss the influence of group interaction and individual idiosyncrasies based on different mixed models, and look at covariates engagement and enjoyment as further explanatory factors. We found that 256kbs is still sufficient to provide a fair overall experience, but video quality is noticed to be poor. On the higher bitrate end, most people will not perceive the difference between 1024kbs and 4096kbs, considering in both cases the quality to be close to excellent. Independent on bitrate, packet loss has a small but significant impact, quantifiable in, on average, less than half a point difference on a 5-point ITU scale.

show abstract

The contrast effect: QoE of mixed video-qualities at the same time

2018

View full text Add to dashboard Cite

In desktop multi-party video-conferencing videostreams of participants are delivered in different qualities, but we know little about how such composition of the screen affects the quality of experience. Do the different videostreams serve as indirect quality references and the perceived video quality is thus dependent on other streams in the same session? How is the relation between the perceived qualities of each stream and the perceived quality of the overall session? To answer these questions we conducted a crowdsourcing study, in which we gathered over 5000 perceived quality ratings of overall sessions and individual streams. Our results show a contrast effect: high quality streams are rated better when more low quality streams are co-present, and vice versa. In turn, the quality perception of an overall session can increase significantly by exchanging one low quality stream with a high quality one. When comparing the means of individual and overall ratings we can further observe that users are more critical when asked for individual streams than for an overall rating. However, the results show that while contrast effect exists, the effect is not strong enough, to optimize the experience by lowering the quality of other participants.

show abstract

Subjective evaluation of conversational multimedia quality in IP networks

Cited by 10 publications

References 2 publications

Towards Individual QoE for Multiparty Videoconferencing

Towards Individual QoE for Multiparty Videoconferencing

1Mbps is enough: Video quality and individual idiosyncrasies in multiparty HD video-conferencing

The contrast effect: QoE of mixed video-qualities at the same time

Contact Info

Product

Resources

About