Human perception of jitter and media synchronization

Steinmetz, Ralf

doi:10.1109/49.481694

Cited by 379 publications

(270 citation statements)

References 14 publications

Supporting

Mentioning

239

Contrasting

Unclassified

Order By: Relevance

“…A considerable body of work on synchronisation in audiovisual streaming originated several years ago [22], and rules-of-thumb for non-noticable and acceptable asynchronies were established at the time [34]. As multiplexing became the predominant solution for audiovisual streaming based on MPEG-1 and MPEG-2, asynchrony became less relevant as a research topic, and as an industry challenge.…”

Section: Introductionmentioning

confidence: 99%

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Griwodz

Halvorsen

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the Multimed Tools Appl (2015) 74:345-365 form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.

show abstract

Section: Introductionmentioning

confidence: 99%

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Griwodz

Halvorsen

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…A similar experiment as the one conducted by Steinmetz [28] has been repeated in [3] where the focus was specifically on mobile environments. The authors argue that different detection and annoyance thresholds may apply in mobile environments due to the change in screen size, viewing distance and frame rate compared to the TV viewing environment.…”

Section: Audio/video Synchronization Perceptibility Thresholdsmentioning

confidence: 99%

“…Steinmetz [28] performed an in-depth analysis of the influence of jitter and media synchronization on perceived quality. The goal of his study was to identify the thresholds at which lip sync becomes noticeable and/or annoying.…”

Section: Audio/video Synchronization Perceptibility Thresholdsmentioning

confidence: 99%

See 1 more Smart Citation

Assessing the importance of audio/video synchronization for simultaneous translation of video sequences

Staelens

Meulenaere²,

Bleumers³

et al. 2012

Multimedia Systems

View full text Add to dashboard Cite

Lip synchronization is considered a key parameter during interactive communication. In the case of video conferencing and television broadcasting, the differential delay between audio and video should remain below certain thresholds, as recommended by several standardization bodies. However, further research has also shown that these thresholds can be relaxed, depending on the targeted application and use case. In this article, we investigate the influence of lip sync on the ability to perform real-time language interpretation during video conferencing. Furthermore, we are also interested in determining proper lip sync visibility thresholds applicable to this use case. Therefore, we conducted a subjective experiment using expert interpreters, which were required to perform a simultaneous translation, and using non-experts. Our results show that significant differences are obtained when conducting subjective experiments with expert interpreters. As interpreters are primarily focused on performing the si-N. Staelens · N. Vercammen · B. Vermeulen · P. Demeester

show abstract

“…Further, the type of content as well as the quality of the video, e.g. video resolution, quantization level, and frame rate are also impacting the perceived skewness [6,22]. In the case of real time two-way communication another important contribution to the quality of experience is delay.…”

mentioning

confidence: 99%

A robust method for estimating synchronization and delay of audio and video for communication services

Rossholm

Lövström

2014

Multimed Tools Appl

View full text Add to dashboard Cite

PostprintThis is the accepted version of a paper published in Multimedia tools and applications. This paper has been peer-reviewed but does not include the final publisher proof-corrections or journal pagination. Abstract One of the main contributions to the quality of experience in streaming services or in two-way communication of audio and video applications is synchronization. This has been shown in several studies and experiments but methods to measure synchronization are less frequent, especially for situations without internal access to the application and independent of platform and device. In this paper we present a method for measuring synchronization skewness as well as delay for audio and video. The solution incorporates audio and video reference streams, where audio and video frames are marked with frame numbers which are decoded on the receiver side to enable calculation of synchronization and delay. The method has been verified in a two-way communication application in a transparent network with and without inserting known delays, as well as in a network with 5 and 10 % packet loss levels. The method can be used for both streaming and two-way communication services, both with and without access to the internal structures, and enables measurements of applications running on e.g. smartphones, tablets, and laptops under various conditions.

show abstract

Human perception of jitter and media synchronization

Cited by 379 publications

References 14 publications

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Assessing the importance of audio/video synchronization for simultaneous translation of video sequences

A robust method for estimating synchronization and delay of audio and video for communication services

Contact Info

Product

Resources

About