Evaluating the influence of frame rate on the temporal aspects of audiovisual speech perception

Vatakis, Argiro; Spence, Charles

doi:10.1016/j.neulet.2006.06.041

Cited by 17 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the perceptual consequences of such a trade-off are largely unexplored. One exception is frame rate, for which studies have found adverse effects on the temporal integration of audio and video [19,38]. Since reduced frame rates affect the temporal nature of the video, they also affect the temporal integration of the audiovisual signals.…”

Section: Temporal Integration and Quality Distortionmentioning

confidence: 99%

“…Since reduced frame rates affect the temporal nature of the video, they also affect the temporal integration of the audiovisual signals. Specifically, the subjective point of perceived synchrony is shifted with low frame rates, demanding that audio must precede video with up to 42 ms to achieve experienced synchrony [38]. Although the chance of congestion is reduced by lower frame rates, there are further reasons why this approach is seldom an attractive alternative.…”

Section: Temporal Integration and Quality Distortionmentioning

confidence: 99%

See 1 more Smart Citation

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Griwodz

Halvorsen

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the Multimed Tools Appl (2015) 74:345-365 form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.

show abstract

Section: Temporal Integration and Quality Distortionmentioning

confidence: 99%

Section: Temporal Integration and Quality Distortionmentioning

confidence: 99%

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Griwodz

Halvorsen

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…It is, in summary, necessary to elucidate whether or not the integration of audiovisual speech is mediated by attentional processes. In our opinion, this issue might be of special relevance for the acquisition of a second language and the proper use of dubbing in cinema and television (see [4,76]). …”

Section: Introductionmentioning

confidence: 99%

Assessing the role of attention in the audiovisual integration of speech

et al. 2010

Self Cite

View full text Add to dashboard Cite

“…Crossmodal simultaneity in multimodal telepresence system 3.1 Influences of visuo-motor action on crossmodal simultaneity Although crossmodal simultaneity has been intensively investigated (Dixon & Spitz, 1980;Fujisaki et al, 2004;Levitin et al, 2000;Spence et al, 2001;Stone et al, 2001;van Erp & Werkhoven, 2004;Vatakis & Spence, 2006), how crossmodal simultaneity is influenced by time delays and self-generated actions in multimodal human-system interfaces has thus far received only little attention. Recently, in an important study, Vogels (2004) examined the effects of time delay on visual-haptic asynchrony judgments, using a task that involved a visual object touching a virtual wall.…”

Section: Perception Of Simultaneity and Related Psychophysical Methodsmentioning

confidence: 99%

Temporal Perception of Visual-Haptic Events in Multimodal Telepresence System

Shi¹,

Zou²,

Müller³

2010

Advances in Haptics

View full text Add to dashboard Cite

Evaluating the influence of frame rate on the temporal aspects of audiovisual speech perception

Cited by 17 publications

References 24 publications

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Audiovisual robustness: exploring perceptual tolerance to asynchrony and quality distortion

Assessing the role of attention in the audiovisual integration of speech

Temporal Perception of Visual-Haptic Events in Multimodal Telepresence System

Contact Info

Product

Resources

About