Effects of stimulus duration on audio-visual synchrony perception

Abstract

The integration of visual and auditory inputs in the human brain occurs only if the components are perceived in temporal proximity, that is, when the intermodal time difference falls within the so-called subjective synchrony range. We used the midpoint of this range to estimate the point of subjective simultaneity (PSS). We measured the PSS for audio-visual (AV) stimuli in a synchrony judgment task, in which subjects had to judge a given AV stimulus using three response categories (audio first, synchronous, video first). The relevant stimulus manipulation was the duration of the auditory and visual components. Results for unimodal auditory and visual stimuli have shown that the perceived onset shifts to relatively later positions with increasing stimulus duration. These unimodal shifts should be reflected in changing PSS values, when AV stimuli with different durations of the auditory and visual components are used. The results for 17 subjects showed indeed a significant shift of the PSS for different duration combinations of the stimulus components. Because the shifts were approximately equal for duration changes in either of the components, no net shift of the PSS was observed as long as the durations of the two components were equal. This result indicates the need to appropriately account for unimodal timing effects when quantifying intermodal synchrony perception.

Quantifying temporal ventriloquism in audiovisual synchrony perception

“…For example, the asynchronies found in synchrony judgment tasks in single-event experiments are comparable to the asymmetry of the cubic function used to describe our results. Comparing our data with the mean synchrony perception curves (Kuling et al, 2012) shows synchrony response probabilities of about 27% for the two SOAs corresponding to the two maxima of the absolute shift. This would suggest that the influence of the auditory distractors exceeds the typical synchrony range (i.e., the range between the 50% points) used in synchrony judgment tasks.…”

“…The auditory target between visual markers was placed later in time than at the bisection of the temporal positions of the second and fourth markers, and the visual target between auditory markers was placed earlier in time than at the bisection of the temporal positions of the second and fourth markers. The size of this relative delay was found to be about 25-30 ms, which is slightly larger than the delay between the auditory and visual stimuli that led to the strongest synchrony percept (10-15 ms) in previous experiments with the same stimuli (Kuling, Van Eijk, Juola, & Kohlrausch, 2012;Van Eijk et al, 2008). In our previous experiments, the methodology involved synchrony perception, in which bimodal stimuli were judged to be synchronous over a range of SOAs around the point of physical synchrony.…”

“…In a more detailed analysis, we compared the SOA leading to the maximum and the minimum values of the overall cubic function with the synchrony curves from earlier experiments with the same short auditory and visual stimuli (Kuling et al, 2012). As Fig.…”

“…For the second fit (dashed curve in Fig. 5), the probability function (parameters b and c, b = 0.013 + d/a, c = 0.117) was derived from the synchrony curves in a synchrony judgment task with the same stimuli (from Kuling et al, 2012), so that only two parameters, a and d, were optimized from the present data set (R 2 = .91; fit parameters: a = 0.184, d = −0.018). Very similar On the left side, the minimum value corresponds to an SOA leading to a synchrony response probability of 27.0%; on the right side, the maximum value corresponds to a synchrony response probability of 27.9%.…”

Larger Stimuli Require Longer Processing Time for Perception

“…Stimulus duration in turn could affect the timing of onset because of the known effect that the perceived onset of a stimulus is delayed for a longer lasting stimulus (e.g., Jaskowski, 1991;Kuling, van Eijk, Juola, & Kohlrausch, 2012). These combined effects could explain the size effect reported in the present study.…”