Causality and cross-modal integration.

Abstract

Schutz and Lipscomb (2007) reported an audiovisual illusion in which the length of the gesture used to produce a sound altered the perception of that sound's duration. This contradicts the widely accepted claim that the auditory system generally dominates temporal tasks because of its superior temporal acuity. Here, in the first of 4 experiments, we show that impact gestures influence duration ratings of percussive but not sustained sounds. In the 2nd, we show that the illusion is present even if the percussive sound occurs up to 700 ms after the visible impact, but disappears if the percussive sound precedes the visible impact. In the 3rd experiment, we show that only the motion after the visible impact influences perceived tone duration. The 4th experiment (replacing the impact gestures with the written text long and short) suggests that the phenomenon is not due to response bias. Given that visual influence in this paradigm is dependent on the presence of an ecologically plausible audiovisual relationship, we conclude that cross-modal causality plays a key role in governing the integration of sensory information.

Surveying the Temporal Structure of Sounds Used in Music Perception

“…This finding is consistent with previous research demonstrating that visual information does not affect auditory judgments of event duration (Walker & Scott, 1981). However, these same participants were unable to ignore visual information when judging the duration of percussive tones, a finding consistent with earlier sensory integration experiments involving percussive sounds (Schutz & Kubovy, 2009b;Schutz & Lipscomb, 2007). This finding suggests that patterns of integration widely agreed upon (previously assessed primarily with flat tones) might not apply to real world sounds with natural decays.…”

Striking movements: A survey of motion analysis of percussionists

“…One would perhaps expect the preparatory height to have an influence on the multisensory information. Interestingly enough, a later study showed the gesture after the actual impact to be more important for how long the notes were rated to be [22]. Thus, although a percussionist cannot prolong the played note as such, the long gesture after impact can still make it sound longer.…”

Multistability, cross-modal binding and the additivity of conjoined grouping principles

“…The perceived duration of the sound is longer if the gesture is long than if is short [18,19]. Schutz & Kubovy [18] demonstrated that this effect was due to cross-modal causality, by first showing that the effect depended on the percussive element of the sound. For example, when the sound associated with the long and short gestures (marimba or piano), the effect was present.…”