Percussionists inadvertently use visual information to strategically manipulate audience perception of note duration. Videos of long (L) and short (S) notes performed by a world-renowned percussionist were separated into visual (Lv, Sv) and auditory (La, Sa) components. Visual components contained only the gesture used to perform the note, auditory components the acoustic note itself. Audio and visual components were then crossed to create realistic musical stimuli. Participants were informed of the mismatch, and asked to rate note duration of these audio-visual pairs based on sound alone. Ratings varied based on visual (Lv versus Sv), but not auditory (La versus Sa) components. Therefore while longer gestures do not make longer notes, longer gestures make longer sounding notes through the integration of sensory information. This finding contradicts previous research showing that audition dominates temporal tasks such as duration judgment.

Here, we demonstrate that "moving to the beat" can improve the perception of timing, providing an intriguing explanation as to why we often move when listening to music. In the first experiment, participants heard a series of isochronous beats and identified whether the timing of a final tone after a short silence was consistent with the timing of the preceding sequence. On half of the trials, participants tapped along with the beat, and on half of the trials, they listened without moving. When the final tone occurred later than expected, performance in the movement condition was significantly better than performance in the no-movement condition. Two additional experiments illustrate that this improved performance is due to improved timekeeping, rather than to a shift in strategy. This work contributes to a growing literature on sensorimotor integration by demonstrating body movement's objective improvement in timekeeping, complementing previous explorations involving subjective tasks.

Schutz and Lipscomb (2007) reported an audiovisual illusion in which the length of the gesture used to produce a sound altered the perception of that sound's duration. This contradicts the widely accepted claim that the auditory system generally dominates temporal tasks because of its superior temporal acuity. Here, in the first of 4 experiments, we show that impact gestures influence duration ratings of percussive but not sustained sounds. In the 2nd, we show that the illusion is present even if the percussive sound occurs up to 700 ms after the visible impact, but disappears if the percussive sound precedes the visible impact. In the 3rd experiment, we show that only the motion after the visible impact influences perceived tone duration. The 4th experiment (replacing the impact gestures with the written text long and short) suggests that the phenomenon is not due to response bias. Given that visual influence in this paradigm is dependent on the presence of an ecologically plausible audiovisual relationship, we conclude that cross-modal causality plays a key role in governing the integration of sensory information.

In this paper we offer a theory of cross-modal objects. To begin, we discuss two kinds of linkages between vision and audition. The first is a duality. The the visual system detects and identifies surfaces; the auditory system detects and identifies sources. Surfaces are illuminated by sources of light; sound is reflected off surfaces. However, the visual system discounts sources and the auditory system discounts surfaces. These and similar considerations lead to the Theory of Indispensable Attributes that states the conditions for the formation of gestalts in the two modalities. The second linkage involves the formation of audiovisual objects, integrated cross-modal experiences. We describe research that reveals the role of cross-modal causality in the formation of such objects. These experiments use the canonical example of a causal link between vision and audition: a visible impact that causes a percussive sound.[A fire is] a terrestrial event with flames and fuel. It is a source of four kinds of stimulation, since it gives off sound, odor, heat and light . . . . One can hear it, smell it, feel it, and see it, or get any combination of these detections, and thereby perceive a fire . . . . For this event, the four kinds of stimulus information and the four perceptual systems are equivalent.

Many studies demonstrate that musicians exhibit superior timing abilities compared to nonmusicians. Here, we investigated how specific musical expertise can mediate the relationship between movement and timing perception. In the present study, a group of highly trained percussionists (n = 33) and a group of non-percussionists (n = 33) were tested on their ability to detect temporal deviations of a tone presented after an isochronous sequence. Participants either tapped along with the sequence using a drumstick (movement condition) or listened without tapping (no-movement condition). Although both groups performed significantly better when moving than when listening alone, percussionists gained a greater benefit from tapping when detecting the smallest probe tone delays compared to non-percussionists. This complements both the musical expertise and timing perception literature by demonstrating that percussionists with high levels of training may further capitalize on the benefits of sensorimotor interactions. Surprisingly, percussionists and non-percussionists performed no differently when listening alone, in contrast to other studies examining the role of training in timing abilities. This raises interesting questions about the degree to which percussionists' known expertise in timing may interact with their use of motion when judging rhythmic precision.