See what I hear? Beat perception in auditory and visual rhythms

Abstract

Our perception of time is affected by the modality in which it is conveyed. Moreover, certain temporal phenomena appear to exist in only one modality. The perception of temporal regularity or structure (e.g., the 'beat') in rhythmic patterns is one such phenomenon: visual beat perception is rare. The modality-specificity for beat perception is puzzling, as the durations that comprise rhythmic patterns are much longer than the limits of visual temporal resolution. Moreover, the optimization that beat perception provides for memory of auditory sequences should be equally relevant to visual sequences. Why does beat perception appear to be modality specific? One possibility is that the nature of the visual stimulus plays a role. Previous studies have usually used brief stimuli (e.g., light flashes) to present visual rhythms. In the current study, a rotating line that appeared sequentially in different spatial orientations was used to present a visual rhythm. Discrimination accuracy for visual rhythms and auditory rhythms was compared for different types of rhythms. The rhythms either had a regular temporal structure that previously has been shown to induce beat perception in the auditory modality, or they had an irregular temporal structure without beat-inducing qualities. Overall, the visual rhythms were discriminated more poorly than the auditory rhythms. The beat-based structure, however, increased accuracy for visual as well as auditory rhythms. These results indicate that beat perception can occur in the visual modality and improve performance on a temporal discrimination task, when certain types of stimuli are used.

Neural Entrainment to the Rhythmic Structure of Music

“…The on-the-beat and off-the-beat conditions, therefore, are analogous to the strongly metrical and weakly metrical sequences (Povel & Essens, 1985) that have been used to study the effects of metrical strength on behavioral performance. Metrical strength has been linked to improved duration discrimination performance (Grahn, 2012;Grube & Griffiths, 2009), less variable beat synchronization (Patel, Iversen, & Chen, 2005), and more accurate rhythm reproduction (Fitch & Rosenfeld, 2007). Our results suggest an explanation for why metrically strong sequences are easier to discriminate and remember: metrically strong sequences enable metrical tracking on multiple timescales simultaneously, whereas metrically weak sequences disrupt beat subdivision.…”

Synchronization to auditory and visual rhythms in hearing and deaf individuals

“…Further study of naturally hierarchical visual stimuli in hearing individuals, such as musical conducting gestures (Luck & Sloboda, 2009), dance movements (Stevens, Schubert, Wang, Kroos, & Halovic, 2009), or point-light images of biological motion (Su, 2014) could reveal whether access to metrical processing advantages is more modality-independent than previously believed. Along these lines, Grahn (2012) has suggested that for hearing individuals perception of a step-wise rotating visual bar shows evidence of metricality.…”

Quantifying temporal ventriloquism in audiovisual synchrony perception

“…By using unimodal markers and targets in the same or different modalities, the precision of the procedure and potential temporal offsets (reflecting at least the relative processing times necessary to perceive auditory vs. visual stimuli) should be found. We expected that the precision of the aligned target position would increase with the number of auditory stimuli (marker or target), because it is reportedly more difficult to recognize visual stimulus rhythms than auditory ones (Glenberg & Jona, 1991;Grahn, 2012;Guttman, Gilroy, & Blake, 2005;Repp & Penel, 2002). Since visual rhythmic patterns activate the auditory cortex (Guttman et al, 2005), the investigation of rhythmic perception of mixed auditory and visual stimuli appears worthwhile.…”