Synchronization to auditory and visual rhythms in hearing and deaf individuals

Abstract

A striking asymmetry in human sensorimotor processing is that humans synchronize movements to rhythmic sound with far greater precision than to temporally equivalent visual stimuli (e.g., to an auditory vs. a flashing visual metronome). Traditionally, this finding is thought to reflect a fundamental difference in auditory vs. visual processing, i.e., superior temporal processing by the auditory system and/or privileged coupling between the auditory and motor systems. It is unclear whether this asymmetry is an inevitable consequence of brain organization or whether it can be modified (or even eliminated) by stimulus characteristics or by experience. With respect to stimulus characteristics, we found that a moving, colliding visual stimulus (a silent image of a bouncing ball with a distinct collision point on the floor) was able to drive synchronization nearly as accurately as sound in hearing participants. To study the role of experience, we compared synchronization to flashing metronomes in hearing and profoundly deaf individuals. Deaf individuals performed better than hearing individuals when synchronizing with visual flashes, suggesting that cross-modal plasticity enhances the ability to synchronize with temporally discrete visual stimuli. Furthermore, when deaf (but not hearing) individuals synchronized with the bouncing ball, their tapping patterns suggest that visual timing may access higher-order beat perception mechanisms for deaf individuals. These results indicate that the auditory advantage in rhythmic synchronization is more experience- and stimulus-dependent than has been previously reported.

Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis

“…We report two separate ANOVAs: Lag 1 autocorrelations and higher order autocorrelations (Lags 2-4). This is a convention used in other studies (see, e.g., Iversen et al, 2015) and is theoretically motivated-the negative autocorrelations found at Lag 1 are thought to reflect a different timing process than at higher order lags. A common observation in synchronized tapping with an auditory metronome is a negative autocorrelation at Lag 1, thought to reflect an error correction process (Repp, 2005).…”

“…To assess timing dependencies between intertap intervals (ITI), we examined the autocorrelation structure of each trial's time series (see Iversen et al, 2015, Section 2.4 for a discussion of this type of analysis of tapping data in relation to models of timekeeping). For each participant and each condition, autocorrelation coefficients were calculated at Lags 1 to 4 for the best trial as identified by the standard deviation of asynchronies.…”

Speed on the dance floor: Auditory and visual cues for musical tempo

“…Similarly, flashes do not give rise to a strong sense of beat (McAuley & Henry 2010), and different brain regions have been shown to be involved with discrete visual as opposed to discrete auditory stimuli (Grahn, Henry, & McAuley 2011;Hove, Fairhurst, Kotz, & Keller 2013). However, Hove, Iversen, Zhang, and Repp (2013) and Iversen, Patel, Nicodemus, & Emmorey (2015) have shown that when a continuous, colliding visual stimulus is used (i.e., a video animation of a bouncing ball) synchronization performance is nearly equivalent to that with discrete auditory tones. In another study of visual cues for beat and tempo Luck & Sloboda (2009) identified absolute acceleration as the most salient cue in synchronizing with a conductor's gesture.…”

“…However, other factors such as rhythmic density, mean rhythmic inter-onset interval, metrical (accentual) structure, and rhythmic complexity can affect perceived tempo (Drake, Gros, & Penel 1999;London 2011). Visual information can also give rise to a perceived beat/tempo (Iversen, et al 2015), and auditory and visual temporal cues can interact and mutually influence each other (Soto-Faraco & Kingston 2004;Spence 2015). A five-part experiment was performed to assess the integration of auditory and visual information in judgments of musical tempo.…”

The Evolution of Rhythm Cognition: Timing in Music and Speech

“…(1) developmental studies of rhythm that are useful in understanding whether rhythm perception and production involve critical acquisition periods, or instead result mostly from enculturation during the whole lifespan (Hannon and Trehub, 2005), (2) comparative and cross-cultural studies of rhythm that serve to explain whether musical enculturation or exposure to specific languages can affect which specific rhythmic patterns can be produced/perceived and how frequently (Greenberg et al, 1978;Rzeszutek et al, 2012), (3) comparisons of rhythm processing in music and speech, at both behavioral and neural levels that help understanding whether common music-speech networks exist and similar behavioral patterns can be observed when humans engage in music and speech production, (4) evidence and comparison of rhythm processing across modalities and domains that are used to understand whether, for instance, metrical expectation in speech is strictly bound to the speech domain or instead recruits the same capacities for metricality available in music, or even in dance and vision (Iversen et al, 2015), (5) studies of rhythm in interaction and context (Yu and Tomonaga, 2015), explaining how social, affective, and other factors affect the emergence of rhythmic patterns, (6) archaeological findings trying to reconstruct rhythmrelated behavior and cognition in our early hominid ancestors (Morley, 2003), (7) mathematical and computational models (e.g., connectionist, symbolic) of the mechanisms underlying perception and production of rhythmic behavior (Desain and Honing, 1989, 1991, 2003, (8) mathematical and computational models of rhythmic capacities as evolved behaviors (Miranda et al, 2003) in line with a long tradition in evolutionary and theoretical biology, (9) evidence of spontaneous rhythmic behavior in other animals (Fuhrmann et al, 2014;Ravignani et al, 2014a) showing how similar rhythmic traits can evolve via similar pressures in phylogenetically distant species, (10) controlled experiments in non-human animals (Cook et al, 2013) probing the potential for producing/perceiving rhythm (even though these are not usually part of these species' natural behavior); these experiments can show the existence of basic, evolutionary conserved cognitive processes that may have been exapted in humans for rhythmic purposes.…”