Priming Gestures with Sounds

Abstract

We report a series of experiments about a little-studied type of compatibility effect between a stimulus and a response: the priming of manual gestures via sounds associated with these gestures. The goal was to investigate the plasticity of the gesture-sound associations mediating this type of priming. Five experiments used a primed choice-reaction task. Participants were cued by a stimulus to perform response gestures that produced response sounds; those sounds were also used as primes before the response cues. We compared arbitrary associations between gestures and sounds (key lifts and pure tones) created during the experiment (i.e. no pre-existing knowledge) with ecological associations corresponding to the structure of the world (tapping gestures and sounds, scraping gestures and sounds) learned through the entire life of the participant (thus existing prior to the experiment). Two results were found. First, the priming effect exists for ecological as well as arbitrary associations between gestures and sounds. Second, the priming effect is greatly reduced for ecologically existing associations and is eliminated for arbitrary associations when the response gesture stops producing the associated sounds. These results provide evidence that auditory-motor priming is mainly created by rapid learning of the association between sounds and the gestures that produce them. Auditory-motor priming is therefore mediated by short-term associations between gestures and sounds that can be readily reconfigured regardless of prior knowledge.

The Egocentric Nature of Action-Sound Associations

“…The resulting relative RT is the average RT from trials with a prime for a given gesture and a given prime minus the RT for the baseline for the same gesture plus the average RT for any prime. This transformation allowed our analysis to be consistent with our previous research ( Lemaitre et al, 2015 ). Note that, by definition, relative RT and raw RT produce the same statistics for the congruency variable (which was the main variable of interest).…”

“… Castiello et al (2010) showed that playing a priming sound before grasping an object sped up the execution of the grasping motion if the priming sound was the same as the sound produced by grasping the object. We recently performed a related series of experiments ( Heller et al, 2012 ; Lemaitre et al, 2015 ), but with a paradigm that measured reaction time to cues that prompted different gestures. Participants were cued to initiate one of two gestures (e.g., tapping or scraping).…”

Affordances after spinal cord injury

“…As an example, through this skill, one can observe a cup and immediately 'implicitly know' if it graspable or not. This concept has been defined as "micro-affordances" (Faber, van Elk, & Jonas, 2016;Freeman, Itthipuripat, & Aron, 2016;Lemaitre, Heller, Navolio, & Z uñiga-Peñaranda, 2015;Tucker & Ellis, 2004). Approximately 25% of rehabilitated patients with spinal cord injuries do not become independent ambulators (Wirz et al, 2005).…”

Timbre, Sound Quality, and Sound Design

“…Lemaitre et al (2009) have shown in particular that a continuous sonic feedback, reacting in real time to the users' gestures, can help them learn a fine gesture more rapidly than does visual feedback. Lemaitre et al (2015b) also have shown that playing the sound of an action (e.g., tapping, scraping) can facilitate (when the action is congruent with the sound) or hinder (when incongruent) the subsequent execution of another action.…”