Communication for coordination: gesture kinematics and conventionality affect synchronization success in piano duos

Abstract

Ensemble musicians often exchange visual cues in the form of body gestures (e.g., rhythmic head nods) to help coordinate piece entrances. These cues must communicate beats clearly, especially if the piece requires interperformer synchronization of the first chord. This study aimed to (1) replicate prior findings suggesting that points of peak acceleration in head gestures communicate beat position and (2) identify the kinematic features of head gestures that encourage successful synchronization. It was expected that increased precision of the alignment between leaders’ head gestures and first note onsets, increased gesture smoothness, magnitude, and prototypicality, and increased leader ensemble/conducting experience would improve gesture synchronizability. Audio/MIDI and motion capture recordings were made of piano duos performing short musical passages under assigned leader/follower conditions. The leader of each trial listened to a particular tempo over headphones, then cued their partner in at the given tempo, without speaking. A subset of motion capture recordings were then presented as point-light videos with corresponding audio to a sample of musicians who tapped in synchrony with the beat. Musicians were found to align their first taps with the period of deceleration following acceleration peaks in leaders’ head gestures, suggesting that acceleration patterns communicate beat position. Musicians’ synchronization with leaders’ first onsets improved as cueing gesture smoothness and magnitude increased and prototypicality decreased. Synchronization was also more successful with more experienced leaders’ gestures. These results might be applied to interactive systems using gesture recognition or reproduction for music-making tasks (e.g., intelligent accompaniment systems).

Shared periodic performer movements coordinate interactions in duo improvisations

“…Studying the synchronization of note onsets between the performers would be a natural extension of the study, to link the factors contributing to musical synchronization with the cues used to coordinate interaction [ 23 ]. For instance, Bishop & Goebl [ 96 ] have found that note-level synchrony is related to kinematic features of communicative head gestures. Such a shift to a lower temporal level (e.g.…”

Collaborative Musical Creativity: How Ensembles Coordinate Spontaneity

“…Communication between performers is thought to support action prediction processes by providing cues to initiate the simulation process. While auditory communication in the form of a musical sound signal is usually sufficient for performers to maintain temporal coordination, they sometimes supplement their audio signals with visual signals (Badino et al, 2014 ; Kawase, 2014 ; Bishop and Goebl, 2017 ). Ensemble musicians are better able to predict the course of observed gestures when those gestures fall within their practiced repertoire (Wöllner and Cañal-Bruland, 2010 ; Bishop and Goebl, 2014 ), and better able to predict such gestures than are novice musicians (Luck and Nte, 2008 ; Petrini et al, 2009 ; Lee and Noppeney, 2014 ).…”

“…Breathing gestures are likely used as well and have the benefit of providing an audiovisual cue, but they are more difficult to measure experimentally. Cueing gestures can be exchanged at moments of sudden tempo or meter change (Kawase, 2014 ) or at piece entrances or re-entrances that require synchronization between performers (Bishop and Goebl, 2015 , 2017 ). These are ambiguous, isolated moments when co-performer's expectations about how to play might not otherwise align.…”

Eye gaze as a means of giving and seeking information during musical interaction

“…During ensemble performance, visual communication between musicians is secondary in importance to auditory communication, but can become more relevant when performers are uncertain of each other’s interpretations ( Bishop & Goebl, 2015 ). Prior research has shown that temporal instability, such as happens at piece entrances, long pauses, and sudden tempo changes ( Bishop and Goebl, 2017 , Davidson, 2012 , Kawase, 2014a , Kawase, 2014b ), and disruptions to the clarity (e.g., loudness) of inter-performer audio feedback ( Fulford, Hopkins, Seiffert, & Ginsborg, 2018 ) can prompt an exchange of visual cues. These are sometimes deliberately integrated into a performance plan across rehearsals ( Williamon & Davidson, 2002 ).…”