Perception of ‘Back-Channeling’ Nonverbal Feedback in Musical Duo Improvisation

Abstract

In witnessing face-to-face conversation, observers perceive authentic communication according to the social contingency of nonverbal feedback cues (‘back-channeling’) by non-speaking interactors. The current study investigated the generality of this function by focusing on nonverbal communication in musical improvisation. A perceptual experiment was conducted to test whether observers can reliably identify genuine versus fake (mismatched) duos from musicians’ nonverbal cues, and how this judgement is affected by observers’ musical background and rhythm perception skill. Twenty-four musicians were recruited to perform duo improvisations, which included solo episodes, in two styles: standard jazz (where rhythm is based on a regular pulse) or free improvisation (where rhythm is non-pulsed). The improvisations were recorded using a motion capture system to generate 16 ten-second point-light displays (with audio) of the soloist and the silent non-soloing musician (‘back-channeler’). Sixteen further displays were created by splicing soloists with back-channelers from different duos. Participants (N = 60) with various musical backgrounds were asked to rate the point-light displays as either real or fake. Results indicated that participants were sensitive to the real/fake distinction in the free improvisation condition independently of musical experience. Individual differences in rhythm perception skill did not account for performance in the free condition, but were positively correlated with accuracy in the standard jazz condition. These findings suggest that the perception of back-channeling in free improvisation is not dependent on music-specific skills but is a general ability. The findings invite further study of the links between interpersonal dynamics in conversation and musical interaction.

Communication for coordination: gesture kinematics and conventionality affect synchronization success in piano duos

“…Usually, perception of jointly-produced sound gives sufficient information for performers to coordinate, but visual communication can be important too (Bishop & Goebl, 2015 ). Visual communication is only rarely a matter of one performer giving directions to another; rather, even if there is a designated leader, collaborating musicians’ body movements interrelate (Chang, Livingstone, Bosnyak, & Trainor, 2017 ; Moran, Hadley, Bader, & Keller, 2015 ) and can be mutually influential (Badino, D’Ausilio, Glowinski, Camurri, & Fadiga, 2014 ). Research has shown that musicians move more predictably when performing with others than when performing alone (Glowinski et al., 2013 ), a finding that parallels observations made elsewhere in the joint action literature (Hart, Noy, Feniger-Schaal, Mayo, & Alon, 2014 ; Vesper, van der Wel, Knoblich, & Sebanz, 2011 ).…”

Determining the end of a musical turn: Effects of tonal cues

“…Musicians indeed seem able to take turns in such music, implying that it is possible to make predictions about when a turn is about to end (Moran et al, 2015;MacDonald et al, 2012). We suggest that in such music, which does not use a clear auditory structural framework (such as tonality or metre), visual cues such as body movement may become critical.…”

“…We suggest that in such music, which does not use a clear auditory structural framework (such as tonality or metre), visual cues such as body movement may become critical. This speculation is supported by the study of Moran et al (2015), in which participants successfully identified authentic pairs of free improvisers from point-light audio-visual displays of the performers' body movement, but were not able to reliably identify equivalent displays of authentic standard jazz improvisers. An alternative explanation, however, relates to the pervasiveness of jazz improvisation compared to free improvisation in everyday life.…”

Extracting Coarse Body Movements from Video in Music Performance: A Comparison of Automated Computer Vision Techniques with Motion Capture Data

“…The second dataset consisted of three performances by jazz duos, a subset of the Improvising Duos corpus described in Moran et al (2015). In this subset (hereafter referred to as "Mixed Instrument Duos"), two duos performed free jazz improvisations and one performed a jazz standard [Autumn Leaves (Kosma, 1945)].…”

“…As such, if video-based analysis methods prove to be fruitful in providing new insights about musical interaction, a large amount of useful research could be done that makes use of such existing video archives (with the appropriate permissions and taking account of ethical considerations), which could thereby minimize the costs that are necessarily incurred when collecting new data. Our study served as a test case in this regard, as it also made use of existing data-in this case, three datasets in which both video and MoCap recordings had been collected [as reported in Glowinski et al (2013) and Moran et al (2015), and one previously unpublished dataset]. This study was therefore able to test whether computer vision techniques could be used to quantify body movements from video recordings that had originally been obtained for other research purposes.…”