The impact of visual display of human motion on observers’ perception of music performance

Abstract

In investigating the influence of body movement in multimodal perception, human motion displays are frequently used as a means of visual standardization and control of external confounders. However, no principle is established regarding the selection of an adequate display for specific study purposes. The aim of this study was to evaluate the effects of adopting 4 visual displays (point-light, stick figure, body mass, skeleton) on the observers’ perception of music performances in 2 expressive conditions (immobile, projected expressiveness). Two hundred eleven participants rated 8 audio-visual samples in expressiveness, match between movement and music, and overall evaluation. The results revealed significant isolated main effects of visual display and expressive condition on the observers’ ratings (in both, p < 0.001), and interaction effects between the two factors (p < 0.001). Displays closer to a human form (mostly skeleton, sometimes body mass) exponentiated the evaluations of expressiveness and music-movement match in the projected expressiveness condition, and of overall evaluation in the immobile condition; the opposite trend occurred with the simplified motion display (stick figure). Projected expressiveness performances were higher rated than immobile performances. Although the expressive conditions remained distinguishable across displays, the more complex ones potentiated the attribution of subjective qualities. We underline the importance of considering the variable display as an influencing factor in perceptual studies.

Increased body movement equals better performance? Not always! Musical style determines motion degree perceived as optimal in music performance

“…Bugaj et al, 2019 ; Davidson, 1993 ; Nusseck & Wanderley, 2009 , see Introduction for more details). However, given that most studies use two (Moura et al, 2023 ; Nápoles et al, 2022 ) or three motion styles (Silveira, 2014 ; Van Zijl & Luck, 2013 ), it is natural that participants incline towards upvoting the condition with larger movement and vice versa. To better understand this phenomenon, we expanded the number of motion degrees included in the design (five for each of the two musical excerpts).…”

“…Body behaviour is the major component of the visual dimension in music performance. Studies focussed on the role of visuo-kinematic cues in audience perception have addressed two complementary views: full-body motion, tackling musicians’ movement as a general concept involving the body as a whole (Bugaj et al, 2019 ; Burger & Wöllner, 2023 ; Davidson, 1993 ; Moura et al, 2023 ; Nápoles et al, 2022 ; Nusseck & Wanderley, 2009 ; Trevor & Huron, 2018 ; Van Zijl & Luck, 2013 ), and motion types, providing a detailed view on specific gestures or body locations (Davidson, 1994 ; Nusseck & Wanderley, 2009 ; Weiss et al, 2018 ).…”

“…In the full-body domain, the common finding between studies is that, across several assessment variables, participants give better ratings to performative conditions with increased amounts of movement (Broughton & Stevens, 2009 ; Bugaj et al, 2019 ; Burger & Wöllner, 2023 ; Davidson, 1993 ; Grady & Gilliam, 2020 ; Juchniewicz, 2008 ; Moura et al, 2023 ; Nápoles et al, 2022 ; Nusseck & Wanderley, 2009 ; Silveira, 2014 ; Van Zijl & Luck, 2013 ). Accordingly, Küssner and Colleagues ( 2020 ) demonstrated that visual attention is directed towards the musicians moving more expressively, even if they are not playing the most relevant musical part.…”

“…Aspects like tonality (López & Anta, 2023 ) or rhythm (Senn et al, 2017 ; Varlet et al, 2020 ) have a significant influence on listeners. Nonetheless, by adopting designs including one musical piece (Moura et al, 2023 ; Nápoles et al, 2022 ; Nusseck & Wanderley, 2009 ; Weiss et al, 2018 ) or grouping a large number of excerpts together in the same analysis (Küssner et al, 2020 ; Lange et al, 2022 ), studies are unable to address the influence of the repertoire. Conversely, an effect of musical style has been reported in studies accounting for this variable (Burger & Wöllner, 2023 ; Huang & Krumhansl, 2011 ; Shoda & Adachi, 2016 ; Wapnick et al, 2004 ).…”

Knee flexion of saxophone players anticipates tonal context of music

“…In cognitive linguistics, the power of nonverbal behavior in communicative tasks is not limited to speech comprehension and production 12 – 16 , but expanded as a tool for enhancing learning processes 14 , 17 – 20 . In music research, there is strong evidence that musicians’ body movements affect audience perception 5 , 21 , 22 and reflect features of the musical speech 10 , 23 , 24 . However, the knowledge about the facilitative role of gestures in music playing is scarce.…”