Does embodied training improve the recognition of mid-level expressive movement qualities sonification?

Abstract

This research is a part of a broader project exploring how movement qualities can be recognized by means of the auditory channel: can we perceive an expressive full-body movement quality by means of its interactive sonification? The paper presents a sonification framework and an experiment to evaluate if embodied sonic training (i.e., experiencing interactive sonification of your own body movements) increases the recognition of such qualities through the auditory channel only, compared to a non-embodied sonic training condition. We focus on the sonification of two mid-level movement qualities: fragility and lightness. We base our sonification models, described in the first part, on the assumption that specific compounds of spectral features of a sound can contribute to the cross-modal perception of a specific movement quality. The experiment, described in the second part, involved 40 participants divided into two groups (embodied sonic training vs. no training). Participants were asked to report the level of lightness and fragility they perceived in 20 audio stimuli generated using the proposed sonification models. Results show that (1) both expressive qualities were correctly recognized from the audio stimuli, (2) a positive effect of embodied sonic training was observed for fragility but not for lightness. The paper is concluded by the description of the artistic performance that took place in 2017 in Genoa (Italy), in which the outcomes of the presented experiment were exploited. KeywordsSonification • Expressive qualities • Lightness • Fragility • Movement qualities • Embodied training M. Mancini participated to this work while being member of the Casa Paganini-

Dance Is More Than Meets the Eye—How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

“…Sonification also offers a wide range of options for making dance audible: movement data that is collected using optical motion capture systems or sensors attached directly to the body can be converted into acoustic signals or sounds (Niewiadomski et al, 2019;Giomi, 2020). What makes this technology particularly attractive is that sonification can be applied in real time, so the acoustic representation of the desired information can be fed back into the ongoing action as a stimulus.…”

“…Afterwards, all dancers stated that they enjoyed controlling sound through movement and that they felt encouraged to try new things, however, it did not improve movement comprehension or mutual understanding. Examples of the sonification of dance movement conveying different qualities (lightness, fragility) are provided as video material by Niewiadomski et al (2019).…”

“…Related questions have recently been addressed in an international project that aimed at developing technical systems for the real-time analysis and sonification of movement qualities for persons with visual impairment (http://dance.dibris.unige.it/). As part of this project, sonification of two defined movement qualities was developed and evaluated in an interactive dance scenario with participants with and without visual impairment (Niewiadomski et al, 2019). Results showed that both qualities were recognized from sonification only, and the authors planned to extend their work to include more movement qualities.…”

Introduction to the special issue on interactive sonification

“…al. studied the recognition of movement qualities by means of auditory display [4]. In their study, they sonified movement fragility and lightness and compared them in an embodied and non-embodied condition.…”

Gestural similarity, mathematics, psychology: Hints from a first experiment and some applications between pedagogy and research

“…Several scholars investigated analogies between images, movement, and music. It has been analyzed how dancers can gesturally render abstract concepts such as "fragility" and "lightness", and if and how the audience can recognize these concepts (Niewiadomski et al, 2018). Leech-Wilkinson (2017) focus on "shape" and shape-related metaphors, describing dynamics, pitch contour, texture, and so on.…”