Towards a Multimodal Repository of Expressive Movement Qualities in Dance

Piana, Stefano; Coletta, Paolo E.; Ghisio, Simone; Niewiadomski, Radosław; Mancini, Maurizio; Sagoleo, Roberto; Volpe, Gualtiero; Camurri, Antonio

doi:10.1145/2948910.2948931

Cited by 8 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each trial had a duration of 1.5 to 2 minutes. At the beginning of each session, dancers were given definitions of the expressive quality by means of textual images (more details on the recording procedure are available in [17]). The dancers were asked to perform: (i) an improvised choreography containing movements that, in their opinion, express the quality convincingly, as well as (ii) several repetitions of predefined sequences of movements by focusing on the given expressive quality.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Using the Audio Respiration Signal for Multimodal Discrimination of Expressive Movement Qualities

Lussu

Niewiadomski

Volpe

et al. 2016

Human Behavior Understanding

Self Cite

View full text Add to dashboard Cite

In this paper we propose a multimodal approach to distinguish between movements displaying three different expressive qualities: fluid, fragmented, and impulsive movements. Our approach is based on the Event Synchronization algorithm, which is applied to compute the amount of synchronization between two low-level features extracted from multimodal data. In more details, we use the energy of the audio respiration signal captured by a standard microphone placed near to the mouth, and the whole body kinetic energy estimated from motion capture data. The method was evaluated on 90 movement segments performed by 5 dancers. Results show that fragmented movements display higher average synchronization than fluid and impulsive movements.

show abstract

Section: Methodsmentioning

confidence: 99%

“…When she perceived a touch on her body, she had to imagine that she was touched by a hot stick that she had to avoid. Thus, for impulsive trials, the dancer was, by default, performing fluid movements and impulsive movements appears only when she is touched (for more details see [17]). Each quality was performed by two different dancers.…”

Section: Methodsmentioning

confidence: 99%

Using the Audio Respiration Signal for Multimodal Discrimination of Expressive Movement Qualities

Lussu

Niewiadomski

Volpe

et al. 2016

Human Behavior Understanding

Self Cite

View full text Add to dashboard Cite

show abstract

“…This last category is more loosely defined, as it refers to a portion of corpus determined by an arbitrary set of morphological characteristics. Analysis of multimodal corpora has been employed for studying several aspects of embodied expressive performance, including interactive postural analysis of violin players (Volta and Volpe, 2019), embodied interaction between humans in virtual environments , and expressive movement qualities in dance (Piana et al, 2016a).…”

Section: Multimodal Music Representation and Analysismentioning

confidence: 99%

Method Development for Multimodal Data Corpus Analysis of Expressive Instrumental Music Performance

Visi

Östersjö

et al. 2020

Front. Psychol.

View full text Add to dashboard Cite

Musical performance is a multimodal experience, for performers and listeners alike. This paper reports on a pilot study which constitutes the first step toward a comprehensive approach to the experience of music as performed. We aim at bridging the gap between qualitative and quantitative approaches, by combining methods for data collection. The purpose is to build a data corpus containing multimodal measures linked to high-level subjective observations. This will allow for a systematic inclusion of the knowledge of music professionals in an analytic framework, which synthesizes methods across established research disciplines. We outline the methods we are currently developing for the creation of a multimodal data corpus dedicated to the analysis and exploration of instrumental music performance from the perspective of embodied music cognition. This will enable the study of the multiple facets of instrumental music performance in great detail, as well as lead to the development of music creation techniques that take advantage of the cross-modal relationships and higher-level qualities emerging from the analysis of this multi-layered, multimodal corpus. The results of the pilot project suggest that qualitative analysis through stimulated recall is an efficient method for generating higher-level understandings of musical performance. Furthermore, the results indicate several directions for further development, regarding observational movement analysis, and computational analysis of coarticulation, chunking, and movement qualities in musical performance. We argue that the development of methods for combining qualitative and quantitative data are required to fully understand expressive musical performance, especially in a broader scenario in which arts, humanities, and science are increasingly entangled. The future work in the project will therefore entail an increasingly multimodal analysis, aiming to become as holistic as is music in performance.

show abstract

“…In several applications the analysis on different types of recordings (audio, video, MoCap) of the same performance (multimodal analysis) allows to achieve better result due to the variety of information that it is possible to collect from the signals [2][10] [11] [12]. In order to perform such multimodal analysis, it is crucial the signals to be synchronized [11][13] [14] [15].…”

Section: Introductionmentioning

confidence: 99%

“…Typically, the alignment of the signals is performed either before or after the data acquisition. In the case the alignment is performed before the acquisition, a common clock signal is delivered across the involved acquisition devices in order to have a common time reference [12]. In the case the alignment is performed after the acquisition, the recordings are manually annotated with the time offset, based on some kind of visual or audio cues.…”

Section: Introductionmentioning

confidence: 99%

Using multi-dimensional correlation for matching and alignment of MoCap and video signals

Buccoli

Giorgi

Zanoni

et al. 2017

2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP)

View full text Add to dashboard Cite

Abstract-Motion analysis and tracking often relies on multimodal signals, e.g., video, depth map, motion capture (MoCap), due to the completeness of information they jointly provide. The joint analysis of multimodal signals requires to know the correct timing, i.e., the signals to be aligned. In this paper we propose an approach to automatically estimate the correct matching and alignment between a video and a MoCap recording acquired from the same session, based on the multi-dimensional correlation of velocity-based features extracted from the two recordings. We validate our approach over a dataset of dance recordings of four genres, and we achieve promising results for both the alignment and matching scenarios.

show abstract

Towards a Multimodal Repository of Expressive Movement Qualities in Dance

Cited by 8 publications

References 18 publications

Using the Audio Respiration Signal for Multimodal Discrimination of Expressive Movement Qualities

Using the Audio Respiration Signal for Multimodal Discrimination of Expressive Movement Qualities

Method Development for Multimodal Data Corpus Analysis of Expressive Instrumental Music Performance

Using multi-dimensional correlation for matching and alignment of MoCap and video signals

Contact Info

Product

Resources

About