Dancing‐to‐Music Character Animation

Abstract

In computer graphics, considerable research has been conducted on realistic human motion synthesis. However, most research does not consider human emotional aspects, which often strongly affect human motion. This paper presents a new approach for synthesizing dance performance matched to input music, based on the emotional aspects of dance performance. Our method consists of a motion analysis, a music analysis, and a motion synthesis based on the extracted features. In the analysis steps, motion and music feature vectors are acquired. Motion vectors are derived from motion rhythm and intensity, while music vectors are derived from musical rhythm, structure, and intensity. For synthesizing dance performance, we first find candidate motion segments whose rhythm features are matched to those of each music segment, and then we find the motion segment set whose intensity is similar to that of music segments. Additionally, our system supports having animators control the synthesis process by assigning desired motion segments to the specified music segments. The experimental results indicate that our method actually creates dance performance as if a character was listening and expressively dancing to the music.

Efficient Generation of Dancing Animation Synchronizing with Music Based on Meta Motion Graphs

“…To enrich the experience of music, automatic generation of dancing animation is attracting ever greater attention, where the dancing motion is concatenated to synchronize with the music by re-using motion capture data [1]- [3]. Basically, these techniques are inspired by the phenomenon of people wanting to dance to music.…”

“…Basically, these techniques are inspired by the phenomenon of people wanting to dance to music. Regarding people's dancing to music, one of the features common to both dance and music is the rhythmic structure [4], [5], thus this feature is commonly used in music synchronization systems [1]- [3]. Obviously, rhythmic structure provides the basis for synchronization in the temporal domain.…”

“…Obviously, rhythmic structure provides the basis for synchronization in the temporal domain. Moreover, the matching of intensity between motion and music is considered to be the next most important feature in Shiratori's system [3] by Laban theory [6], which is related to the spatial domain. As pointed out by Shiratori et al [3], it is derived from the fact that people feel quiet and relaxed when listening to relaxing music such as a ballad, and they feel excited when listening to intense music such as hard rock music.…”

A parameterizable spatiotemporal representation of popular dance styles for humanoid dancing characters

“…The resulting key-poses were clustered and interpolated a posteriori for generating variations of the captured dance. Similarly, [9] extracted motion key-poses by means of motion rhythm and intensity features calculated from local minima in the motion signal (stop motions), which were based on the Laban's concept of "weight effort" [10]. On the other hand, [4,11] generated rhythmic motion patterns, such as dancing and locomotion, by clustering and interpolating unlabeled MoCap segments as "motion beats".…”

Dance-the-Music: an educational platform for the modeling, recognition and audiovisual monitoring of dance steps using spatiotemporal motion templates

“…music annotation/analysis [33][34][35][36]. The opposite approach, namely generating music by automatic dance analysis, is explored in the domain of gesture-based human-computer interaction [37][38][39] and music information retrieval [10].…”