Real-Time Dance Generation to Music for a Legged Robot

Abstract

The development of robots that can dance has received considerable attention. However, they are often either limited to a pre-defined set of movements and music or demonstrate little variance when reacting to external stimuli, such as microphone or camera input. In this paper, we contribute with a novel approach allowing a legged robot to listen to live music while dancing in synchronization with the music in a diverse fashion. This is achieved by extracting the beat from an onboard microphone in real-time, and subsequently creating a dance choreography by picking from a user-generated dance motion library at every new beat. Dance motions include various stepping and base motions. The process of picking from the library is defined by a probabilistic model, namely a Markov chain, that depends on the previously picked dance motion and the current music tempo. Finally, delays are determined online by time-shifting a measured signal and a reference signal, and minimizing the least squares error with the time-shift as parameter. Delays are then compensated for by using a combined feedforward and feedback delay controller which shifts the robot whole-body controller reference input in time. Results from experiments on a quadrupedal robot demonstrate the fast convergence and synchrony to the perceived music.

Resonance as a Design Strategy for AI and Social Robots

“…They found that, when dancing in synchrony, children were more likely to socially interact and to do so for longer. While there are many dancing robots (reviewed in Bi et al, 2018 ), most involve pre-programmed motions that are unable to adjust to external visual or auditory stimuli. Responding in real-time to external motions (such as a human dancer) is often limited by hardware and software processing delays.…”

When Creators Meet the Metaverse: A Survey on Computational Arts

“…For example, Nakaoka et al [160] developed a bipedal robot that can imitate the movements of a human dancer of a Japanese folk dance. In addition, improvising dances allow dance robots to perform more human-like, including dance synchronised to music [22]. Moreover, the development of robots with more flexible mechanical frames [167] could allow robots to perform more complex motions since there are no joints in the robot structure like in a human body.…”

Interactive Multi-Robot Painting Through Colored Motion Trails

“…The intersection of robots and arts has become an active object of study as both researchers and artists push the boundaries of the traditional conceptions of different forms of art by making robotic agents dance (Nakazawa et al, 2002 ; LaViers et al, 2014 ; Bi et al, 2018 ), create music (Hoffman and Weinberg, 2010 ), support stage performances (Ackerman, 2014 ), create paintings (Lindemeier et al, 2013 ; Tresset and Leymarie, 2013 ), or become art exhibits by themselves (Dean et al, 2008 ; Dunstan et al, 2016 ; Jochum and Goldberg, 2016 ; Vlachos et al, 2018 ). On a smaller scale, the artistic possibilities of robotic swarms have also been explored in the context of choreographed movements to music (Ackerman, 2014 ; Alonso-Mora et al, 2014 ; Schoellig et al, 2014 ), emotionally expressive motions (Dietz et al, 2017 ; Levillain et al, 2018 ; St.-Onge et al, 2019 ; Santos and Egerstedt, 2020 ), or interactive music generation based on the interactions between agents (Albin et al, 2012 ), among others.…”