A robot uses its own microphone to synchronize its steps to musical beats while scatting and singing

Abstract

Abstract-Musical beat tracking is one of the effective technologies for human-robot interaction such as musical sessions. Since such interaction should be performed in various environments in a natural way, musical beat tracking for a robot should cope with noise sources such as environmental noise, its own motor noises, and self voices, by using its own microphone. This paper addresses a musical beat tracking robot which can step, scat and sing according to musical beats by using its own microphone. To realize such a robot, we propose a robust beat tracking method by introducing two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self noises such as stepping, scatting, and singing. We implemented the proposed beat tracking method for Honda ASIMO. Experimental results showed ten times faster adaptation to tempo changes and high robustness in beat tracking for stepping, scatting and singing noises. We also demonstrated the robot times its steps while scatting or singing to musical beats.

Music-Ensemble Robot That Is Capable of Playing the Theremin While Listening to the Accompanied Music

“…Especially, self-generated sound such as robot's own voice that is mixed into the music sound robot hears has been revealed to affect the quality of subsequent synchronization algorithm [6] [7].…”

“…The clean signal from the theremin is easily acquired because theremin directly generates an electric waveform. The second synchronization function is a tempo-adaptive beat tracking called spectro-temporal pattern matching [7]. The algorithm consists of tempo estimation, beat detection, and beat time prediction.…”

Temporal scaling of leg motion for music feedback system of a dancing humanoid robot

“…Simple temporal scaling often causes movements that exceed joint limitations of robots or unstable motions which fail to maintain balance. Yoshii et al and Murata et al proposed a method that enables a biped humanoid robot to step with synchronization to musical tempo [10], [11]. However, their target motion is simple stepping and complicated motion like dance cannot be handled.…”

Real-Time Audio-to-Score Alignment Using Particle Filter for Coplayer Music Robots

“…Since most existing music robots that pay attention to the onset of a human's musical performance have focused on the rhythm level, their musical expressions are limited to repetitive or random expressions such as drumming [4], shaking their body [5], stepping, or scatting [6,7]. Pan et al developed a humanoid robot system that plays the vibraphone based on visual and audio cues [8].…”