An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds

Abstract

This paper describes a real-time beat tracking system that recognizes a hierarchical beat structure comprising the quarter-note, half-note, and measure levels in real-world audio signals sampled from popular-music compact discs. Most previous beat-tracking systems dealt with MIDI signals and had difficulty in processing, in real time, audio signals containing sounds of various instruments and in tracking beats above the quarter-note level. The system described here can process music with drums and music without drums and can recognize the hierarchical beat structure by using three kinds of musical knowledge: of onset times, of chord changes, and of drum patterns. This paper also describes several applications of beat tracking, such as beat-driven real-time computer graphics and lighting control. IntroductionThe goal of this study is to build a real-time system that can track musical beats in real-world audio signals, such as those sampled from compact discs. I think that building such a system that even in its preliminary implementation can work in real-world environments is an important initial step in the computational modeling of music understanding. This is because, as known from the scaling-up problem (Kitano, 1993) in the domain of artificial intelligence, it is hard to scale-up a system whose preliminary implementation works only in laboratory (toy-world) environments. This real-world oriented approach also facilitates the implementation of various practical applications in which music synchronization is necessary.Most previous beat-tracking related systems had difficulty working in real-world acoustic environments. Most of them (Dannenberg & Mont-Reynaud, 1987;Desain & Honing, 1989Allen & Dannenberg, 1990;Driesse, 1991; Rosenthal,1992aRosenthal, ,1992bRowe,1993;Large,1995)usedas their input MIDI-like representations, and their applications are limited because it is not easy to obtain complete MIDI representations from real-world audio signals. Some systems (Schloss, 1985;Katayose, Kato, Imai, & Inokuchi, 1989;Vercoe, 1994;Todd, 1994;Todd & Brown, 1996;Scheirer, 1998) dealt with audio signals, but they either did not consider the higher-level beat structureabovethequarter-notelevelor did not process popular music sampled from compact discs in real time. Although I developed two beat-tracking systems for real-world audio signals, one for music with drums (Goto & Muraoka, 1994, 1995, 1998 and the other for music without drums (Goto & Muraoka, 1996, 1999, they were separate systems and the former was not able to recognize the measure level.This paper describes a beat-tracking system that can deal with the audio signals of popular-music compact discs in real time regardless of whether or not those signals contain drum sounds. The system can recognize the hierarchical beat structure comprising the quarter-note level (almost regularly spaced beat times), the half-note level, and the measure level (bar-lines).1 This structure is shown in Figure 1. It assumes that the time-signature of an input song is 4/...

Beat tracking of musical performances using low-level audio features

“…When the location of IOIs is successful, the beat tracking can proceed analogously to when the input is a MIDI file. The pulse finding technique of [13] tracks the temporal position of quarter notes, half notes and measures, and incorporates a multi-agent expert system. Each agent predicts (using a windowed autocorrelation) when the next beat will occur based on onset time vectors that are derived from a frequency decomposition.…”

Recent studies on music information processing

“…Another method for estimating the F 0 of the bass line was later proposed by Hainsworth et al [32]. Beat While beat tracking and measure (bar line) estimation for obtaining a hierarchical beat structure (including beat and measure levels) were researched in the 1980s using MIDI signals, Goto et al began a series of studies [24,[33][34][35][36] on real-world sound mixtures in 1994. Then, taking a hint from these studies, Scheirer proposed a beat-tracking method that could accommodate changes in tempo [37].…”

“…Timbre of musical instruments The second half of the 1990s saw the development of sound-source identification methods [49,50] that recognize the musicalinstrument name of each component sound in a polyphonic sound mixture simultaneously with F 0 estimation. This period also saw the development of methods for estimating the timing of drum sounds in musical performances [33,34,36,51,52]. Sound-source identification methods for isolated monophonic sounds have also been researched from various viewpoints [53][54][55][56][57][58].…”

Simultaneous processing of sound source separation and musical instrument identification using Bayesian spectral modeling

“…These problems have thus far been treated independently. For example, methods of musical instrument identification [1,2] have been reported based on fundamental frequency (F0) estimates [3,4,5], tempo estimates and beat tracking [6,7,8]. Methods of sound source separation have also been reported for separating harmonic sounds [9,10] and separating percussive ones [11,12].…”