A Conditional Random Field Framework for Robust and Scalable Audio-to-Score Matching

Abstract

Abstract-In the present work, we introduce the use of Conditional Random Fields (CRFs) for the audio-to-score alignment task. This framework encompasses the statistical models which are used in the literature and allows for more flexible dependency structures. In particular, it allows observation functions to be computed from several analysis frames.Three different CRF models are proposed for our task, for different choices of tradeoff between accuracy and complexity. Three types of features are used, characterizing the local harmony, note attacks and tempo.We also propose a novel hierarchical approach, which takes advantage of the score structure for an approximate decoding of the statistical model. This strategy reduces the complexity, yielding a better overall efficiency than the classic beam search method used in HMM-based models.Experiments run on a large database of classical piano and popular music exhibit very accurate alignments. Indeed, with the best performing system, more than 95% of the note onsets are detected with a precision finer than 100 ms. We additionally show how the proposed framework can be modified in order to be robust to possible structural differences between the score and the musical performance.

Playing Fast and Loose with Music Recognition

“…Automatic music transcription converts audio files to symbolic representations such as sheet music [3]; audio fingerprinting recognises specific audio recordings, for example in Shazam [32] or in query-by-humming [27]. Turning to live performance, automatic score following (audio-to-score) automatically synchronises live audio or MIDI input to a pre-composed score [15] so as to control computer-generated accompaniment, digital effects or trigger extra-musical events such as lighting and visuals [14]. More flexibly, Cypher [26] analyses MIDI data in real-time, extracting key, chord, beat and phrase group features so as to generate musical accompaniment while the Analyser plugin [29] for Digital Audio Workstations extracts real-time audio features and maps them to Open Sound Control (OSC) to control live visuals.…”

Off-line refinement of audio-to-score alignment by observation template adaptation

“…We exploit an HMM model, where the hidden states represent the concurrencies played at each time frame. For complexity reasons, we choose a prior model similar to the Markovian model of [8], which only constrains the concurrency sequence to follow the same structure as in the score. We assume that the concurrencies are numbered in the order in which they appear in the score.…”

“…Other types of information have also been exploited, such as note onsets [7] or tempo information [8,9].…”

Section-level modeling of musical audio for linking performances to scores in Turkish makam music

“…Inference in this model is performed using causal inference, since the goal is real-time score following in live performances. A perspective on audio-to-score alignment using Conditional Random Fields is taken by the authors of [6]. They propose models of various complexities, with the best-performing model resembling a HHMM with the duration of note events influenced by an additional tempo variable.…”

“…Since we aim to link the scores and the performance in the section-level and not directly aim at a note-to-note alignment, we avoid modeling strategies as presented in [5,6] and come up with a computationally lighter but precise model for section linking.…”