The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research

Abstract

The Music Information Retrieval Evaluation eXchange (MIREX) is the community-based framework for the formal evaluation of Music Information Retrieval (MIR) systems and algorithms. By looking at the background, structure, challenges, and contributions of MIREX this paper provides some insights into the world of MIR research. Because MIREX tasks are defined by the community they reflect the interests, techniques, and research paradigms of the community as a whole. Both MIREX and MIR have a strong bias toward audio-based approaches as most MIR researchers have strengths in signal processing. Spectral-based approaches to MIR tasks have led to advancements in the MIR field but they now appear to be reaching their limits of effectiveness. This limitation is called the ''glass ceiling'' problem and the MIREX results data support its existence. The post-hoc analyses of MIREX results data indicate that there are groups of systems that perform equally well within various MIR tasks. There are many challenges facing MIREX and MIR research most of which have their root causes in the intellectual property issues surrounding music. The current inability of researchers to test their approaches against the MIREX test collections outside the annual MIREX cycle is hindering the rapid development of improved MIR systems.

Predictability of Music Descriptor Time Series and its Application to Cover Song Detection

“…This measure is routinely employed in the IR [50] and MIR [14] communities and, in particular, in the cover song identification task [16].…”

“…To properly asses this accuracy one has to compare it against the accuracies of state-of-the-art approaches. According to an international MIR evaluation framework (the yearly MIR evaluation exchange, MIREX [14]), the best accuracy achieved to date within the cover song identification task 6 was obtained from a previous system by Serrà et al [20]. This system reached ψ = 0.66 with the MIREX dataset and yields ψ = 0.698…”

“…MIR is an interdisciplinary research field that aims at automatically understanding, describing, retrieving and organizing musical content. In particular, much effort is focused on extracting qualitative and quantitative information from the audio signal of real recordings in order to represent certain musical aspects such as timbre, rhythm, melody, main tonality, chords or tempo [12], [14]. Quantitative descriptions of such aspects are computed in a short-time moving window either from a temporal, spectral or cepstral representation of the audio signal [15].…”

An Effective Approach for Vocal Melody Extraction from Polyphonic Music on GPU

“…The complexity of the task is twofold -firstly, due to the superposition of all instruments which play simultaneously, the accuracy of the extraction is still hovering at a relatively low level. This can be seen from the results of the melody extraction of Music Information Retrieval Evaluation eXchange (MIREX) [3] in recent years. Secondly, most methods submitted to MIREX are very time-consuming [1].…”

Nonlinear audio recurrence analysis with application to genre classification

“…For the calculation of the STFT, the audio signal is cut into short overlapping frames, a windowing function is applied to each frame, and the magnitude of the spectrum is taken. For many applications, music descriptors are fed to data mining or machine learning algorithms, which exploit the information contained in the descriptors [1,3]. A paradigmatic example of this approach is automatic genre classification [4].…”