Toward Studying Music Cognition with Information Retrieval Techniques: Lessons Learned from the OpenMIIR Initiative

Abstract

As an emerging sub-field of music information retrieval (MIR), music imagery information retrieval (MIIR) aims to retrieve information from brain activity recorded during music cognition–such as listening to or imagining music pieces. This is a highly inter-disciplinary endeavor that requires expertise in MIR as well as cognitive neuroscience and psychology. The OpenMIIR initiative strives to foster collaborations between these fields to advance the state of the art in MIIR. As a first step, electroencephalography (EEG) recordings of music perception and imagination have been made publicly available, enabling MIR researchers to easily test and adapt their existing approaches for music analysis like fingerprinting, beat tracking or tempo estimation on this new kind of data. This paper reports on first results of MIIR experiments using these OpenMIIR datasets and points out how these findings could drive new research in cognitive neuroscience.

EEG-based decoding and recognition of imagined music

“…We conclude that decoding imagined music does not work optimally if there is no compensation mechanism for the (unknown) onset mismatch of the imagined music, even during cued experiments. Similar problems were encountered by Stober et al 14 in a study on the same data, leading to non-significant results for music imagination. Also in this study, this outcome was attributed to an unsuccessful EEG-stimulus synchronization and investigation of data patterns.…”

“…We will consider three different classification experiments; a two-song classifier, a K -song classifier (with K the number of available songs in the respective data set), and a music category classifier, where the latter distinguishes between the three song categories in the OpenMIIR data set 14 . The latter could reveal if (the omission of) lyrical content leads to vastly different reconstructed music envelopes and whether this affects the classification results.…”

“…To perform a pairwise comparison of Pearson correlations across experiments or subject groups (musicians vs. nonmusicians), we use a Wilcoxon signed-rank test and Wilcoxon rank-sum test respectively. The 5% significance level for classification accuracies is computed from a binomial distribution, which determines the distribution of the observed accuracy of a random classifier given a number of trials per subject (60 for the OpenMIIR data set, 30 for the Bach data set) and the succes rate of correctly labeling one trial (1/12 for the OpenMIIR data set, 1/10 for the Bach data set) 14,15 . The 95 percentile of this ‘random classifier’ distribution is then used as the significance level, leading to a significant threshold of 15% and 20% for the OpenMIIR and the Bach data set, respectively.…”

“…Classification approaches, such as in the study of Schaefer et al 4 , have shown that it is possible to distinguish between seven distinct songs during music perception by using a quadratically regularized linear logistic regression classifier on EEG data. Stober et al 1, 5 conducted classification experiments for 12 songs during music perception, using a similarity-constraint encoding approach. Apart from working on perception data, the latter also concentrated on the EEG responses of imagination experiments.…”

A Statistical Inference Framework for Understanding Music-Related Brain Activity

“…Since the literature already includes mature solutions to the above-mentioned tasks, MIR is expanding towards the exploitation of signals representing brain activity while listening or imagining music pieces. This sub-area of MIR is called Music Imagery Information Retrieval (MIIR) and has only recently emerged [7]. It aims to support existing MIR solutions in applications such as query by singing, humming, tapping, or beat-boxing, to name but a few, with the ultimate goal S. Ntalampiras is with Università degli studi di Milano, Department of Computer Science, via Celoria 18, 20135, Milan, Italy, stavros.ntalampiras@unimi.it, https://sites.google.com/site/stavrosntalampiras/ home.…”