Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

Abstract

Underlying the experience of listening to music are parallel streams of auditory, categorical, and schematic qualia, whose representations and cortical organization remain largely unresolved. We collected high-field (7T) fMRI data in a music listening task, and analyzed the data using multivariate decoding and stimulus-encoding models. Twenty subjects participated in the experiment, which measured BOLD responses evoked by naturalistic listening to twenty-five music clips from five genres. Our first analysis applied machine classification to the multivoxel patterns that were evoked in temporal cortex. Results yielded above-chance levels for both stimulus identification and genre classification–cross-validated by holding out data from multiple of the stimuli during model training and then testing decoding performance on the held-out data. Genre model misclassifications were significantly correlated with those in a corresponding behavioral music categorization task, supporting the hypothesis that geometric properties of multivoxel pattern spaces underlie observed musical behavior. A second analysis employed a spherical searchlight regression analysis which predicted multivoxel pattern responses to music features representing melody and harmony across a large area of cortex. The resulting prediction-accuracy maps yielded significant clusters in the temporal, frontal, parietal, and occipital lobes, as well as in the parahippocampal gyrus and the cerebellum. These maps provide evidence in support of our hypothesis that geometric properties of music cognition are neurally encoded as multivoxel representational spaces. The maps also reveal a cortical topography that differentially encodes categorical and absolute-pitch information in distributed and overlapping networks, with smaller specialized regions that encode tonal music information in relative-pitch representations.

Correspondence of categorical and feature‐based representations of music in the human brain

“…Ghaemmaghami and Sebe (2017) used magnetoencephalogram and electroencephalogram datasets to classify musical stimuli as either pop or rock using SVM (Ghaemmaghami & Sebe, 2016). Further, Case y (2017) and Sengupta et al. (2018) used fMRI data with five distinct music genres, followed by activity‐based multi‐class classification using SVM.…”

“…However, there remains considerable uncertainty as to how such genre categories are perceived from complex auditory stimuli and how the human brain subserves this categorization. Neuroimaging studies have decoded music genres from brain activity using support vector machines (SVM) (Case y, 2017; Ghaemmaghami & Sebe, 2016; Sengupta et al., 2018); however, these studies did not clarify how cortical representations of music genres contribute to genre classification.…”

Spatial band-pass filtering aids decoding musical genres from auditory cortex 7T fMRI

“…Data were taken from a published dataset 2 which were repeatedly analyzed previously 4 , 5 , and publicly available from the studyforrest.org project of 20 participants passively listening to five natural, stereo, high-quality music stimuli (6 s duration; 44.1 kHz sampling rate) for each of five different musical genres: 1) Ambient, 2) Roots Country 3) Heavy Metal, 4) 50s Rock’n’Roll, and 5) Symphonic, while fMRI data were recorded in a 7 Tesla Siemens scanner (1.4 mm isotropic voxel size, TR=2 s, matrix size 160 × 160, 36 axial slices, 10% interslice gap). fMRI data were scanner-side corrected for spatial distortions 6 .…”

Intracranial recordings from human auditory cortex reveal a neural population selective for song

“…Researchers have long debated whether the human brain has neural mechanisms dedicated to music, and if so, what computations those mechanisms perform (Patel, 2012;Peretz et al, 2015). These questions have important implications for understanding the organization of auditory cortex (Leaver and Rauschecker, 2010;Norman-Haignere et al, 2015), the neural basis of sensory deficits such as amusia (Peterson and Pennington, 2015;Norman-Haignere et al, 2016;Peretz, 2016), the consequences of auditory expertise (Herholz and Zatorre, 2012), and the computational underpinnings of auditory behavior (Casey, 2017;Kell et al, 2018). Neuroimaging studies have suggested that representations of music diverge from those of other sound categories in non-primary human auditory cortex: some non-primary voxels show partial selectivity for music compared with other categories (Leaver and Rauschecker, 2010;Fedorenko et al, 2012;Angulo-Perkins et al, 2014), and recent studies from our lab, which modeled voxels as weighted sums of multiple response profiles, inferred a component of the fMRI response with clear selectivity for music (Norman-Haignere et al, 2015;Boebinger et al, 2020) that was distinct from nearby speech-selective responses.…”