Population Codes Representing Musical Timbre for High-Level fMRI Categorization of Music Genres

Casey, Michael A.; Thompson, Jessica A. F.; Kang, Olivia; Raizada, Rajeev D. S.; Wheatley, Thalia

doi:10.1007/978-3-642-34713-9_5

Cited by 20 publications

(31 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from the use of high-field fMRI in the current study, differences between the current and the two former studies include 5,000-voxel feature selection by ROI in the current study, no sensitivity based selection in Casey et al (2012), and 1,000-voxel feature-selection from whole brain voxels in Guntupalli (2013). The latter study also employed a different cross-validation scheme, which also accounts for some of the difference in accuracy.…”

Section: Discussionmentioning

confidence: 89%

“…Regions of interest (ROIs) were selected spanning primary and secondary auditory cortex due to their implication in prior music classification studies (Casey et al, 2012; Guntupalli, 2013); these were: Heschl's gyrus (HG), anterior superior temporal gyrus (aSTG), and posterior superior temporal gyrus (pSTG).…”

Section: Methodsmentioning

confidence: 99%

“…Guntupalli (2013) repeated the experiment of Casey et al (2012), and additionally performed whole-brain hyperalignment to create between-subject models of stimulus encoding and reconstruction for spectral and timbral acoustic features. Lee et al (2011) also used voxel-based decoding to classify melodic contour of ascending and descending major and minor scales.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

Casey

2017

Front. Psychol.

View full text Add to dashboard Cite

Underlying the experience of listening to music are parallel streams of auditory, categorical, and schematic qualia, whose representations and cortical organization remain largely unresolved. We collected high-field (7T) fMRI data in a music listening task, and analyzed the data using multivariate decoding and stimulus-encoding models. Twenty subjects participated in the experiment, which measured BOLD responses evoked by naturalistic listening to twenty-five music clips from five genres. Our first analysis applied machine classification to the multivoxel patterns that were evoked in temporal cortex. Results yielded above-chance levels for both stimulus identification and genre classification–cross-validated by holding out data from multiple of the stimuli during model training and then testing decoding performance on the held-out data. Genre model misclassifications were significantly correlated with those in a corresponding behavioral music categorization task, supporting the hypothesis that geometric properties of multivoxel pattern spaces underlie observed musical behavior. A second analysis employed a spherical searchlight regression analysis which predicted multivoxel pattern responses to music features representing melody and harmony across a large area of cortex. The resulting prediction-accuracy maps yielded significant clusters in the temporal, frontal, parietal, and occipital lobes, as well as in the parahippocampal gyrus and the cerebellum. These maps provide evidence in support of our hypothesis that geometric properties of music cognition are neurally encoded as multivoxel representational spaces. The maps also reveal a cortical topography that differentially encodes categorical and absolute-pitch information in distributed and overlapping networks, with smaller specialized regions that encode tonal music information in relative-pitch representations.

show abstract

Section: Discussionmentioning

confidence: 89%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

Casey

2017

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Kay et al [2008] used stimulus identication accuracy, and Naselaris et al [2009] used stimulus reconstruction quality as a metric for encoding performance. In auditory neuroscience, only binary retrieval accuracy [Casey et al, 2012, Hoee et al, 2018 and stimulus identication [Santoro et al, 2014, Allen et al, 2018 have been used. However, while providing novel insights into the cortical processing of sensory stimuli, it is unknown how additional parameters of the data analysis that are unrelated to the encoding models themselves aect these quality metrics.…”

Section: Introductionmentioning

confidence: 99%

The role of auxiliary parameters in evaluating voxel-wise encoding models for 3T and 7T BOLD fMRI data

Boos

Guntupalli²,

Rieger

et al. 2020

Preprint

View full text Add to dashboard Cite

In neuroimaging, voxel-wise encoding models are a popular tool to predict brain activity elicited by a stimulus. To evaluate the accuracy of these predictions across multiple voxels, one can choose between multiple quality metrics. However, each quality metric requires specifying auxiliary parameters such as the number and selection criteria of voxels, whose inuence on model validation is unknown. In this study, we systematically vary these parameters and observe their eects on three common quality metrics of voxel-wise encoding models in two open datasets of 3-and 7-Tesla BOLD fMRI activity elicited by musical stimuli. We show that such auxiliary parameters not only exert substantial inuence on model validation, but also dier in how they aect each quality metric. Finally, we give several recommendations for validating voxel-wise encoding models that may limit variability due to dierent numbers of voxels, voxel selection criteria, and magnetic eld strengths.

show abstract

“…Similar approaches have been used to predict the neural response to novel natural images using Gabor filter features (Kay et al, 2008), to novel colors based on color tuning curve features (Brouwer and Heeger, 2009), to novel music clips based on acoustic timbre features (Casey et al, 2012), to natural sounds based on frequency, spectral and temporal modulations (Santoro et al, 2014), to novel faces based on a PCA decomposition of face features (Lee and Kuhl, 2016), to novel words based on subjective sensory-motor ratings (Fernandino et al, 2016). The motivating question behind many of these studies has been about the nature of the representations used by the brain in encoding the experimental stimuli, and the results are often cited as evidence that the neural representation is based on the constituent features of the stimuli used in the model.…”

mentioning

confidence: 99%

Practices and pitfalls in inferring neural representations

2018

View full text Add to dashboard Cite

A B S T R A C TA key challenge for cognitive neuroscience is deciphering the representational schemes of the brain. Stimulusfeature-based encoding models are becoming increasingly popular for inferring the dimensions of neural representational spaces from stimulus-feature spaces. We argue that such inferences are not always valid because successful prediction can occur even if the two representational spaces use different, but correlated, representational schemes. We support this claim with three simulations in which we achieved high prediction accuracy despite systematic differences in the geometries and dimensions of the underlying representations. Detailed analysis of the encoding models' predictions showed systematic deviations from ground-truth, indicating that high prediction accuracy is insufficient for making representational inferences. This fallacy applies to the prediction of actual neural patterns from stimulus-feature spaces and we urge caution in inferring the nature of the neural code from such methods. We discuss ways to overcome these inferential limitations, including model comparison, absolute model performance, visualization techniques and attentional modulation. IntroductionA key challenge for cognitive neuroscience is to understand the neural code that underlies the encoding and representation of sensory, motor, spatial, emotional, semantic and other types of information. To decipher the representational schemes of the brain, researchers often employ neuroimaging techniques such as functional magnetic resonance imaging (fMRI). fMRI measures the blood oxygenation level-dependent (BOLD) activation in the brain that is elicited when participants engage with different stimuli. The neural representation underlying each stimulus is assumed to have measurable but complex effects on the BOLD activation patterns. In order to understand what those patterns of activity can tell us about how the brain processes and represents information, researchers have used various analytical tools such as univariate subtraction methods, multivariate pattern (MVP) classification, representational similarity analysis (RSA) and, recently, explicit stimulus-feature-based encoding and decoding models (for reviews, see Davis and Poldrack, 2013, Haxby et al., 2014, or Naselaris et al., 2011. Despite their differences, all of these methods have the same goal -to quantify how changes in task conditions and the properties of the stimuli relate to changes in BOLD activation and vice versa. One way in which these methods differ is in how they achieve that mapping and in what inferences they allow us to draw.In this article, we review some of the known inferential limitations of existing fMRI analysis methods and we highlight an often-overlooked issue in interpreting results from stimulus-feature-based encoding and decoding models. The latter are steadily becoming the de facto gold standard for investigating neural representational spaces (Haxby et al., 2014;Naselaris and Kay, 2015). Using simulated data with known representati...

show abstract

Population Codes Representing Musical Timbre for High-Level fMRI Categorization of Music Genres

Cited by 20 publications

References 18 publications

Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

Music of the 7Ts: Predicting and Decoding Multivoxel fMRI Responses with Acoustic, Schematic, and Categorical Music Features

The role of auxiliary parameters in evaluating voxel-wise encoding models for 3T and 7T BOLD fMRI data

Practices and pitfalls in inferring neural representations

Contact Info

Product

Resources

About