High-Order Areas and Auditory Cortex Both Represent the High-Level Event Structure of Music

Abstract

Recent fMRI studies of event segmentation have found that default mode regions represent high-level event structure during movie watching. In these regions, neural patterns are relatively stable during events and shift at event boundaries. Music, like narratives, contains hierarchical event structure (e.g., sections are composed of phrases). Here, we tested the hypothesis that brain activity patterns in default mode regions reflect the high-level event structure of music. We used fMRI to record brain activity from 25 participants (male and female) as they listened to a continuous playlist of 16 musical excerpts and additionally collected annotations for these excerpts by asking a separate group of participants to mark when meaningful changes occurred in each one. We then identified temporal boundaries between stable patterns of brain activity using a hidden Markov model and compared the location of the model boundaries to the location of the human annotations. We identified multiple brain regions with significant matches to the observer-identified boundaries, including auditory cortex, medial pFC, parietal cortex, and angular gyrus. From these results, we conclude that both higher-order and sensory areas contain information relating to the high-level event structure of music. Moreover, the higher-order areas in this study overlap with areas found in previous studies of event perception in movies and audio narratives, including regions in the default mode network.

Event segmentation in continuous, naturalistic videos from model-based, data-driven, and human perspectives

“…fMRI signals related to specific regions or networks) of participants during naturalistic stimulation (e.g. movies, audio-narratives, music pieces - Baldassano et al, 2017;Cohen et al, 2022;Williams et al, 2022).…”

“…HMM has been shown to be able to retrieve neural states at different hierarchies corresponding with the events reported by participants when segmenting different kinds of content. Examples range from adults (Baldassano et al, 2017) or children (Cohen et al, 2022) watching movies while undergoing fMRI or EEG (Silva et al, 2019) to the detection of events across different brain regions matching participants' phenomenological segmentation of different musical excerpts (Williams et al, 2022). Both HMM and GSBS are not designed to identify recurrent states (the first time point is always within event 1 and the last one with event k), however they differ in terms of both the selected measure of distance and fitting time.…”

“…Other studies instead addressed whether the segmentation sequences obtained from a specific model (e.g., HMM) matched the annotations provided by human raters by means of different two-steps procedures. First, a measure of similarity/distance is typically defined: either number of matching boundaries between sequences within a pre-specified time-window (e.g., 3 seconds - Baldassano et al, 2017;Williams et al, 2022), the point-biserial correlation between different segmentations (Franklin et al, 2020;Zacks et al, 2006) or the Jaccard index to estimate the similarity between two time-series , etc (e.g., see Cohen et al, 2022;Lee et al, 2021). Second, the selected metric is typically re-computed many times within a permutation-based approach (e.g., 1000) by shuffling the event boundaries of one of the two sequences in time and by keeping both the number of boundaries and the distance between them constant.…”

Internally and Externally Oriented Perception of Music

“…Both music and the brain operate according to a particular structure. When interacting with one another, changes in the structure of the music or the brain have an influence on one another (Williams et al, 2022). The brain processes information using a hierarchy of temporal windows.…”

“…Similarly, music has a higher order hierarchical event structure at the level of phrases, sections, and entire songs. The sensory regions in music are the dynamic change, pitch, mode, rhythm, and metre (Williams et al, 2022). Music is composed using a hierarchy of temporal windows similar to how the brain processes information through temporal windows (Lalitte and Bigand, 2006;Williams et al, 2002).…”