Exploring Hierarchical Auditory Representation via a Neural Encoding Model

Abstract

By integrating hierarchical feature modeling of auditory information using deep neural networks (DNNs), recent functional magnetic resonance imaging (fMRI) encoding studies have revealed the hierarchical neural auditory representation in the superior temporal gyrus (STG). Most of these studies adopted supervised DNNs (e.g., for audio classification) to derive the hierarchical feature representation of external auditory stimuli. One possible limitation is that the extracted features could be biased toward discriminative features while ignoring general attributes shared by auditory information in multiple categories. Consequently, the hierarchy of neural acoustic processing revealed by the encoding model might be biased toward classification. In this study, we explored the hierarchical neural auditory representation via an fMRI encoding framework in which an unsupervised deep convolutional auto-encoder (DCAE) model was adopted to derive the hierarchical feature representations of the stimuli (naturalistic auditory excerpts in different categories) in fMRI acquisition. The experimental results showed that the neural representation of hierarchical auditory features is not limited to previously reported STG, but also involves the bilateral insula, ventral visual cortex, and thalamus. The current study may provide complementary evidence to understand the hierarchical auditory processing in the human brain.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception

“…This approach has been embraced in vision studies which showed that deep neural networks trained on identifying objects reveal progressions of hierarchical processing from simple features like edges and textures to more complex object categories mimicking gradients of processing in the visual what stream (Güçlü & van Gerven, 2015 ). Similar insights about processing gradients in the auditory stream have been gleaned using deep learning methods that not only shed light on cortical processing hierarchies, but also specialized processing in non-primary cortical pathways for processing speech and music sounds (Kell et al, 2018 ; Wang et al, 2022 ). In the current study, deep neural networks were also used as an investigative tool to infer relationships between behavioral responses to natural auditory scenes and nonlinear abstractions of these scenes extracted by learned models.…”

Multivariate analysis of brain activity patterns as a tool to understand predictive processes in speech perception

“…Additionally, a representational hierarchy of speech processing was apparent with intermediate layers mapping best to neural activity in PAC while later layers best predicted activity in adjacent regions along the temporal plane (Kell et al, 2018). This representational gradient was demonstrated to be independent of the learning modality, i.e., supervised or unsupervised learning in the neural network (Wang et al, 2019(Wang et al, , 2022.…”

Is this a family business? Effectiveness of implementing family businesses branding strategies on a radio platform

“…The consumer assessment of firms' products promoted by radio ads depends on the mental representation of the message, modeled by sounds (auditory stimulus) (Lombardi Vallauri, 2017; Soto‐Sanfiel et al, 2021). Radio is considered an effective platform for creating images of the consumer mind, as the human brain can synthesize signals, catalog them, and coordinate them (Lezama‐Espinosa & Hernandez‐Montiel, 2020; Wang et al, 2022). This platform allows the use of different forms of audio branding such as audio logos, claim sounds, commercial songs, jingles, brand songs, brand voices, and product sounds (Barrio Fraile et al, 2021; Rodero & Larrea, 2020; Rodriguez et al, 2022).…”