Distributed Neural Representations of Phonological Features during Speech Perception

Abstract

A fundamental goal of the human auditory system is to map complex acoustic signals onto stable internal representations of the basic sound patterns of speech. Phonemes and the distinctive features that they comprise constitute the basic building blocks from which higher-level linguistic representations, such as words and sentences, are formed. Although the neural structures underlying phonemic representations have been well studied, there is considerable debate regarding frontal-motor cortical contributions to speech as well as the extent of lateralization of phonological representations within auditory cortex. Here we used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis to investigate the distributed patterns of activation that are associated with the categorical and perceptual similarity structure of 16 consonant exemplars in the English language used in Miller and Nicely's (1955) classic study of acoustic confusability. Participants performed an incidental task while listening to phonemes in the MRI scanner. Neural activity in bilateral anterior superior temporal gyrus and supratemporal plane was correlated with the first two components derived from a multidimensional scaling analysis of a behaviorally derived confusability matrix. We further showed that neural representations corresponding to the categorical features of voicing, manner of articulation, and place of articulation were widely distributed throughout bilateral primary, secondary, and association areas of the superior temporal cortex, but not motor cortex. Although classification of phonological features was generally bilateral, we found that multivariate pattern information was moderately stronger in the left compared with the right hemisphere for place but not for voicing or manner of articulation.

Amplification of local changes along the timescale processing hierarchy

“…This hierarchy, as well as the divergence of neural responses observed here, is additionally consistent with previously proposed linguistic hierarchies: low-level regions (A1+) represent phonemes (32,33), syllables (34), and pseudowords (35), while medium-level regions (areas along A1+ to STS) represent sentences (36,37). At the top of the hierarchy, high-level regions (bilateral TPJ, precuneus, and medial prefrontal cortex) can integrate words and sentences into a meaningful, coherent whole narrative (5,15,(38)(39)(40).…”

Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions

“…Articulation and especially voicing-related representations within the right anterior STS may relate to its involvement in the processing of human voices (Belin et al, 2000), and possibly to the proposed specialization of this area in perceiving vocaltract properties of speakers (e.g., shape and characteristics of the vocal folds). A role of this phonological feature in the processing of human voices is also compatible with the previous finding that voicing was more robustly classified than either place or manner of articulation when subjects performed an active gender discrimination task (Arsenault and Buchsbaum, 2015).…”

“…Together with feedback from auditory cortex, this somatosensory feedback may form a state feedback control system for speech production (SFC) (Houde and Nagarajan, 2011). The involvement of somatosensory areas in the representation of articulatory features during passive speech perception extends recent findings showing the involvement of these regions in the neural decoding of place and manner of articulation during speech production (Bouchard et al, 2013), and of place of articulation during an active perceptual task in English listeners (Arsenault and Buchsbaum, 2015). In particular, they may indicate automatic information transfer from auditory to somatosensory representations during speech perception (Cogan et al, 2014) similar to their integration as part of SFC systems for speech production.…”

“…Indeed, it remains unknown whether articulatory speech representations play an active role in speech perception and/or whether this is dependent on taskspecific sensorimotor goals. Tasks explicitly requiring sensoryto-motor control, such as speech repetition (Caplan and Waters, 1995;Hickok et al, 2009), humming (Hickok et al, 2003), and verbal rehearsal in working memory (Baddeley et al, 1998;Jacquemot and Scott, 2006;Buchsbaum et al, 2011), activate the dorsal auditory pathway, including sensorimotor regions at the border of the posterior temporal and parietal lobes, the sylvian-parieto-temporal region and supramarginal gyrus (SMG) (Hickok and Poeppel, 2007). Also, in experimental paradigms that do not explicitly require sensorimotor integration, the perception of speech may involve a coactivation of motor speech regions (Zatorre et al, 1992;Wilson et al, 2004).…”

“…This approach has been successful in demonstrating auditory cortical representations of speech (Formisano et al, 2008;KilianHütten et al, 2011;Lee et al, 2012;Bonte et al, 2014;Arsenault and Buchsbaum, 2015;Evans and Davis, 2015). Crucially, MVPA enables isolating neural representations of stimulus classes from variation across other stimulus dimensions, such as the representation of vowels independent of acoustic variation across speakers' pronunciations (Formisano et al, 2008) or the representation of semantic concepts independent of the input language in bilingual listeners (Correia et al, 2014).…”