Speaker-normalized sound representations in the human auditory cortex

Abstract

The acoustic dimensions that distinguish speech sounds (like the vowel differences in “boot” and “boat”) also differentiate speakers’ voices. Therefore, listeners must normalize across speakers without losing linguistic information. Past behavioral work suggests an important role for auditory contrast enhancement in normalization: preceding context affects listeners’ perception of subsequent speech sounds. Here, using intracranial electrocorticography in humans, we investigate whether and how such context effects arise in auditory cortex. Participants identified speech sounds that were preceded by phrases from two different speakers whose voices differed along the same acoustic dimension as target words (the lowest resonance of the vocal tract). In every participant, target vowels evoke a speaker-dependent neural response that is consistent with the listener’s perception, and which follows from a contrast enhancement model. Auditory cortex processing thus displays a critical feature of normalization, allowing listeners to extract meaningful content from the voices of diverse speakers.

Spectral contrast effects are modulated by selective attention in “cocktail party” settings

“…That is, target perception is not different when attending the fast versus the slow sentence, in contrast to the outcomes about spectral contrast effects in Experiment 2a. This could indicate that differential cognitive mechanisms underlie the two seemingly analogous normalization processes (spectral normalization as studied here vs. rate normalization), as indeed suggested by recent neurobiological studies (Kösem et al, 2018;Sjerps, Fox, Johnson, & Chang, 2019).…”

Temporal contrast effects in human speech perception are immune to selective attention

“…This acoustic context effect induced by the surrounding speech rate, known as a temporal contrast effect or rate normalization, has been shown to influence a wide range of different duration-based phonological cues such as voice onset time (VOT; 11,12 ), formant transition duration 13 , vowel duration 14,15 , lexical stress 16 , and word segmentation 17,18 . In fact, a similar contrastive effect is found in the spectral domain: a sentence with a relatively low first formant (F1) can bias the perception of a following target with an ambiguous F1 (e.g., ambiguous between "bit" and "bet") towards a high F1 percept ("bet"; known as a spectral contrast effect or spectral normalization 6,19,20 ).…”

“…Interestingly, recent evidence suggests that spectral contrast effects (i.e., low formant frequencies in context make following formant frequencies sound higher) are modulated by selective attention 27,28 . This indicates that the neurobiological mechanisms underlying these two forms of acoustic context effects are likely to differ (sustained entrained neural oscillators vs. adaptive gain control 20,33,53 ). In www.nature.com/scientificreports www.nature.com/scientificreports/ particular, context effects induced by higher-level properties of language (e.g., based on talker-identity, language, and situation-specific expectations 45,46,[54][55][56] ) are likely to be influenced by attention.…”

Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution

“…Linear mixed-effects modeling was used to compare the fits of the additive model (with only auditory and visual terms) and the interaction model (with auditory, visual, and interaction terms) to the responses in each electrode ( Schepers et al, 2014 ; Ozker et al, 2017 , 2018a ; Karas et al, 2019 ; Sjerps et al, 2019 ). The lme4 package was used for model construction ( Bates et al, 2015 ), followed by t tests with Satterthwaite-approximated degrees of freedom ( Kuznetsova et al, 2017 ).…”