The Learning Signal in Perceptual Tuning of Speech: Bottom Up Versus Top‐Down Information

Abstract

Cognitive systems face a tension between stability and plasticity. The maintenance of long‐term representations that reflect the global regularities of the environment is often at odds with pressure to flexibly adjust to short‐term input regularities that may deviate from the norm. This tension is abundantly clear in speech communication when talkers with accents or dialects produce input that deviates from a listener's language community norms. Prior research demonstrates that when bottom‐up acoustic information or top‐down word knowledge is available to disambiguate speech input, there is short‐term adaptive plasticity such that subsequent speech perception is shifted even in the absence of the disambiguating information. Although such effects are well‐documented, it is not yet known whether bottom‐up and top‐down resolution of ambiguity may operate through common processes, or how these information sources may interact in guiding the adaptive plasticity of speech perception. The present study investigates the joint contributions of bottom‐up information from the acoustic signal and top‐down information from lexical knowledge in the adaptive plasticity of speech categorization according to short‐term input regularities. The results implicate speech category activation, whether from top‐down or bottom‐up sources, in driving rapid adjustment of listeners' reliance on acoustic dimensions in speech categorization. Broadly, this pattern of perception is consistent with dynamic mapping of input to category representations that is flexibly tuned according to interactive processing accommodating both lexical knowledge and idiosyncrasies of the acoustic input.

Phonetic category activation predicts the direction and magnitude of perceptual adaptation to accented speech.

“…Two prior studies are especially interesting in this regard. In a dimension-based statistical learning paradigm like the one used here, Zhang et al (2021) examined the impact of speech category activation on perceptual reweighting through top–down lexical knowledge. In their study, listeners identified nonminimal-pair word-nonword pairs, like beef-peef and beace-peace .…”

“…Yet, upon encountering a block of trials that conveys an “accent” in which the F0 × VOT relationship is reversed from English norms, listeners rapidly down-weight reliance on F0 such that it no longer reliably signals /b/ versus /p/. These results, replicated many times and apparent for vowel as well as consonant categorization, demonstrate the flexibility of the mapping from acoustics to speech categories upon encountering short-term regularities that depart from the norm (Idemaru & Holt, 2011, 2014, 2020; Lehet & Holt, 2017, 2020; Liu & Holt, 2015; Schertz et al, 2016; Zhang et al, 2021; Zhang & Holt, 2018).…”

“…Here, we test these two predictions in the context of perceptual adaptation driven by statistical regularities in acoustic speech input, so-called dimension-based statistical learning (Idemaru & Holt, 2011, 2014, 2020; Lehet & Holt, 2017, 2020; Liu & Holt, 2015; Schertz et al, 2016; Zhang & Holt, 2018; Zhang et al, 2021). This paradigm elicits perceptual adaptation through systematic shifts in speech acoustic regularities, simulating the introduction of accented speech by shifting correlations across acoustic speech dimensions.…”

Short-term perceptual re-weighting in suprasegmental categorization

“…Though there remain many open questions even for adaptive plasticity across segmental speech categorization, in general patterns of generalization seem to be consistent with an account that category activation drives reweighting (Idemaru & Holt, 2011; Zhang et al, 2021; Wu & Holt, under review; Wu, 2020) and the mechanism for this may involve supervised error-driven learning (Guediche et al, 2014; Wu, 2000) or reinforcement learning (Harmon et al, 2019). To the extent that short-term input regularities across acoustic dimensions are effective in activating a suprasegmental category even as they deviate from long-term expectations of correlations among input dimensions, we would anticipate re-weighting and modest generalization.…”

Oxytocin facilitates communicative adjustments by upregulating broadband aperiodic activity in the prefrontal cortex

“…This observation suggests that, in the initial stages of the communicative interaction, oxytocin increased exploration of different adjustments to the two presumed addressees, followed by a convergence of the communicative adjustments directed to those addressees. It is known that communicators spontaneously integrate prior knowledge about their interlocutor with the ongoing communicative experience (Brennan et al, 2010;Clark & Krych, 2004;Hawkins et al, 2021;Kuhlen et al, 2012;Zhang et al, 2021). The behavioral finding of this study illustrates how oxytocin enhances that integration and facilitates the implementation of the Gricean Maxim of Quantity, namely producing utterances that convey no more information than necessary (Blokpoel et al, 2012;Brennan & Clark, 1996;de Ruiter et al, 2010;Garrod & Anderson, 1987;Grice, 1975;Levinson, 2000;Liu et al, 2019;Yoon & Brown-Schmidt, 2018).…”