Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel.

Abstract

Successful speech perception requires that listeners map the acoustic signal to linguistic categories. These mappings are not only probabilistic, but change depending on the situation. For example, one talker’s /p/ might be physically indistinguishable from another talker’s /b/ (cf. lack of invariance). We characterize the computational problem posed by such a subjectively non-stationary world and propose that the speech perception system overcomes this challenge by (1) recognizing previously encountered situations, (2) generalizing to other situations based on previous similar experience, and (3) adapting to novel situations. We formalize this proposal in the ideal adapter framework: (1) to (3) can be understood as inference under uncertainty about the appropriate generative model for the current talker, thereby facilitating robust speech perception despite the lack of invariance. We focus on two critical aspects of the ideal adapter. First, in situations that clearly deviate from previous experience, listeners need to adapt. We develop a distributional (belief-updating) learning model of incremental adaptation. The model provides a good fit against known and novel phonetic adaptation data, including perceptual recalibration and selective adaptation. Second, robust speech recognition requires listeners learn to represent the structured component of cross-situation variability in the speech signal. We discuss how these two aspects of the ideal adapter provide a unifying explanation for adaptation, talker-specificity, and generalization across talkers and groups of talkers (e.g., accents and dialects). The ideal adapter provides a guiding framework for future investigations into speech perception and adaptation, and more broadly language comprehension.

Toward an integrative model of talker normalization.

“…Fourthly, future studies may examine the relative contribution of talker familiarity, talker typicality and context to the probabilistic mapping of speech signals to linguistic categories using computational modeling (cf. Kleinschmidt & Jaeger, 2015). In this model, we proposed these factors without examining their contribution to probabilistic mapping.…”

A unified account of categorical effects in phonetic perception

“…We argue that while perception of the cues to different sounds is implemented differently at a neural level, strong categorical effects in consonant perception and the largely continuous perception of vowels reflect solutions to the same abstract problem of speech perception at Marr's 1982 computational level. Our analysis appeals to a kind of scientific Occam's Razor to argue that our unified account is the more parsimonious theory; a similar argument was used to substantiate a unified account of cumulative exposure on selective adaptation and phonetic recalibration by Kleinschmidt and Jaeger (2015).…”

The Role of Lexical Status and Individual Differences for Perceptual Learning in Younger and Older Adults

“…Kleinschmidt and Jaeger (2015b) propose an "ideal adapter" framework to account for the challenges faced in speech perception, including adapting to novel talkers. Under this framework, listeners adapt to novel situations by comparing how well each representation predicts the input.…”