Sequence Effects and Speech Processing: Cognitive Load for Speaker-Switching Within and Across Accents

Abstract

Prior work in speech perception indicates that listening tasks with multiple speakers (as opposed to a single speaker) result in slower and less accurate processing. Notably, the trial-to-trial cognitive demands of switching between speakers or switching between accents have yet to be examined. We used pupillometry, a physiological index of cognitive load, to examine the demands of processing native- and nonnative-accented speech when listening to sentences produced by the same speaker consecutively (no switch), a novel speaker of the same accent (within-accent switch), and a novel speaker with a different accent (across-accent switch). Inspired by research on sequential adjustments in cognitive control, we anticipated that by examining the trial-to-trial demands of speech perception we would be able to identify the cognitive demands of accommodating a novel speaker and accent. Our results indicate that switching between speakers was more cognitively demanding than listening to the same speaker consecutively. Additionally, switching to a novel speaker with a different accent was more cognitively demanding than switching between speakers of the same accent. However, there was an asymmetry for across-accent switches, such that switching from native to nonnative accent was more demanding than the reverse. Findings from the present study align with work examining multi-talker processing costs and provide novel evidence that listeners dynamically adjust cognitive resources to accommodate speaker and accent variability. We discuss these novel findings in the context of congruency sequence effects and a linguistic account of the cognitive demands of resolving acoustic-phonetic variability.

Exposure to second language accent prompts recalibration of phonemic categories

“…Although the present study focuses on how listeners accommodate variation present in L2 speech, the challenge of reconciling systematic segmental differences (as well as suprasegmental differences) between a speaker's productions and the listener's own perceptual categories is substantial in all types of multi-talker and multi-accent communication. The task of rapidly accommodating unique (L1) speaker productions has been shown to induce a processing cost, such that blocks with multiple talkers (changing from trial-to-trial) result in slower and less accurate perception by listeners than blocks with a single talker (Choi et al, 2018;Kapadia & Perrachione, 2020;Magnuson & Nusbaum, 2007), and changing between speakers of different accents (e.g., L1 to L2 accents) poses an even greater cognitive challenge (McLaughlin et al, 2023). When external acoustic information and internal perceptual categories are misaligned, processing is slowed (Porretta et al, 2020), cognitive demands are increased , and accuracy is reduced (Munro & Derwing, 2020).…”

Sequence effects during speech perception reveal multi-accent processing costs

“…The initial evidence surrounding these questions indicates that the processing costs associated with switching between talkers are even greater when switching from an L1 accent to an L2 accent (McLaughlin et al, 2023). It is possible that switching costs scale with the "phonetic distance" between talkers' productions.…”

“…Further evidence for parallel mechanisms was found in McLaughlin et al (2023). Using pupillometry as a physiological index of cognitive processing load, the authors examined speech processing in a multi-talker block with talkers of L1 (American English) and L2 (Mandarin) accents.…”

“…Moreover, switching from an L1-accented speaker to an L2-accented speaker ("across-accent") was more challenging than switching between two L2-accented speakers of the same accent ("within-accent"). One interpretation of the outcomes of McLaughlin et al (2023) is that the costs observed for across-accent switches reflect engagement of two parallel mechanisms: A mechanism that induces a processing cost any time a change in talkers occurs (i.e., an auditory streaming mechanism) and a recalibration mechanism that induces a cost when recruited for computing mappings to L2 accent (i.e., an active control mechanism). However, an alternative interpretation of the latter mechanism could not be ruled out due to the design of the task: Rather than a mechanism that induces a cost while computing mappings to L2 accent, it may be the case that transitioning from an easier (L1 accent) to a more challenging (L2 accent) listening condition induces a greater demand on cognitive resources.…”

“…To address the two competing explanations of the across-accent switching costs observed in McLaughlin et al (2023), we first investigate multi-talker processing costs in a setting with multiple L2 accents. Specifically, in Experiment 1 we examine switching costs for two L2 accents that are matched for overall difficulty.…”