Task-evoked pupillary response to completely intelligible accented speech

Abstract

Listening to second language- (L2-) accented speech is often described as an effortful process, even when L2 speakers are highly proficient. This increase in listening effort is likely caused by systematic segmental and suprasegmental deviations from native-speaker norms, which require additional cognitive resources to process (Van Engen and Peelle, 2014). In this view of speech perception, even when an L2 speaker is completely intelligible (i.e., their words can all be correctly identified), perception nonetheless requires increased cognitive load compared to native speech. We used pupillometry (the measure of pupil diameter over time) as a psychophysiological index of cognitive load to address this hypothesis. Task-evoked pupillary response (TEPR) was measured while participants listened to sentences produced by a Mandarin Chinese-accented English speaker and a standard American-accented English speaker. Results from a first experiment showed that TEPR was larger for L2-accented speech than the native speech, indicating greater cognitive load during speech processing. In a second experiment, we controlled for differences in the speech rate between the two speakers. Preliminary evidence from this replication also shows larger TEPR for the L2-accented speech condition. Together, this evidence suggests that processing accented speech—even when completely intelligible—requires additional cognitive resources.

Rapid adaptation to fully intelligible nonnative-accented speech reduces listening effort

“…Consistent with our hypothesis, the significant main effect of accent indicated that on average, pupil dilation was greater for nonnative- relative to native-accented speech within the time window of interest (β = 115.50, SE = 34.43, t = 3.36, p = .001; Figure 3). Further, the significant interaction between accent and the linear polynomial time term indicated that pupil dilation increased more rapidly after sentence onset for nonnative- relative to native-accented speech (β = 566.92, SE = 88.33, t = 6.42, p < .001), consistent with previous research (McLaughlin & Van Engen, 2020).…”

“…The results from the two tasks were largely consistent with one another, providing converging evidence that (1) processing nonnative-accented speech requires greater listening effort than processing native-accented speech, as indicated by a main effect of accent, (2) the amount of listening effort required to process speech decreases rapidly with exposure, as indicated by a main effect of trial number, and (3) the reduction in listening effort over time may occur more rapidly for nonnative relative to native speech, as indicated by an interaction between accent and trial number in Experiment 2. Thus, it appears that although processing nonnative-accented speech requires greater listening effort than processing native-accented speech—even when intelligibility is equated across these conditions—these differences become less pronounced with exposure (replicating and extending the findings of McLaughlin & Van Engen, 2020). We did not explicitly test whether differences in effort between the native and nonnative conditions persisted following adaptation, but visual inspection of Figures 1 and 4 suggest that listeners still expended greater effort in the nonnative condition following adaptation.…”

“…The pre-registration form for Experiment 2, which includes hypotheses, planned analyses, exclusion criteria, and sample size justification, can be found at https://osf.io/z74fp. We predicted that we would replicate the results of McLaughlin and Van Engen 2020 by showing that processing intelligible nonnative-accented speech results in greater pupil dilation, indicative of greater cognitive demand, than processing speech produced by a native talker. Further, we predicted that pupil dilation would reduce to a greater extent over the course of an experimental session (indicating greater reduction in listening effort) for nonnative- relative to native-accented speech.…”

“…Compared with processing speech produced by a native-accented talker, processing nonnative-accented speech also appears to increase listening effort. In a recent pupillometry study, McLaughlin and Van Engen (2020) found that native English-speaking participants show larger increases in pupil dilation—indicating greater listening effort—when listening to Mandarin Chinese–accented English relative to standard American English sentences. Notably, the stimuli used in that study were fully intelligible (i.e., all four keywords in the sentences included in the primary analyses were correctly identified by participants), suggesting that even when speech intelligibility is at ceiling, resolving mismatches between the accented speech input and representations of words in the listener’s mental lexicon may require additional cognitive resources (e.g., working memory, attention; Van Engen & Peelle, 2014).…”

“…It is not yet known, however, whether adaptation to nonnative-accented speech occurs for listening effort when intelligibility is at ceiling. One possibility is that even after near-perfect accuracy is attained for understanding nonnative-accented speech, differences in listening effort persist (McLaughlin & Van Engen, 2020), and effort does not reduce with exposure. A growing body of research indicates that differences in listening effort may be observed even when intelligibility is equated.…”