The effort of repairing a misperceived word can impair perception of following words, especially for listeners with cochlear implants

Abstract

ObjectivesIn clinical and laboratory settings, speech recognition is typically assessed in a way that cannot distinguish accurate auditory perception from misperception that was mentally repaired or inferred from context. Previous work showed that repairing misperceptions elicits greater listening effort, and that this elevated effort lingers well after the sentence is heard. That result suggests that cognitive repair strategies might appear successful when testing a single utterance but fail for everyday continuous conversational speech. The current study tested the hypothesis that the effort of repairing misperceptions has the consequence of interfering with perception of later words after the sentence. MethodStimuli were open-set coherent sentences that were presented intact or with a word early in the sentence replaced with noise, forcing the listener to use later context to mentally repair the missing word. Sentences were immediately followed by digit triplets, which served to probe the lasting effects of the effort from the sentence. Control conditions allowed for the comparison to intact sentences that did not demand mental repair, as well as listening conditions that removed the need to attend to the post-sentence stimuli, or removed the post-sentence digits altogether. Intelligibility for the sentences and digits were accompanied by time-series measurements of pupil dilation to assess cognitive load during the task, as well as subjective rating of effort. Participants included adults with cochlear implants, as well as an age-matched and a younger group of listeners with typical hearing (TH) for comparison. ResultsFor the CI group, needing to repair a missing word resulted in more mistakes on the subsequent digits, especially when the repair process did not result in a coherent response. Mentally repaired sentences also resulted in greater prevalence of errors within the sentence itself (even when the masked word was successfully repaired). All groups showed a clear increase of pupil dilation when sentences required repair, but that response in the CI group was diminished when needing to attend to digits immediately after the sentence. Younger TH listeners showed clear differences in moment-to-moment allocation of effort in the different conditions, while the other groups did not. ConclusionFor CI listeners, the effort of repairing misperceptions in a sentence can last long enough to interfere with words that follow the sentence. This pattern could pose a serious problem for regular communication but would go overlooked in typical testing with single utterances, where a listener has a chance to repair misperceptions before responding. The consequence of this lingering repair process does not demand any physiological data to uncover, as it is evident in behavioral data when sentences are followed immediately by extra probe words such as digits.