Audiovisual Temporal Processing in Postlingually Deafened Adults with Cochlear Implants

Abstract

For many cochlear implant (CI) users, visual cues are vitally important for interpreting the impoverished auditory speech information that an implant conveys. Although the temporal relationship between auditory and visual stimuli is crucial for how this information is integrated, audiovisual temporal processing in CI users is poorly understood. In this study, we tested unisensory (auditory alone, visual alone) and multisensory (audiovisual) temporal processing in postlingually deafened CI users (n = 48) and normal-hearing controls (n = 54) using simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks. We varied the timing onsets between the auditory and visual components of either a syllable/viseme or a simple flash/beep pairing, and participants indicated either which stimulus appeared first (TOJ) or if the pair occurred simultaneously (SJ). Results indicate that temporal binding windows—the interval within which stimuli are likely to be perceptually ‘bound’—are not significantly different between groups for either speech or non-speech stimuli. However, the point of subjective simultaneity for speech was less visually leading in CI users, who interestingly, also had improved visual-only TOJ thresholds. Further signal detection analysis suggests that this SJ shift may be due to greater visual bias within the CI group, perhaps reflecting heightened attentional allocation to visual cues.

Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review

“…Indeed, a number of fMRI studies in blind individuals have shown that in the absence of vision, visual cortex activation commonly associated with visual objects is utilized to encode sound objects (Amedi et al 2003;van den Hurk et al 2017;Vetter et al 2014). Similar recruitment of auditory areas and reweighting of visual cues has been found in deaf individuals and cochlear implant users (Benetti et al 2017;Bola et al 2017;Butera et al 2018). Overall, these studies demonstrate the brain's capacity for marked cross-modal plasticity, in which areas normally associated with one sensory modality can be influenced (and even taken over) by other sensory modalities.…”

Visually-biased perception in cochlear implant users: a study of the McGurk and sound-induced flash illusions

“…This novel finding is noteworthy given that relatively little is known about low-level audiovisual stimulus detection in CI users (Stevenson et al 2017). However, based on our prior work, we do know that CI users' temporal judgments of synchrony for these same flashbeep stimuli are indistinguishable from controls in a simultaneity judgement experiment (Butera et al 2018). This suggests that low-level AV temporal function is likely intact in adult CI users, so any broader issues may be subtle.…”

“…(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (Butera et al 2018). This suggests that low-level AV temporal function is likely intact in adult CI users, so any broader issues may be subtle.…”

“…All responses were collected using a standard keyboard. These experiments were part of a larger testing battery in which both trial and task order was pseudorandomized (Butera et al 2018).…”

Rapid Assessment of Non-Verbal Auditory Perception in Normal-Hearing Participants and Cochlear Implant Users

“…The participants were not diagnosed with psychiatric or neurological disorders, as assessed by a short questionnaire at inclusion in the study. The control participants were selected for their hearing-in-quiet capacities rather than their age, leading to age differences between the groups [ 20 , 69 , 90 ]. As age-difference could have played a role in the CI results, we performed Pearson correlations between each score obtained in the Results section and the age of the CI users, with Bonferroni correction for multiple testing.…”