Temporal Cortex Activation to Audiovisual Speech in Normal-Hearing and Cochlear Implant Users Measured with Functional Near-Infrared Spectroscopy

Abstract

Background: Speech understanding may rely not only on auditory, but also on visual information. Non-invasive functional neuroimaging techniques can expose the neural processes underlying the integration of multisensory processes required for speech understanding in humans. Nevertheless, noise (from functional MRI, fMRI) limits the usefulness in auditory experiments, and electromagnetic artifacts caused by electronic implants worn by subjects can severely distort the scans (EEG, fMRI). Therefore, we assessed audio-visual activation of temporal cortex with a silent, optical neuroimaging technique: functional near-infrared spectroscopy (fNIRS).Methods: We studied temporal cortical activation as represented by concentration changes of oxy- and deoxy-hemoglobin in four, easy-to-apply fNIRS optical channels of 33 normal-hearing adult subjects and five post-lingually deaf cochlear implant (CI) users in response to supra-threshold unisensory auditory and visual, as well as to congruent auditory-visual speech stimuli.Results: Activation effects were not visible from single fNIRS channels. However, by discounting physiological noise through reference channel subtraction (RCS), auditory, visual and audiovisual (AV) speech stimuli evoked concentration changes for all sensory modalities in both cohorts (p < 0.001). Auditory stimulation evoked larger concentration changes than visual stimuli (p < 0.001). A saturation effect was observed for the AV condition.Conclusions: Physiological, systemic noise can be removed from fNIRS signals by RCS. The observed multisensory enhancement of an auditory cortical channel can be plausibly described by a simple addition of the auditory and visual signals with saturation.

Measuring Cortical Activity During Auditory Processing With Functional Near-Infrared Spectroscopy

“…In Figure 3 an example is shown of how, at the single-subject level, RCS affects the average response during auditory stimulation. In general, it improves the signal response (Figure 3B; for further explanation see (27)).…”

“…Figure 2 shows the probe template for two optical sources (S) and one photodetector (D) using source–detector distances of 25 and 35 mm, respectively (termed reference or shallow, and deep channel, respectively) (27). In this figure, the detector records the transmitted light coming from two sources, and each source–detector combination is defined as a channel.…”

“…These physiological sources of noise, such as heart beat, respiration, or Mayer waves (44) may hide experimental effects which are usually of much smaller amplitude, and it may require sophisticated methods to identify the latter. Using a ‘reference channel’ offers a possible way to increase the reliability of estimating the hemodynamic response from fNIRS signals (27,44,45). A reference channel is characterized by a short source–detector distance (range of 1–2 cm, see Figure 2), and makes use of the direct relation between source–detector distance and depth reached by photons in tissues underlying the scalp (46–48).…”

“…The sources and detectors were positioned over the left temporal hemisphere. Image adapted from Van de Rijt et al, 2016 (27). …”

“…Red dotted lines denote the average path from source to detector, estimated to be part of an ellipsoid with a penetration depth of approximately 2–3 cm. Image adapted from Van de Rijt et al, 2016 (27). …”