Neural mechanisms of eye contact when listening to another person talking

Abstract

Eye contact occurs frequently and voluntarily during face-to-face verbal communication. However, the neural mechanisms underlying eye contact when it is accompanied by spoken language remain unexplored to date. Here we used a novel approach, fixation-based event-related functional magnetic resonance imaging (fMRI), to simulate the listener making eye contact with a speaker during verbal communication. Participants’ eye movements and fMRI data were recorded simultaneously while they were freely viewing a pre-recorded speaker talking. The eye tracking data were then used to define events for the fMRI analyses. The results showed that eye contact in contrast to mouth fixation involved visual cortical areas (cuneus, calcarine sulcus), brain regions related to theory of mind/intentionality processing (temporoparietal junction, posterior superior temporal sulcus, medial prefrontal cortex) and the dorsolateral prefrontal cortex. In addition, increased effective connectivity was found between these regions for eye contact in contrast to mouth fixations. The results provide first evidence for neural mechanisms underlying eye contact when watching and listening to another person talking. The network we found might be well suited for processing the intentions of communication partners during eye contact in verbal communication.

Free viewing of talking faces reveals mouth and eye preferring regions of the human superior temporal sulcus

“…Like Jiang et al (2016), we observed a preference for eye observations in the occipital lobe and adjacent inferior ventral and parieto-occipital cortex.…”

“…While we were most interested in the anterior pSTS mouth-preferring region, our study complements the recent study of Jiang et al (2016) who focused on brain regions, including posterior pSTS, that were more active when participant fixated the eyes of talking faces. Like Jiang et al (2016), we observed a preference for eye observations in the occipital lobe and adjacent inferior ventral and parieto-occipital cortex.…”

“…In the present study, the center-of-mass of mouth-preferring activity in the pSTS was located 6 mm more anterior than eye-preferring activity (y = −43 vs. −49 in the left hemisphere, −40 vs. −46 on the right; see Table 1), similar to the values reported by Zhu & Beauchamp (2017) (6.5 mm; y = −46 vs. −52 in left hemisphere and y = −43 vs. −50, right hemisphere). Pelphrey et al (2005) reported that right-hemisphere mouth-preferring activity was 21 mm anterior to eye-preferring activity (y = −37 vs. −58) while Jiang et al (2016) reported 24 mm, y = −33 vs. −57). The larger anterior-posterior differences in these studies could be attributable to their use of spatial-smoothing filters on the fMRI data.…”

“…A recent fMRI study capitalized on the existence of these extended fixations to search for the neural correlates of fixating the eyes of a dynamic talking face (Jiang et al, 2016). The authors describe the existence of an “eye contact network” that includes portions of the pSTS, the temporo-parietal junction and other brain areas.…”

Dorsal‐movement and ventral‐form regions are functionally connected during visual‐speech recognition

“…Some studies reported that individuals with ASD gaze less to the face and the mouth during visual-speech recognition compared to typically developing controls (Irwin & Brancazio, 2014;Irwin, Tornatore, Brancazio, & Whalen, 2011, but see Foxe et al, 2015Saalasti et al, 2012). Since gaze behavior influences brain responses to faces (Dalton et al, 2005;Jiang, Borowiak, Tudge, Otto, & von Kriegstein, 2017), we used an eye tracker in the MRI environment to assess where participants looked during visual-speech recognition.…”