Look into my eyes: Investigating joint attention using interactive eye-tracking and fMRI in a developmental sample

Oberwelland, Eileen; Schilbach, Leonhard; Barisic, Iva; Krall, Sarah; Vogeley, Kai; Fink, Gereon R.; Herpertz‐Dahlmann, Beate; Konrad, Kerstin; Schulte‐Rüther, Martin

doi:10.1016/j.neuroimage.2016.02.026

Cited by 72 publications

(109 citation statements)

References 77 publications

Supporting

Mentioning

100

Contrasting

Unclassified

Order By: Relevance

“…The validity of the predictions of the models in the context of verbal communication was not self-evident. The way of eliciting eye contact in the current study differed in multiple aspects from the ways in previous studies (Conty et al , 2007; Ethofer et al , 2011; von dem Hagen et al , 2014; Cavallo et al , 2015; Oberwelland et al , 2016) on which the models are largely based. First, in our study eye contact occurred in a verbal listening context.…”

Section: Discussionmentioning

confidence: 90%

“…We expect that our experimental paradigm together with other interactive approaches (e.g. Oberwelland et al , 2016) will be a solid foundation for integrating these features in future studies.…”

Section: Discussionmentioning

confidence: 99%

“…In addition, the regions of the social brain network are modulated by the dorsolateral prefrontal cortex (dlPFC) according to task demands and social context. There is solid evidence that these accounts can explain neural mechanisms involved in eye contact in non-verbal situations in which participants view faces with averted or direct gaze (Calder et al , 2002; Kampe et al , 2003; Pelphrey et al , 2004; von dem Hagen et al , 2014; Cavallo et al , 2015; Oberwelland et al , 2016). To date, it is unknown whether the three accounts are also suitable to explain neural mechanisms underlying eye contact in verbal communication.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Neural mechanisms of eye contact when listening to another person talking

Jiang

Borowiak

Tudge

et al. 2016

Soc Cogn Affect Neurosci

View full text Add to dashboard Cite

Eye contact occurs frequently and voluntarily during face-to-face verbal communication. However, the neural mechanisms underlying eye contact when it is accompanied by spoken language remain unexplored to date. Here we used a novel approach, ﬁxation-based event-related functional magnetic resonance imaging (fMRI), to simulate the listener making eye contact with a speaker during verbal communication. Participants’ eye movements and fMRI data were recorded simultaneously while they were freely viewing a pre-recorded speaker talking. The eye tracking data were then used to define events for the fMRI analyses. The results showed that eye contact in contrast to mouth fixation involved visual cortical areas (cuneus, calcarine sulcus), brain regions related to theory of mind/intentionality processing (temporoparietal junction, posterior superior temporal sulcus, medial prefrontal cortex) and the dorsolateral prefrontal cortex. In addition, increased effective connectivity was found between these regions for eye contact in contrast to mouth fixations. The results provide first evidence for neural mechanisms underlying eye contact when watching and listening to another person talking. The network we found might be well suited for processing the intentions of communication partners during eye contact in verbal communication.

show abstract

Section: Discussionmentioning

confidence: 90%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Neural mechanisms of eye contact when listening to another person talking

Jiang

Borowiak

Tudge

et al. 2016

Soc Cogn Affect Neurosci

View full text Add to dashboard Cite

show abstract

“…Similar tasks have been used in other fMRI studies using either gaze-contingent avatars (Oberwelland et al, in press) or live-video links to a real social partner (Redcay et al, 2012; Saito et al, 2010). Together, these interactive paradigms represent an important step towards an ecologically valid measure of joint attention.…”

Section: Introductionmentioning

confidence: 99%

Detecting communicative intent in a computerised test of joint attention

Caruana

McArthur

Woolgar

et al. 2017

PeerJ

View full text Add to dashboard Cite

The successful navigation of social interactions depends on a range of cognitive faculties—including the ability to achieve joint attention with others to share information and experiences. We investigated the influence that intention monitoring processes have on gaze-following response times during joint attention. We employed a virtual reality task in which 16 healthy adults engaged in a collaborative game with a virtual partner to locate a target in a visual array. In the Search task, the virtual partner was programmed to engage in non-communicative gaze shifts in search of the target, establish eye contact, and then display a communicative gaze shift to guide the participant to the target. In the NoSearch task, the virtual partner simply established eye contact and then made a single communicative gaze shift towards the target (i.e., there were no non-communicative gaze shifts in search of the target). Thus, only the Search task required participants to monitor their partner’s communicative intent before responding to joint attention bids. We found that gaze following was significantly slower in the Search task than the NoSearch task. However, the same effect on response times was not observed when participants completed non-social control versions of the Search and NoSearch tasks, in which the avatar’s gaze was replaced by arrow cues. These data demonstrate that the intention monitoring processes involved in differentiating communicative and non-communicative gaze shifts during the Search task had a measurable influence on subsequent joint attention behaviour. The empirical and methodological implications of these findings for the fields of autism and social neuroscience will be discussed.

show abstract

“…When approached by a virtual character exhibiting an angry expression, increased activation was found in the participant’s superior temporal sulcus, lateral fusiform gyrus, and a region of the middle temporal gyrus [104]. Studies have also used virtual characters for studies of joint attention [18,105,106,107,108,109]. Schilbach and colleagues [18,108] have introduced dynamic virtual characters in the scanner to characterize the neural correlates of being involved in social interactions.…”

Section: Virtual Reality For Simulating Impossible Social Interactmentioning

confidence: 99%

Virtual Reality for Research in Social Neuroscience

2017

View full text Add to dashboard Cite

The emergence of social neuroscience has significantly advanced our understanding of the relationship that exists between social processes and their neurobiological underpinnings. Social neuroscience research often involves the use of simple and static stimuli lacking many of the potentially important aspects of real world activities and social interactions. Whilst this research has merit, there is a growing interest in the presentation of dynamic stimuli in a manner that allows researchers to assess the integrative processes carried out by perceivers over time. Herein, we discuss the potential of virtual reality for enhancing ecological validity while maintaining experimental control in social neuroscience research. Virtual reality is a technology that allows for the creation of fully interactive, three-dimensional computerized models of social situations that can be fully controlled by the experimenter. Furthermore, the introduction of interactive virtual characters—either driven by a human or by a computer—allows the researcher to test, in a systematic and independent manner, the effects of various social cues. We first introduce key technical features and concepts related to virtual reality. Next, we discuss the potential of this technology for enhancing social neuroscience protocols, drawing on illustrative experiments from the literature.

show abstract

Look into my eyes: Investigating joint attention using interactive eye-tracking and fMRI in a developmental sample

Cited by 72 publications

References 77 publications

Neural mechanisms of eye contact when listening to another person talking

Neural mechanisms of eye contact when listening to another person talking

Detecting communicative intent in a computerised test of joint attention

Virtual Reality for Research in Social Neuroscience

Contact Info

Product

Resources

About