The role of motion in the neural representation of social interactions in the posterior temporal cortex

Landsiedel, Julia; Daughters, Katie; Downing, Paul E.; Koldewyn, Kami

doi:10.1016/j.neuroimage.2022.119533

Cited by 30 publications

(23 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accumulating evidence ( Pitcher et al, 2014 ; Walbrin and Koldewyn, 2019 ; Landsiedel et al, 2022 ) suggests that dynamic visual aspects of social perception (e.g., face, hand and body movements across the visual field) cannot be easily accommodated within the classic dual-stream model for visual perception ( Ungerleider and Mishkin, 1982 ). Accordingly, resting on both anatomical and functional evidence in humans and non-human primates, Pitcher and Ungerleider (2021) proposed the existence of a third visual processing pathway ( Figure 1B ) that projects on the lateral cortical surface from the early visual cortex into the mid-posterior superior temporal sulcus (pSTS) via motion-selective occipito-temporal areas (V5/hMT).…”

Section: Multimodal Processing In Face-to-face Interactions: a Possib...mentioning

confidence: 99%

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

Benetti

Ferrari²,

Pavani³

2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

In face-to-face communication, humans are faced with multiple layers of discontinuous multimodal signals, such as head, face, hand gestures, speech and non-speech sounds, which need to be interpreted as coherent and unified communicative actions. This implies a fundamental computational challenge: optimally binding only signals belonging to the same communicative action while segregating signals that are not connected by the communicative content. How do we achieve such an extraordinary feat, reliably, and efficiently? To address this question, we need to further move the study of human communication beyond speech-centred perspectives and promote a multimodal approach combined with interdisciplinary cooperation. Accordingly, we seek to reconcile two explanatory frameworks recently proposed in psycholinguistics and sensory neuroscience into a neurocognitive model of multimodal face-to-face communication. First, we introduce a psycholinguistic framework that characterises face-to-face communication at three parallel processing levels: multiplex signals, multimodal gestalts and multilevel predictions. Second, we consider the recent proposal of a lateral neural visual pathway specifically dedicated to the dynamic aspects of social perception and reconceive it from a multimodal perspective (“lateral processing pathway”). Third, we reconcile the two frameworks into a neurocognitive model that proposes how multiplex signals, multimodal gestalts, and multilevel predictions may be implemented along the lateral processing pathway. Finally, we advocate a multimodal and multidisciplinary research approach, combining state-of-the-art imaging techniques, computational modelling and artificial intelligence for future empirical testing of our model.

show abstract

Section: Multimodal Processing In Face-to-face Interactions: a Possib...mentioning

confidence: 99%

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

Benetti

Ferrari²,

Pavani³

2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Specifically, the posterior superior temporal sulcus social interaction region (SI-pSTS) has been found to play a key role in the representation of visually perceived dynamic social interactions. Across a range of stimuli that vary in the strength of relevant social cues (e.g., point-light displays, animated shapes, videos of dyads), the SI-pSTS responds about twice as strongly to interacting dyads compared to two independently acting individuals (Isik et al, 2017; Walbrin et al, 2018; Walbrin & Koldewyn, 2019) and shows this selectivity even in naturalistic videos (Landsiedel et al, 2022; Masson & Isik, 2021). Further, multivariate decoding analyses have found that the right SI-pSTS not only discriminates between interactors and non-interactors, but also appears to represent the type and emotional content of interactions (e.g., competing/cooperating; Isik et al, 2017; Walbrin et al, 2018; Walbrin & Koldewyn, 2019).…”

Section: Introductionmentioning

confidence: 99%

Auditory dyadic interactions through the ‘eye’ of the social brain: How visual is the posterior STS interaction region?

Landsiedel

Koldewyn

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Human interactions contain potent social cues that not only meet the eye but also the ear. Although research has identified a region in the posterior superior temporal sulcus as being particularly sensitive to visually presented social interactions (SI-pSTS), its response to auditory interactions has not been tested. Here, we used fMRI to explore brain response to auditory interactions, with a focus on temporal regions known to be important in auditory processing and social interaction perception. In Experiment 1, monolingual participants listened to two-speaker conversations (intact or sentence-scrambled) and one-speaker narrations in both a known and unknown language. Speaker number and conversational coherence were explored in separately localised regions-of-interest (ROI). In Experiment 2, bilingual participants were scanned to explore the role of language comprehension. Combining univariate and multivariate analyses, we found initial evidence for a heteromodal response to social interactions in SI-pSTS. Specifically, right SI-pSTS preferred auditory interactions over control stimuli and represented information about both speaker number and interactive coherence. Bilateral temporal voice areas (TVA) showed a similar, but less specific, profile. Exploratory analyses identified another auditory-interaction sensitive area in anterior STS. Indeed, direct comparison suggests modality specific tuning, with SI-pSTS preferring visual information while aSTS prefers auditory information. Altogether, these results suggest that right SI-pSTS is a heteromodal region that represents information about social interactions in both visual and auditory domains. Future work is needed to clarify the roles of TVA and aSTS in auditory interaction perception and further probe right SI-pSTS interaction-selectivity using non-semantic prosodic cues.

show abstract

“…The dominant mode within social neuroscience has been to seek out specialised neural subsystems dedicated to processing social (as opposed to more general kinds of) information (Apperly et al, 2005; Happé et al, 2017; Saxe & Powell, 2006; Spunt & Adolphs, 2017). This approach has uncovered evidence for the existence of category-sensitive cortex; regions that preferentially activate during the perception of certain social stimuli, such as faces (Kanwisher & Yovel, 2006), bodies (Downing & Kanwisher, 2010), and dyadic social interactions (Landsiedel et al, 2022). It has been argued that more complex inferential processes such as mental state attribution, or Theory of Mind, also engage highly specialised social brain areas (Apperly et al, 2005; Brüne & Brüne-Cohrs, 2006; Dodell-Feder et al, 2011; Gweon et al, 2012; Jacoby et al, 2016; Jenkins et al, 2014; Koster-Hale & Saxe, 2013; Richardson & Saxe, 2020; Ross & Olson, 2010; Saxe & Baron-Cohen, 2006; Saxe & Kanwisher, 2003b; Saxe & Wexler, 2005; Scholz et al, 2009; Simmons et al, 2010).…”

Section: Introductionmentioning

confidence: 99%

Overlapping Neural Correlates Underpin Theory of Mind and Semantic Cognition: Evidence from a Meta-Analysis of 344 Functional Neuroimaging Studies

Balgova,

Diveica,

Jackson

et al. 2023

Preprint

View full text Add to dashboard Cite

Key unanswered questions for cognitive neuroscience include whether social cognition is underpinned by specialised brain regions, and to what extent it simultaneously depends on more domain-general systems. Until we glean a better understanding of the contribution made by domain-general cognitive systems, theories of social cognition will remain fundamentally limited. In the present study, we evaluate a recent and novel proposal that the semantic cognition network plays a crucial role in supporting social processes. We specifically focus on theory of mind (ToM) abilities and adopt a meta-analytic activation likelihood estimation approach to synthesise the results of a large set of functional neuroimaging studies. Our primary aim was to establish the degree of topological overlap between the cortical networks involved in ToM and semantic tasks. Moreover, we sought to account for key methodological differences across the two sets of tasks, including the fact that ToM studies tend to use nonverbal stimuli while the semantics literature is dominated by language-based tasks. We observed extensive overlap between the two networks in regions strongly implicated in semantic cognition, including the anterior temporal lobes and the left temporoparietal junction (TPJ). Activation specific to ToM was identified in the right TPJ, bilateral anterior mPFC, and right precuneus. These findings persisted even after controlling for discrepancies in the types of experimental stimuli used in each domain. Overall, the findings support the claim that ToM draws upon more general semantic retrieval processes and are against the view that ToM is underpinned solely by a domain-specific social neurocognitive system.

show abstract

The role of motion in the neural representation of social interactions in the posterior temporal cortex

Cited by 30 publications

References 58 publications

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience

Auditory dyadic interactions through the ‘eye’ of the social brain: How visual is the posterior STS interaction region?

Overlapping Neural Correlates Underpin Theory of Mind and Semantic Cognition: Evidence from a Meta-Analysis of 344 Functional Neuroimaging Studies

Contact Info

Product

Resources

About