Auditory dyadic interactions through the ‘eye’ of the social brain: How visual is the posterior STS interaction region?

Abstract

Human interactions contain potent social cues that not only meet the eye but also the ear. Although research has identified a region in the posterior superior temporal sulcus as being particularly sensitive to visually presented social interactions (SI-pSTS), its response to auditory interactions has not been tested. Here, we used fMRI to explore brain response to auditory interactions, with a focus on temporal regions known to be important in auditory processing and social interaction perception. In Experiment 1, monolingual participants listened to two-speaker conversations (intact or sentence-scrambled) and one-speaker narrations in both a known and unknown language. Speaker number and conversational coherence were explored in separately localised regions-of-interest (ROI). In Experiment 2, bilingual participants were scanned to explore the role of language comprehension. Combining univariate and multivariate analyses, we found initial evidence for a heteromodal response to social interactions in SI-pSTS. Specifically, right SI-pSTS preferred auditory interactions over control stimuli and represented information about both speaker number and interactive coherence. Bilateral temporal voice areas (TVA) showed a similar, but less specific, profile. Exploratory analyses identified another auditory-interaction sensitive area in anterior STS. Indeed, direct comparison suggests modality specific tuning, with SI-pSTS preferring visual information while aSTS prefers auditory information. Altogether, these results suggest that right SI-pSTS is a heteromodal region that represents information about social interactions in both visual and auditory domains. Future work is needed to clarify the roles of TVA and aSTS in auditory interaction perception and further probe right SI-pSTS interaction-selectivity using non-semantic prosodic cues.

Influence of social and semantic contexts on phonetic encoding in naturalistic conversations

“…Thus, we could have expected to find that the difference of performance in processing intact versus scrambled conversations would be higher for dialogues than monologues. Here, previous attempts in finding analogous effect in the brain has been so far unconclusive (Landsiedel & Koldewyn, 2023; Olson et al, 2023), so this interaction, if it exists, has yet to be found. Still, in the current study we found the lowest RTs in processing intact dialogues as compared to every other condition in some experiments.…”

Multidimensional neural representations of social features during movie viewing

“…The current findings on the representations of all three types of speech in STG, MTG, and the auditory cortex align well with the previous literature. The temporal cortex has been long implicated in speech comprehension (Crinion et al, 2003; Lindenberg and Scheef, 2007; Leonard and Chang, 2014) and listening to dialogs between people activates those areas (Landsiedel and Koldewyn, 2023; Santavirta et al, 2023). Intriguingly, we observed subtle differences in how each speaking feature was represented in the mentalization network.…”

“…The copyright holder for this preprint this version posted November 22, 2023. ; https://doi.org/10. 1101 Increased STS responses have been linked to a broad range of social cognitive processes, including perceiving biological motion, goal-directed action, social interaction, and social communication, extracting meaning from speech, mental state inference, and social norm processing (Pelphrey et al, 2004;Herrington et al, 2011;Deen et al, 2015;de Heer et al, 2017;Isik et al, 2017;Pegado et al, 2018;Lee Masson and Isik, 2021;Landsiedel and Koldewyn, 2023;McMahon et al, 2023). The STG and MTG have also been implicated in social cognitive processes, including social signal detection, the integration of verbal and nonverbal social cues, extracting social-affective meaning from observed touch, extracting meaning from speech, and perceiving social communication and antisocial behavior (Price, 2012;Sugiura et al, 2014;Holler et al, 2015;Lee Masson et al, 2018;Santavirta et al, 2023).…”

“…Beyond those systems, prior work has identified regions in the STS that show functional selectively to social interaction in both controlled experiments using simple stimuli (Isik et al, 2017; Walbrin et al, 2018) and in more ecologically valid studies that involve natural viewing (Lee Masson and Isik, 2021; McMahon et al, 2023). Furthermore, the STS has shown selective neural responses to visually presented social communication (McMahon et al, 2023) and speech-based social communication (Landsiedel and Koldewyn, 2023). Recent data-driven work has further suggested that communication and antisocial behavior elicit responses in the STG and MTG, respectively (Santavirta et al, 2023).…”