Audiovisual integration in depth: multisensory binding and gain as a function of distance

Abstract

The integration of information across sensory modalities is dependent on the spatiotemporal characteristics of the stimuli that are paired. Despite large variation in the distance over which events occur in our environment, relatively little is known regarding how stimulus-observer distance affects multisensory integration. Prior work has suggested that exteroceptive stimuli are integrated over larger temporal intervals in near relative to far space, and that larger multisensory facilitations are evident in far relative to near space. Here, we sought to examine the interrelationship between these previously established distance-related features of multisensory processing. Participants performed an audiovisual simultaneity judgment and redundant target task in near and far space, while audiovisual stimuli were presented at a range of temporal delays (i.e., stimulus onset asynchronies). In line with the previous findings, temporal acuity was poorer in near relative to far space. Furthermore, reaction time to asynchronously presented audiovisual targets suggested a temporal window for fast detection—a range of stimuli asynchronies that was also larger in near as compared to far space. However, the range of reaction times over which multisensory response enhancement was observed was limited to a restricted range of relatively small (i.e., 150 ms) asynchronies, and did not differ significantly between near and far space. Furthermore, for synchronous presentations, these distance-related (i.e., near vs. far) modulations in temporal acuity and multisensory gain correlated negatively at an individual subject level. Thus, the findings support the conclusion that multisensory temporal binding and gain are asymmetrically modulated as a function of distance from the observer, and specifies that this relationship is specific for temporally synchronous audiovisual stimulus presentations.Electronic supplementary materialThe online version of this article (10.1007/s00221-018-5274-7) contains supplementary material, which is available to authorized users.

Interactive Parallel Models: No Virginia, Violation of Miller's Race Inequality does not Imply Coactivation and Yes Virginia, Context Invariance is Testable

“…For instance, some investigators have used terms such as binding, integration, multi-modal advantage, etc. (e. g., Raij, Uutela, & Hari, 2000;Harrar, Harris, & Spence, 2017;Noel, Modi, Wallace, & Van der Stoep, 2018).…”

Frontier of Self and Impact Prediction

“…Likewise, other auditory or visuo-tactile integration studies ( Canzoneri et al, 2012 ; Kandula et al, 2015 ) have shown that RTs are shorter when a tactile stimulus is delivered at the impact time of the looming stimulus and suggest that looming stimuli predictively speed up tactile processing. Specifically, the speed of the looming stimulus seems to guide the nervous system in defining a high touch/impact probability window not unlike the multisensory temporal binding window described during the physiological and perceptual binding of two stimuli into the representation of a same and single external source and defining the degree of temporal tolerance of the brain in this binding process ( De Paepe et al, 2016 ; Noel et al, 2016 , 2018b ; for review, see Wallace and Stevenson, 2014 ).…”

Atypical audiovisual temporal function in autism and schizophrenia: similar phenotype, different cause

“…However, different forms of sensory energy (i.e., light vs. sound) travel at distinct speeds and possess different neural transmission times. Hence, stimuli generated simultaneously from an environmental event may not be synchronous when they arrive at the peripheral organs (for example, due to distance; Van der Stoep et al ., ; Noel et al ., , in press), and these differences are likely to be propagated and amplified as the sensory information is processed in the brain. To account for these physical and neural differences, as well as the inherent noisiness involved in multisensory temporal coincidence detection, sensory information from the different modalities is integrated across a substantial temporal range—the temporal binding window (TBW; Dixon & Spitz, ; Lewkowicz, ; Munhall et al ., ; Diederich & Colonius, ; van Wassenhove et al ., ; Stevenson et al ., ; Wallace & Stevenson, ; Noel et al ., , ).…”