Recalibration of the Multisensory Temporal Window of Integration Results from Changing Task Demands

Abstract

The notion of the temporal window of integration, when applied in a multisensory context, refers to the breadth of the interval across which the brain perceives two stimuli from different sensory modalities as synchronous. It maintains a unitary perception of multisensory events despite physical and biophysical timing differences between the senses. The boundaries of the window can be influenced by attention and past sensory experience. Here we examined whether task demands could also influence the multisensory temporal window of integration. We varied the stimulus onset asynchrony between simple, short-lasting auditory and visual stimuli while participants performed two tasks in separate blocks: a temporal order judgment task that required the discrimination of subtle auditory-visual asynchronies, and a reaction time task to the first incoming stimulus irrespective of its sensory modality. We defined the temporal window of integration as the range of stimulus onset asynchronies where performance was below 75% in the temporal order judgment task, as well as the range of stimulus onset asynchronies where responses showed multisensory facilitation (race model violation) in the reaction time task. In 5 of 11 participants, we observed audio-visual stimulus onset asynchronies where reaction time was significantly accelerated (indicating successful integration in this task) while performance was accurate in the temporal order judgment task (indicating successful segregation in that task). This dissociation suggests that in some participants, the boundaries of the temporal window of integration can adaptively recalibrate in order to optimize performance according to specific task demands.

Relative Unisensory Strength and Timing Predict Their Multisensory Product

“…Therefore, repeated exposure to a particular cross-modal stimulus pairing effectively leads to the stronger-first arrangement that the present results show to be maximally effective. This provides a potential mechanism whereby neurons can adapt and become maximally responsive to those crossmodal cue relationships that are most frequently encountered in their particular environment and they would likely do so quite readily early in life when these relationships are first encountered (Yu et al, 2010;Xu et al, 2012;Stein et al, 2014) and possibly throughout life as a mechanism for temporal recalibration (Fujisaki et al, 2004;Vatakis et al, 2007;Mégevand et al, 2013). However, these possibilities remain to be explored.…”

Putative mechanisms mediating tolerance for audiovisual stimulus onset asynchrony

“…For example, listeners are more tolerant of AVOA for speech than nonspeech and for words than syllables (Vatakis and Spence 2006). Also, the window of AV integration (AVOA tolerance) may recalibrate, expand, or contract, depending on task demands (Mégevand et al 2013;Powers et al 2009Powers et al , 2012Stevenson et al 2013). Recent studies found that training subjects on assessing AV asynchrony for flashes and tone bursts with AV (Powers et al 2009(Powers et al , 2012 or visual-only (Stevenson et al 2013) tasks resulted in tightening of the window of integration.…”

The time window of multisensory integration: Relating reaction times and judgments of temporal order.

“…In the TOJ task, the width of the window determines how often the two stimuli will be "bound together" and thereby how often the subject can only guess that the visual stimulus occurred first, requiring the introduction of a response bias parameter (3 into the model. The viability of this proposal is illustrated by a reanalysis of data from Megevand et al (2013), supporting and extending their hypothesis of a smaller time window for the TOJ task compared with the cross-modal RT task.…”

“…The decision on whether the two window estimates are equal or one is smaller than the other, is then based on the nonparametric Wilcoxon signed-ranks test (Hollander & Wolfe, 1999) applied to these differences. We illustrate this ap proach on the set of data from Megevand et al, (2013) with N = 1, 000.…”

“…The TWIN reanalysis of Megevand et al (2013) Figure 2 shows an example fit of the extended TWIN model for mean bimodal RTs and the TOJ psychometric function across the SOA range. Subsequently, the extended TWIN model was fitted to each of N = 1000 bootstrap samples from the joint RT and TOJ data using the same objective function as before.…”