Dissociation of Perception and Action in Audiovisual Multisensory Integration

Leone, Lynnette; McCourt, Mark E.

doi:10.1167/15.12.854

Cited by 7 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That rapid recalibration should be contingent on the physical asynchronies to which one has previously been exposed rather than on one's subjective evaluation of these asynchronies implies that one need not be consciously aware of the temporal structure of events during everyday tasks to experience the perceptual benefits that recalibration may afford. This notion is consistent with results from recent studies by Leone and McCourt (2015) and Harrar, Harris, and Spence (2017) showing that multisensory integration depends on physical temporal alignment and not on the perceived temporal alignment.…”

Section: Discussionsupporting

confidence: 93%

“…Although it may seem unlikely at first that the true physical timing would be available as a basis for recalibration, several recent studies have suggested that estimates of physical timing are indeed available. Leone and McCourt (2015) showed that in a multisensory context timing for action is veridical, despite perceived time lags, and work by Harrar, Harris, and Spence (2017) also demonstrated that physical audiovisual synchrony, rather than perceived synchrony, affects multisensory integration. Recalibration can occur without feedback (unsupervised recalibration; Zaidel, Ma, & Angelaki, 2013), and various factors may guide the computation of physical synchrony, such as internal models of spatial layout and distances and known lags for various actions and modalities.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Rapid recalibration to audiovisual asynchrony follows the physical—not the perceived—temporal order

Burg

Alais

Cass

2018

Atten Percept Psychophys

View full text Add to dashboard Cite

In natural scenes, audiovisual events deriving from the same source are synchronized at their origin. However, from the perspective of the observer, there are likely to be significant multisensory delays due to physical and neural latencies. Fortunately, our brain appears to compensate for the resulting latency differences by rapidly adapting to asynchronous audiovisual events by shifting the point of subjective synchrony (PSS) in the direction of the leading modality of the most recent event. Here we examined whether it is the perceived modality order of this prior lag or its physical order that determines the direction of the subsequent rapid recalibration. On each experimental trial, a brief tone pip and flash were presented across a range of stimulus onset asynchronies (SOAs). The participants' task alternated over trials: On adaptor trials, audition either led or lagged vision with fixed SOAs, and participants judged the order of the audiovisual event; on test trials, the SOA as well as the modality order varied randomly, and participants judged whether or not the event was synchronized. For test trials, we showed that the PSS shifted in the direction of the physical rather than the perceived (reported) modality order of the preceding adaptor trial. These results suggest that rapid temporal recalibration is determined by the physical timing of the preceding events, not by one's prior perceptual decisions.

show abstract

Section: Discussionsupporting

confidence: 93%

mentioning

confidence: 99%

Rapid recalibration to audiovisual asynchrony follows the physical—not the perceived—temporal order

Burg

Alais

Cass

2018

Atten Percept Psychophys

View full text Add to dashboard Cite

show abstract

“…Almost all the emphasis in empirical work has been whether or not the race inequality was violated. If a violation occurred, the inference, according to the operational definition (Miller, 1982), was that the system was coactive (i.e., Schröger & Widmann, 1998;Besle, Fort, Delpuech, & Giard, 2004;Leone & McCourt, 2015), a particular variant of parallel processing as we noted. Even before initiating our present more technical analyses, we pause to note that this had to be falacious or at least exceedingly suspect due to our earlier demonstrations that quite pedestrian systems, even serial processing systems, could cause violations of that bound (Townsend & Nozawa, 1997).…”

Section: Inmentioning

confidence: 91%

Interactive Parallel Models: No Virginia, Violation of Miller's Race Inequality does not Imply Coactivation and Yes Virginia, Context Invariance is Testable

Townsend¹,

Liu²,

Zhang³

et al. 2020

TQMP

View full text Add to dashboard Cite

One vein of our research on psychological systems has focused on parallel processing models in disjunctive (OR) and conjunctive (AND) stopping-rule designs. One branch of that research has emphasized that a common strategy of inference in the OR situations is logically flawed. That strategy equates a violation of the popular Miller race bound with a coactive parallel system. Pointedly, Townsend & Nozawa (1997) revealed that even processing systems associated with extreme limited capacity are capable of violating that bound. With regard to the present investigation, previous theoretical work has proven that interactive parallel models with separate decision criteria on each channel can readily evoke capacity sufficiently super to violate that bound (e. g., Colonius & Townsend, 1997; Townsend & Nozawa, 1995; Townsend & Wenger, 2004). In addition, we have supplemented the usual OR task with an AND task to seek greater testability of architectural, decisional, and capacity mechanisms (e. g., Eidels et al., 2011; Eidels et al., 2015). The present study presents a broad meta-theoretical structure within which the past and new theoretical results are embedded. We further exploit the broad class of stochastic linear systems and discover that interesting classical results from Colonius (1990) can be given an elegant process interpretation within that class. In addition, we learn that conjoining OR with AND data affords an experimental test of the crucial assumption of context invariance, long thought to be untestable.

show abstract

“…A great deal of research has focused on perception of audiovisual simultaneity, capitalizing on behavioural paradigms such as simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks using different measures of perceived simultaneity such as the TBW as well as the point of subjective simultaneity (PSS; for a review, see Keetels & Vroomen, ). These studies showed that perception of simultaneity varies across a number of factors, including individual differences (Stevenson et al ., ), developmental stages (Hillock et al ., ), tasks/stimulus features (van Eijk et al ., ; Stevenson & Wallace, ; Leone & McCourt, ) and attended sensory modality (Zampini et al ., ; Spence & Parise, ). Studies specifically focusing on the plasticity of simultaneity judgments reported temporal recalibration effects (i.e.…”

Section: Introductionmentioning

confidence: 99%

Behavioural evidence for separate mechanisms of audiovisual temporal binding as a function of leading sensory modality

Cecere

Groß

Thut

2016

Eur J of Neuroscience

View full text Add to dashboard Cite

The ability to integrate auditory and visual information is critical for effective perception and interaction with the environment, and is thought to be abnormal in some clinical populations. Several studies have investigated the time window over which audiovisual events are integrated, also called the temporal binding window, and revealed asymmetries depending on the order of audiovisual input (i.e. the leading sense). When judging audiovisual simultaneity, the binding window appears narrower and non-malleable for auditory-leading stimulus pairs and wider and trainable for visual-leading pairs. Here we specifically examined the level of independence of binding mechanisms when auditory-before-visual vs. visual-before-auditory input is bound. Three groups of healthy participants practiced audiovisual simultaneity detection with feedback, selectively training on auditory-leading stimulus pairs (group 1), visual-leading stimulus pairs (group 2) or both (group 3). Subsequently, we tested for learning transfer (crossover) from trained stimulus pairs to non-trained pairs with opposite audiovisual input. Our data confirmed the known asymmetry in size and trainability for auditory-visual vs. visual-auditory binding windows. More importantly, practicing one type of audiovisual integration (e.g. auditory-visual) did not affect the other type (e.g. visual-auditory), even if trainable by within-condition practice. Together, these results provide crucial evidence that audiovisual temporal binding for auditory-leading vs. visual-leading stimulus pairs are independent, possibly tapping into different circuits for audiovisual integration due to engagement of different multisensory sampling mechanisms depending on leading sense. Our results have implications for informing the study of multisensory interactions in healthy participants and clinical populations with dysfunctional multisensory integration.

show abstract

Dissociation of Perception and Action in Audiovisual Multisensory Integration

Cited by 7 publications

References 0 publications

Rapid recalibration to audiovisual asynchrony follows the physical—not the perceived—temporal order

Rapid recalibration to audiovisual asynchrony follows the physical—not the perceived—temporal order

Interactive Parallel Models: No Virginia, Violation of Miller's Race Inequality does not Imply Coactivation and Yes Virginia, Context Invariance is Testable

Behavioural evidence for separate mechanisms of audiovisual temporal binding as a function of leading sensory modality

Contact Info

Product

Resources

About