Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness

Senkowski, Daniel; Saint-Amour, Dave; Höfle, Marion; Foxe, John J.

doi:10.1016/j.neuroimage.2011.03.075

Cited by 142 publications

(118 citation statements)

References 46 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…6a, arrows). Hence, our results agree with a recent study showing that the RSE does not follow the PoIE when interpreted in terms of physical signal strength (Chandrasekaran et al, 2011; but see Senkowski et al, 2011).…”

Section: Experiments 2: Signal Strengthsupporting

confidence: 93%

“…Typically, response times (RTs) are faster in redundant compared with single signal conditions (which is the RSE). In our experiments, we used visual motion and auditory sound signals embedded in an audiovisual background, but in principle any combination of signals can be studied in both humans and nonhumans (Diederich and Colonius, 1987;Giray and Ulrich, 1993;Mordkoff and Yantis, 1993;Hughes et al, 1994;Schröger and Widmann, 1998;Gondan et al, 2004;Molholm et al, 2004;Murray et al, 2005;Katzner et al, 2006;Whitchurch and Takahashi, 2006;Martuzzi et al, 2007;Collignon et al, 2008;Hecht et al, 2008;Hirokawa et al, 2008;Tamietto and de Gelder, 2008;Poom, 2009;Suied et al, 2009;Zehetleitner et al, 2009;Cappe et al, 2010;Veldhuizen et al, 2010;Chandrasekaran et al, 2011;Senkowski et al, 2011;Brang et al, 2012;Buchholz et al, 2012). To uncover the conditions that maximize multisensory benefits in behavior, we varied the relative onset (experiment 1) and the strength (experiment 2) of the redundant motion and sound signals.…”

Section: Methodsmentioning

confidence: 99%

“…Enhancements of neuronal responses in multisensory compared with unisensory conditions are largest when physical signals occur approximately at the same location ("spatial rule"), when signals are presented at approximately the same time ("temporal rule"), and when signal strength is rather low, eliciting only weak unisensory responses ("principle of inverse effectiveness"; PoIE). In view of this influential work, the same three principles are often also considered with respect to behavioral benefits that can be observed with multisensory signals (for recent examples, see Chandrasekaran et al, 2011;Senkowski et al, 2011;Buchholz et al, 2012;Cappe et al, 2012;Stevenson et al, 2012). However, it is remarkable that, for example, the speedup of responses in multisensory compared with unisensory conditions (known as the redundant signal effect or RSE; Todd, 1912;Hershenson, 1962;Miller, 1982) is actually not following these principles.…”

Section: Introductionmentioning

confidence: 99%

“…The temporal rule fails because the RSE with auditory and visual signals is typically largest when auditory signals are delayed, but the optimal delay is highly variable for different signals (Hershenson, 1962;Miller, 1986). Finally, the PoIE fails at least when the physical signal strength is manipulated in some conditions (Chandrasekaran et al, 2011) but not in others (Senkowski et al, 2011). In summary, it is unclear how to best predict behavioral benefits with multisensory signals such as the speedup of responses and how to bridge the apparent gap between electrophysiology and behavior.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Principles of Multisensory Behavior

2013

View full text Add to dashboard Cite

The combined use of multisensory signals is often beneficial. Based on neuronal recordings in the superior colliculus of cats, three basic rules were formulated to describe the effectiveness of multisensory signals: the enhancement of neuronal responses to multisensory compared with unisensory signals is largest when signals occur at the same location ("spatial rule"), when signals are presented at the same time ("temporal rule"), and when signals are rather weak ("principle of inverse effectiveness"). These rules are also considered with respect to multisensory benefits as observed with behavioral measures, but do they capture these benefits best? To uncover the principles that rule benefits in multisensory behavior, we here investigated the classical redundant signal effect (RSE; i.e., the speedup of response times in multisensory compared with unisensory conditions) in humans. Based on theoretical considerations using probability summation, we derived two alternative principles to explain the effect. First, the "principle of congruent effectiveness" states that the benefit in multisensory behavior (here the speedup of response times) is largest when behavioral performance in corresponding unisensory conditions is similar. Second, the "variability rule" states that the benefit is largest when performance in corresponding unisensory conditions is unreliable. We then tested these predictions in two experiments, in which we manipulated the relative onset and the physical strength of distinct audiovisual signals. Our results, which are based on a systematic analysis of response time distributions, show that the RSE follows these principles very well, thereby providing compelling evidence in favor of probability summation as the underlying combination rule.

show abstract

Section: Experiments 2: Signal Strengthsupporting

confidence: 93%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Principles of Multisensory Behavior

2013

View full text Add to dashboard Cite

show abstract

“…To understand better the dynamics of multisensory processes gauged in perceptual tasks, we applied these analytical methods to ERPs in response to task-irrelevant AV stimuli that required top-down attention but no motor responses (Cappe et al, 2010 These studies have typically reported multisensory interactions in the ERPs measured at individual electrodes onsetting ~40-55ms and visible as a parieto-occipital positivity in the difference map at the scalp (Fort et al, 2002a;Giard and Peronnet, 1999;Molholm et al, 2002;Vidal et al, 2008;Senkowski et al, 2011;Stekelenburg and Vroomen, 2012;Altieri et al, 2014; see also Stevenson et al, 2012 as well as Barth et al, 1995 for corresponding data in the rat brain). The analysis of ERPs evoked by attended but task-irrelevant stimuli that required no motor response circumvented the caveat concerning motor-related activity (Gondan and Röder, 2006;Teder-Sälejärvi et al, 2002).…”

Section: Electromagnetic Signals (Eeg/meg)mentioning

confidence: 99%

The multisensory function of the human primary visual cortex

et al. 2016

View full text Add to dashboard Cite

2It has been nearly ten years since Ghazanfar and Schroeder (2006) proposed that the neocortex is essentially multisensory in nature. However, it is only recently that sufficient and hard evidence that supports this proposal has accrued. We review evidence that activity within the human primary visual cortex plays an active role in multisensory processes and directly impacts behavioural outcome. This evidence emerges from a full pallet of human brain imaging and brain mapping methods with which multisensory processes are quantitatively assessed by taking advantage of particular strengths of each technique as well as advances in signal analyses. Several general conclusions about multisensory processes in primary visual cortex of humans are supported relatively solidly. First, haemodynamic methods (fMRI/PET) show that there is both convergence and integration occurring within primary visual cortex. Second, primary visual cortex is involved in multisensory processes during early post-stimulus stages (as revealed by EEG/ERP/ERFs as well as TMS). Third, multisensory effects in primary visual cortex directly impact behaviour and perception, as revealed by correlational (EEG/ERPs/ERFs) as well as more causal measures (TMS/tACS). While the provocative claim of Ghazanfar and Schroeder (2006) that the whole of neocortex is multisensory in function has yet to be demonstrated, this can now be considered established in the case of the human primary visual cortex.

show abstract

Auditory and audio–visual processing in patients with cochlear, auditory brainstem, and auditory midbrain implants: An EEG study

Schierholz

Finke

Kral

et al. 2017

Human Brain Mapping

View full text Add to dashboard Cite

There is substantial variability in speech recognition ability across patients with cochlear implants (CIs), auditory brainstem implants (ABIs), and auditory midbrain implants (AMIs). To better understand how this variability is related to central processing differences, the current electroencephalography (EEG) study compared hearing abilities and auditory-cortex activation in patients with electrical stimulation at different sites of the auditory pathway. Three different groups of patients with auditory implants (Hannover Medical School; ABI: n = 6, CI: n = 6; AMI: n = 2) performed a speeded response task and a speech recognition test with auditory, visual, and audio-visual stimuli. Behavioral performance and cortical processing of auditory and audio-visual stimuli were compared between groups. ABI and AMI patients showed prolonged response times on auditory and audio-visual stimuli compared with NH listeners and CI patients. This was confirmed by prolonged N1 latencies and reduced N1 amplitudes in ABI and AMI patients. However, patients with central auditory implants showed a remarkable gain in performance when visual and auditory input was combined, in both speech and non-speech conditions, which was reflected by a strong visual modulation of auditory-cortex activation in these individuals. In sum, the results suggest that the behavioral improvement for audio-visual conditions in central auditory implant patients is based on enhanced audio-visual interactions in the auditory cortex. Their findings may provide important implications for the optimization of electrical stimulation and rehabilitation strategies in patients with central auditory prostheses. Hum Brain Mapp 38:2206-2225, 2017. © 2017 Wiley Periodicals, Inc.

show abstract

Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness

Cited by 142 publications

References 46 publications

Principles of Multisensory Behavior

Principles of Multisensory Behavior

The multisensory function of the human primary visual cortex

Auditory and audio–visual processing in patients with cochlear, auditory brainstem, and auditory midbrain implants: An EEG study

Contact Info

Product

Resources

About