Published estimates of group differences in multisensory integration are inflated

Abstract

Acknowledgments:We thank Kristen Smith for help in reviewing studies of the McGurk effect. Effect inflation in MSI 2 AbstractA common measure of multisensory integration is the McGurk effect, an illusion in which incongruent auditory and visual speech are integrated to produce an entirely different percept. Published studies report that participants who differ in age, gender, culture, native language, or traits related to neurological or psychiatric disorders also differ in their susceptibility to the McGurk effect. These group-level differences are used as evidence for fundamental alterations in sensory processing between populations. Using empirical data and statistical simulations tested under a range of conditions, we show that published estimates of group differences in the McGurk effect are inflated. With a sample size typical of published studies, a group difference of 10% would be reported as 31%. As a consequence of this inflation, follow-up studies often fail to replicate published reports of large between-group differences.Inaccurate estimates of effect sizes and replication failures are especially problematic in studies of clinical populations involving expensive and time-consuming interventions, such as training paradigms to improve sensory processing. Reducing effect size inflation and increasing replicability requires increasing the number of participants by an order of magnitude compared with current practice.Effect inflation in MSI 3

A relationship between Autism-Spectrum Quotient and face viewing behavior in 98 participants

“…If there are large differences within groups, as there are known to be for face viewing, then detecting a difference between groups requires a larger sample size than those commonly reported. For instance, to detect a difference between groups of 20% (higher than that reported in the literature) with 80% power, a sample size of 50 participants per group is required [27], larger than nearly all of the studies in the literature. Underpowered studies are problematic in two different ways: they may fail to detect actual differences (type II error) or they may over-estimate the size of the difference due to the existence of the so-called statistical significance filter.…”

“…Underpowered studies are problematic in two different ways: they may fail to detect actual differences (type II error) or they may over-estimate the size of the difference due to the existence of the so-called statistical significance filter. If many underpowered studies or statistical comparisons are performed and only the significant comparisons are reported, between-group difference can be inflated by as much as 10-fold [27].…”

Bayesian causal inference in visuotactile integration in children and adults

“…We note that gender distributions were not matched across the three participant groups. Based on adult studies, it has been suggested that women have a stronger tendency to bind multisensory signals than men (Barnett‐Cowan et al., 2010; Claypoole & Brill, 2019; Collignon et al., 2010; but see Magnotti & Beauchamp, 2018). However, our results suggests that the adult group, which consisted of slightly more females, showed the smallest prior on common cause, and thus the weakest tendency to bind signals.…”

Treatment of Age-Related Hearing Loss Alters Audiovisual Integration and Resting-State Functional Connectivity: A Randomized Controlled Pilot Trial

“…On a different note, it should be mentioned, that previous research showed rather large variability in the susceptibility of the McGurk illusion (Magnotti and Beauchamp, 2018;Mallick et al, 2015).…”

“…Differences in susceptibility were attributed to age, gender, culture or native language (Magnotti and Beauchamp, 2018) but also lip-reading abilities (Brown et al, 2018;Strand et al, 2014). Similarly, high variation for fusion effects have been reported for other audio-visual paradigms as for instance the sound-flash illusion (de Haas et al, 2012).…”