On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies

Abstract

A growing number of brain imaging studies are being undertaken in order to better understand the contributions of multisensory processes to human behavior and perception. Many of these studies are designed on the basis of the physiological findings from single neurons in animal models, which have shown that multisensory neurons have the capacity for integrating their different sensory inputs and give rise to a product that differs significantly from either of the unisensory responses. At certain points these multisensory interactions can be superadditive, resulting in a neural response that exceeds the sum of the unisensory responses. Because of the difficulties inherent in interpreting the results of imaging large neuronal populations, superadditivity has been put forth as a stringent criterion for identifying potential sites of multisensory integration. In the present manuscript we discuss issues related to using the superadditive model in human brain imaging studies, focusing on population responses to multisensory stimuli and the relationship between single neuron measures and functional brain imaging measures. We suggest that the results of brain imaging studies be interpreted with caution in regards to multisensory integration. Future directions for imaging multisensory integration are discussed in light of the ideas presented.

Touch, sound and vision in human superior temporal sulcus

“…That is, the response to auditory-tactile stimuli was greater than the response to auditory or tactile stimuli in isolation, but was not greater than the summed response to auditory and tactile unisensory stimuli (Stein and Meredith, 1993). Previous fMRI studies of auditory-visual integration in STS (Beauchamp et al, 2004a;Beauchamp et al, 2004b;Hein et al, 2007;Van Atteveldt et al, 2004;van Atteveldt et al, 2007) and auditory-tactile integration in auditory cortex (Kayser et al, 2005) have also not observed super-additive changes in the BOLD signal, perhaps because only a few single neurons show super-additivity (Laurienti et al, 2005;Perrault et al, 2005). Supporting this idea, in single-unit recording studies, only a small fraction of STP neurons respond to both auditory and tactile stimulation (Bruce et al, 1981;Hikosaka et al, 1988); the same is true in multisensory regions of cat cortex (Clemo et al, 2007).…”

Neural correlates of audiovisual speech processing in a second language

“…This set of criteria, inherited from single cell physiology, has been shown to have its advantages (it is safe against falsepositives from areas containing separate populations of visual and auditory unisensory neurons). However, it may be overly conservative (Beauchamp, 2005;Goebel and van Atteveldt, 2009;Laurienti et al, 2005) due to the saturation effects in the BOLD signal and its dependence on the relative proportion of multisensory to unisensory neurons in a given region (e.g., pSTS: Laurienti et al, 2005). Therefore, we decided to use the max criterion, as described by Beauchamp (2005), in the present study to reveal the multimodal responses to L1 and L2 AV speech processing.…”

“…Therefore, we decided to use the max criterion, as described by Beauchamp (2005), in the present study to reveal the multimodal responses to L1 and L2 AV speech processing. Beauchamp proposed that the multisensory response should be greater than the maximum of the unisensory responses (for further reading on these and other related issues, see Beauchamp, 2005;Goebel and van Atteveldt, 2009;Laurienti et al, 2005). For regions which survived the max criteria, we further explored the multisensory interaction pattern by looking at non-linearity using the approach described by van Attenveldt et al (2007).…”

Auditory–Visual Multisensory Interactions in Humans: Timing, Topography, Directionality, and Sources

“…Analyses of voltage waveforms do not unequivocally disambiguate the directionality of effects due to their dependence on the choice of the recording reference (Murray et al, 2008). In animal models, there is an increasing recognition of the preponderance of subadditive effects (Laurienti et al, 2005;Kayser et al, 2008;Angelaki et al, 2009). Another is whether nonlinearities stem from modulations in response strength and/or response topography, the latter of which would indicate the recruitment of distinct configurations of brain generators during multisensory processing (albeit potentially also within low-level cortices).…”