Visuo-haptic Neuronal Convergence Demonstrated with an Inversely Effective Pattern of BOLD Activation

Abstract

We investigated the neural substrates involved in visuo-haptic neuronal convergence using an additive-factors design in combination with fMRI. Stimuli were explored under three sensory modality conditions: viewing the object through a mirror without touching (V), touching the object with eyes closed (H), or simultaneously viewing and touching the object (VH). This modality factor was crossed with a task difficulty factor, which had two levels. On the basis of an idea similar to the principle of inverse effectiveness, we predicted that increasing difficulty would increase the relative level of multisensory gain in brain regions where visual and haptic sensory inputs converged. An ROI analysis focused on the lateral occipital tactile-visual area found evidence of inverse effectiveness in the left lateral occipital tactile-visual area, but not in the right. A whole-brain analysis also found evidence for the same pattern in the anterior aspect of the intraparietal sulcus, the premotor cortex, and the posterior insula, all in the left hemisphere. In conclusion, this study is the first to demonstrate visuo-haptic neuronal convergence based on an inversely effective pattern of brain activation.

Multisensory convergence of visual and haptic object preference across development

“…The patterns of activation in the left and right hemispheres were similar for the LOtv and cIPS, which was to be expected as many studies have consistently found bilateral activity for visuohaptic object preference (Amedi et al, 2001, 2002; Saito et al, 2003; Zhang et al, 2004). Yet, it is important to note that previous studies have also reported laterality effects—some have shown what appear to be weaker signals in the right than in the left hemisphere (Kim & James, 2010; Kim, Stevenson, & James, 2012), while others have found task-dependent lateralization in the LOC (Large, Aldcroft, & Vilis, 2007). As such, the issue of laterality remains complex (for a brief, but pertinent discussion of the lateralization of visual and haptic processing, see Stilla & Sathian, 2008).…”

“…Many studies have shown that the convergence of visual and haptic inputs for object preference in adults occurs at two particular sites: the lateral occipital tactile-visual region (LOtv, located within the LOC) for object recognition, and the anterior/middle aspects of the IPS for object-directed motor actions (Amedi et al, 2001, 2002; James et al, 2002a; James & Kim, 2010; Stilla & Sathian, 2008). Several more recent studies have shown that information from vision and haptics is also combined at these sites (Kim & James, 2010; Kim, Stevenson, & James, 2012; Tal & Amedi, 2009). …”

Intertrial Variability in the Premotor Cortex Accounts for Individual Differences in Peripersonal Space

“…Neuroimaging studies investigating multisensory interaction in humans show that BOLD responses to multisensory stimuli can change as a function of both task difficulty and stimulus quality (James et al, 2012;Noppeney, 2012). Specifically, as the difficulty of the task increases (Kim et al, 2012) or the effectiveness of the constituent unisensory stimuli lowers (Werner and Noppeney, 2010), the multisensory enhancement increases. This in turn positively affects the BOLD response.…”

The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities

“…Thus, behavioral and perceptual facilitations have been shown to be greatest for stimuli that are close together in space and time [90, 98-128], and the proportional benefits of combining stimuli across different modalities appear to be greatest when the individual stimuli are weakly effective [18, 21, 22, 113]. In addition, and much like for the neuronal data described above, recent studies have also illustrated the interdependency of these principles in human performance and perception [20, 58, 129].…”