Keeping in touch with the visual system: spatial alignment and multisensory integration of visual-somatosensory inputs

Abstract

Correlated sensory inputs coursing along the individual sensory processing hierarchies arrive at multisensory convergence zones in cortex where inputs are processed in an integrative manner. The exact hierarchical level of multisensory convergence zones and the timing of their inputs are still under debate, although increasingly, evidence points to multisensory integration (MSI) at very early sensory processing levels. While MSI is said to be governed by stimulus properties including space, time, and magnitude, violations of these rules have been documented. The objective of the current study was to determine, both psychophysically and electrophysiologically, whether differential visual-somatosensory (VS) integration patterns exist for stimuli presented to the same versus opposite hemifields. Using high-density electrical mapping and complementary psychophysical data, we examined multisensory integrative processing for combinations of visual and somatosensory inputs presented to both left and right spatial locations. We assessed how early during sensory processing VS interactions were seen in the event-related potential and whether spatial alignment of the visual and somatosensory elements resulted in differential integration effects. Reaction times to all VS pairings were significantly faster than those to the unisensory conditions, regardless of spatial alignment, pointing to engagement of integrative multisensory processing in all conditions. In support, electrophysiological results revealed significant differences between multisensory simultaneous VS and summed V + S responses, regardless of the spatial alignment of the constituent inputs. Nonetheless, multisensory effects were earlier in the aligned conditions, and were found to be particularly robust in the case of right-sided inputs (beginning at just 55 ms). In contrast to previous work on audio-visual and audio-somatosensory inputs, the current work suggests a degree of spatial specificity to the earliest detectable multisensory integrative effects in response to VS pairings.

Developmental Recovery of Impaired Multisensory Processing in Autism and the Cost of Switching Sensory Modality

“…While a race model account of the RSE predicts that a response is triggered independently by the fastest sensory modality (Raab, 1962), the RSE typically exceeds the benefit predicted by mere statistical facilitation (Miller, 1982). Violation of the race model has been demonstrated using bisensory detection tasks for several decades and is widely interpreted as reflecting the multisensory gain due to pooled or integrated information processing (Gielen et al, 1983;Miller, 1986;Diederich and Colonius, 1987;Harrington and Peck, 1998;Molholm et al, 2002;Murray et al, 2004;Stevenson et al, 2012;Mégevand et al, 2013;Mahoney et al, 2015;Innes and Otto, 2019).…”

Stimulus intensity modulates multisensory temporal processing

“…Although these studies have illustrated the central importance of time in dictating human multisensory interactions, other studies have focused on the roles of space (Bertelson & Radeau, 1981; Ghose & Wallace, 2014; Kadunce, Vaughan, Wallace, & Stein, 2001; Krueger, Royal, Fister, & Wallace, 2009; Macaluso, et al, 2004; Mahoney, et al, 2015; Meredith & Stein, 1986, 1996; Radeau & Bertelson, 1974; Royal, Carriere, & Wallace, 2009; Royal, Krueger, Fister, & Wallace, 2010; Sarko, et al, 2012; Vroomen, Bertelson, & de Gelder, 2001; Wallace, et al., 2004) and effectiveness (James & Stevenson, 2012a; James, et al., 2012; Kim & James, 2010; Kim, Stevenson, & James, 2012; Leone & McCourt, 2013; Liu, Lin, Gao, & Dang, 2013; Nath & Beauchamp, 2011; Stevenson & James, 2009; Werner & Noppeney, 2010; Yalachkov, Kaiser, Doehrmann, & Naumer, 2015). Collectively, we have learned a great deal from these studies about how stimulus-related factors shape the multisensory process, but most have treated time, space and effectiveness as independent contributors to the final multisensory product.…”

Retinal innervation tunes circuits that drive nonphotic entrainment to food

“… Daily changes in light and food availability are major time-cues influencing circadian timing 1 . Little is known, however, about the circuits integrating these time-cues to drive a coherent circadian output 1 – 3 . Here, we investigated whether retinal inputs modulate the entrainment to non-photic cues, such as time-restricted feeding.…”

“…The circadian system is composed of a central circadian pacemaker, housed in the suprachiasmatic nucleus (SCN), which orchestrates rhythmic functions of peripheral clocks located throughout the body 1 . This system integrates multiple time-cues from the sensory, as well as the circadian and metabolic systems to generate a coherent perception of the environment 2 , 3 . At present, little is known about the brain circuits and mechanisms that integrate different time-cues to drive a coordinated circadian output.…”