Is neocortex essentially multisensory?

Abstract

Although sensory perception and neurobiology are traditionally investigated one modality at a time, real world behaviour and perception are driven by the integration of information from multiple sensory sources. Mounting evidence suggests that the neural underpinnings of multisensory integration extend into early sensory processing. This article examines the notion that neocortical operations are essentially multisensory. We first review what is known about multisensory processing in higher-order association cortices and then discuss recent anatomical and physiological findings in presumptive unimodal sensory areas. The pervasiveness of multisensory influences on all levels of cortical processing compels us to reconsider thinking about neural processing in unisensory terms. Indeed, the multisensory nature of most, possibly all, of the neocortex forces us to abandon the notion that the senses ever operate independently during real-world cognition.

Multisensory anatomical pathways

“…As mentioned earlier (see also other chapters in this issue), many human studies conducted with EEG or fMRI have revealed multisensory interactions in low-level cortices such as V1 (see Ghazanfar and Schroeder, 2006 for a review and Martuzzi et al, 2007;Romei et al, 2007Romei et al, , 2008 for recent evidences) and/or at a latency that strongly suggests that such phenomena are supported by the heteromodal connections directly linking areas of different sensory modalities. For example, the visual information derived from the lip movements during speech processing can affect directly the responses recorded in the auditory cortex (Besle et al, 2004).…”

“…In this context, given the extensive connections between the neocortex, the superior colliculus and the thalamus, different authors have proposed that subcortical structures integrate the senses even before the neocortex (Ghazanfar and Schroeder, 2006 for review). The thalamus is a possible candidate considering its strong input-output connections with multiple cortical areas, both sensory and motor.…”

“…In support of this view was the absence of strong anatomical links between the areas of different modalities at the level of the first stages of sensory processing (see Jones and Powell, 1970). Recent anatomical and electrophysiological studies in non-human primates as well as human functional brain studies led to a reappraisal of this concept (see Ghazanfar and Schroeder, 2006;Driver and Noesselt, 2008 for recent reviews) and highlight that the mechanisms for multisensory interplay are believed to include several levels of brain processing, from the thalamus to the primary sensory areas and higher stages of sensory processing. Such an increase in the diversity of the pathways by which multisensory interactions can occur is probably reflecting an adaptive mechanism by which individual perceptual or sensory-motor situations involve a specific multisensory network.…”

A deafening flash! Visual interference of auditory signal detection

“…Indeed, neurons responding to visual stimulation have been found in auditory cortex which may receive stimulation from higher areas or from subcortical regions via interconnections found in normal anatomy (Ghazanfar & Schroeder, 2006;Schroeder & Foxe, 2005). Such connections may function to improve spatial localisation of sound or amplify the auditory response to visually identifiable sources (Schroeder & Foxe, 2005).…”

Rapidly acquired multisensory association in the olfactory cortex

“…2009; Smeets and Dijksterhuis 2014). Olfactory sensory memory reflects activity, in part, within the primary olfactory cortex (POC), which like most sensory cortices (Ghazanfar and Schroeder 2006) receives direct and/or indirect convergent multisensory inputs (Gottfried et al. 2004; Wesson and Wilson 2010; Maier et al.…”