Experience Creates the Multisensory Transform in the Superior Colliculus

Abstract

Although the ability to integrate information across the senses is compromised in some individuals for unknown reasons, similar defects have been observed when animals are reared without multisensory experience. The experience-dependent development of multisensory integration has been studied most extensively using the visual-auditory neuron of the cat superior colliculus (SC) as a neural model. In the normally-developed adult, SC neurons react to concordant visual-auditory stimuli by integrating their inputs in real-time to produce non-linearly amplified multisensory responses. However, when prevented from gathering visual-auditory experience, their multisensory responses are no more robust than their responses to the individual component stimuli. The mechanisms operating in this defective state are poorly understood. Here we examined the responses of SC neurons in "naïve" (i.e., dark-reared) and "neurotypic" (i.e., normallyreared) animals on a millisecond-by-millisecond basis to determine whether multisensory experience changes the operation by which unisensory signals are converted into multisensory outputs (the "multisensory transform"), or whether it changes the dynamics of the unisensory inputs to that transform (e.g., their synchronization and/or alignment). The results reveal that the major impact of experience was on the multisensory transform itself. Whereas neurotypic multisensory responses exhibited non-linear amplification near their onset followed by linear amplification thereafter, the naive responses showed no integration in the initial phase of the response and a computation consistent with competition in its later phases. The results suggest that multisensory experience creates an entirely new computation by which convergent unisensory inputs are used cooperatively to enhance the physiological salience of cross-modal events and thereby facilitate normal perception and behavior.

The tectum/superior colliculus as the vertebrate solution for spatial sensory integration and action

“…The integration between vision and auditory stimuli has been studied in considerable detail in the cat 158 and in primates including humans 159,160 . As in other vertebrates, visual and auditory stimuli originating from the same point in space facilitate each other, and conversely, if they originate from different areas in space, they will instead inhibit each other 161 .…”

Multisensory enhancement of overt behavior requires multisensory experience

“…Previously, a variety of metrics have been used to assess the magnitude of multisensory enhancement in behavioral and physiological studies (Stein & Meredith, 1993, p. 93; Meredith & Stein, 1983; Rowland, Quessy, et al., 2007; Gingras et al., 2009; Stanford et al., 2005; Colonius & Diederich, 2017). The statistical facilitation measure adopted here (Ma et al., 2006) is particularly well suited to evaluate performance in compromised populations (Wang et al., 2020). Conceptually, it is similar to the “race model” metric (Miller, 1982) that is used as a criterion to determine whether there is multisensory enhancement in reaction time tasks.…”

“…Multisensory enhancement was evaluated by comparing the multisensory responses to the predictions of a model of “statistical facilitation” (SF) based on the visual and auditory responses (also known as “separate‐activation,” see Gondan et al., 2005; Miller, 1982; Nava et al., 2014; Otto et al., 2013; Wang et al, 2020). The statistical facilitation referent represents what is theoretically achievable from unisensory processing alone, and is equivalent to the “best” unisensory performance calculated on a trial‐by‐trial basis (Stein et al., 2009).…”

“…These midbrain neurons can integrate signals transduced from spatiotemporally congruent visual and auditory stimuli (e.g., stimuli derived from the same events) to significantly enhance their physiological responses (Meredith & Stein, 1983; Rowland et al., 2007; Stein & Meredith, 1993; Stein & Stanford, 2008). However, this capacity is not innate nor does it simply reflect the simple convergence of afferents from different senses onto common target neurons (Stein et al., 2014; Wallace & Stein, 1997, 2001; Wang et al., 2020; Yu et al., 2013). Rather, it is a capability beyond simple convergence that must develop based on experience.…”

“…It is not until they have obtained sufficient experience with particular input combinations that they are able to elicit enhanced multisensory responses (Stein et al, 2014). Thus, when an animal is reared in the dark, or in omnidirectional masking noise, both of which preclude patterned visual-auditory experience, SC neurons respond to the combined cues no better than to the most effective of them individually (Wallace et al, 2004;Wang et al, 2020;Xu et al, 2012Xu et al, , 2014Xu et al, , 2017Yu et al, 2010Yu et al, , 2013. Their multisensory responses may even be less robust than their best unisensory response due to an inherent competition between the different sensory inputs (Yu et al, 2018).…”