Noise, multisensory integration, and previous response in perceptual disambiguation

Abstract

Sensory information about the state of the world is generally ambiguous. Understanding how the nervous system resolves such ambiguities to infer the actual state of the world is a central quest for sensory neuroscience. However, the computational principles of perceptual disambiguation are still poorly understood: What drives perceptual decision-making between multiple equally valid solutions? Here we investigate how humans gather and combine sensory information–within and across modalities–to disambiguate motion perception in an ambiguous audiovisual display, where two moving stimuli could appear as either streaming through, or bouncing off each other. By combining psychophysical classification tasks with reverse correlation analyses, we identified the particular spatiotemporal stimulus patterns that elicit a stream or a bounce percept, respectively. From that, we developed and tested a computational model for uni- and multi-sensory perceptual disambiguation that tightly replicates human performance. Specifically, disambiguation relies on knowledge of prototypical bouncing events that contain characteristic patterns of motion energy in the dynamic visual display. Next, the visual information is linearly integrated with auditory cues and prior knowledge about the history of recent perceptual interpretations. What is more, we demonstrate that perceptual decision-making with ambiguous displays is systematically driven by noise, whose random patterns not only promote alternation, but also provide signal-like information that biases perception in highly predictable fashion.

Feeling the Beat (and Seeing It, Too): Vibrotactile, Visual, and Bimodal Rate Discrimination

“…We opted not to use this approach, however, because of the amount of data that we would have excluded to accommodate such an analysis. Our full model undoubtedly omitted other potentially consequential variables as well, such as interactions among IPIs, trial-to-trial variation in attention (e.g., Chambers and Pressnitzer, 2014;Parise and Ernst, 2017;Schwiedrzik et al, 2014), and variability in the decision criterion (Cabrera et al, 2015;Mueller and Weidemann, 2008), as discussed above.…”

Multisensory Integration as per Technological Advances: A Review

“…They might alter the clarity of my vision (e.g., by means of a special screen) or my perceived object size (e.g., by means of a shape-changing object). This modulation implies changing and varying the stimulus properties ( Burns and Blohm, 2010 ; Parise and Ernst, 2017 ) requiring precise computer-controlled delivery. However, it has been suggested that many behavioral studies on multisensory integration rely on “century-old measures of behavior—task accuracy and latency” ( Razavi et al, 2020 ) and are commonly constrained by in-laboratory and desktop-based settings ( Wrzus and Mehl, 2015 ).…”

Stability and flexibility in multisensory sampling: insights from perceptual illusions

“…The modulability of TBW challenges the traditional view that hypothesized the existence of a relatively wide window for multisensory integration in humans that, in turn, would be simply insensitive to small differences in the arrival time of stimuli. Psychophysical studies indicate that our brain takes into account the distance of a sound source and physical features of the stimuli (e.g., velocity, degradation, noise) so adapting TBW (Parise and Ernst 2017). Thus, data suggests that we do not have rigid TBWs, but rather modulable and adaptable TBWs.…”