Naturalistic Stimulus Structure Determines the Integration of Audiovisual Looming Signals in Binocular Rivalry

Abstract

Rapid integration of biologically relevant information is crucial for the survival of an organism. Most prominently, humans should be biased to attend and respond to looming stimuli that signal approaching danger (e.g. predator) and hence require rapid action. This psychophysics study used binocular rivalry to investigate the perceptual advantage of looming (relative to receding) visual signals (i.e. looming bias) and how this bias can be influenced by concurrent auditory looming/receding stimuli and the statistical structure of the auditory and visual signals.Subjects were dichoptically presented with looming/receding visual stimuli that were paired with looming or receding sounds. The visual signals conformed to two different statistical structures: (1) a ‘simple’ random-dot kinematogram showing a starfield and (2) a “naturalistic” visual Shepard stimulus. Likewise, the looming/receding sound was (1) a simple amplitude- and frequency-modulated (AM-FM) tone or (2) a complex Shepard tone. Our results show that the perceptual looming bias (i.e. the increase in dominance times for looming versus receding percepts) is amplified by looming sounds, yet reduced and even converted into a receding bias by receding sounds. Moreover, the influence of looming/receding sounds on the visual looming bias depends on the statistical structure of both the visual and auditory signals. It is enhanced when audiovisual signals are Shepard stimuli.In conclusion, visual perception prioritizes processing of biologically significant looming stimuli especially when paired with looming auditory signals. Critically, these audiovisual interactions are amplified for statistically complex signals that are more naturalistic and known to engage neural processing at multiple levels of the cortical hierarchy.

Does direction of walking impact binocular rivalry between competing patterns of optic flow?

“…This realization led us to analyze EXP and CON dominance durations measured during the sitting condition in Experiments 2 and 3, where one's own actions do not imply self-production of optic flow on the retina. In that combined data set, EXP dominance durations were significantly longer than were CON durations (3.72 vs 3.175 s, T-test P = .003, ks-test P = .00006), mirroring earlier findings (Conrad et al, 2013;Malek et al, 2012; but see Parker & Alais, 2007). So, perhaps walking vetoes the sense of looming and pending collision signified by centrifugal radial motion viewed by a stationary observer.…”

“…Turning to other aspects of our findings, we did not find a reliable tendency for EXP to enjoy longer dominance durations than CON-a result consistent with one other study (Parker & Alais, 2007), but inconsistent with another (Conrad et al, 2013). The authors of the study that did find EXP > CON durations opined that this bias for visual looming could arise from the association of looming with potential danger or collision.…”

“…First, a person with a tube inserted into one nostril is more likely to perceive a given line drawing of an object (e.g., a rose) presented on a video monitor as dominant in rivalry when the inhaled odor being delivered through the tube (e.g., floral) is congruent with the visual object (Zhou et al, 2010). Second, an AMand FM-modulated tone heard over headphones biases rivalry when conflicting looming and receding motion displays are viewed dichoptically on a pair of video monitors (Conrad et al, 2013). And third, a rotating rival target generated on a video monitor predominates more during rivalry when an observer simultaneously moves a hand-held mouse back and forth in rhythm with the pattern of rotation of the visual stimulus (Maruya et al, 2007).…”

“…Ours is not the first study to employ complex optic flow patterns as rival stimuli, but it is the first to do so in the context of self-motion. In several earlier studies using rival targets simulating complex motion patterns, it was found that visual motion simulating expansion enjoyed a bias during rivalry over dots simulating contraction, although there was some disagreement among these studies about the specific dependence of that bias on the form of the motion stimuli (Parker & Alais, 2007;Malek et al, 2012;Conrad et al, 2013;Holten et al, 2015). Our aim, however, was to learn whether the predominance of expansion and contraction was dependent on one's walking direction.…”

Distributed neural signatures of natural audiovisual speech and music in the human auditory cortex

“…The trade-off was that the stimulus was not the most common type of narrative speech but more like listening to poetry reading. As many previous studies (Beauchamp et al 2004, Romanski & Hwang 2012, Wayne & Johnsrude 2012, Conrad et al 2013, Li et al 2015, we used complex naturalistic stimuli in order to activate widespread temporal cortex areas associated with audiovisual processing. Such complex stimulation is important also, because it includes nuanced spectro-temporal features that are critical in discriminating between real-life music and speech.…”