The Ventriloquist Effect Results from Near-Optimal Bimodal Integration

Abstract

Italy second stimulus was seen to the left of the first, as a function of actual physical displacement. Following 2 Auditory Neuroscience Laboratory Department of Physiology standard practice, the data were fitted with cumulative Gaussian functions free to vary in position and width: University of Sydney New South Wales 2006 the position of the median (50% leftward) is termed the point of subjective equality (PSE), and the width Australia 3 Department of Psychology represents the estimate of localization accuracy (pre-sumed to depend on internal noise). For all unimodal University of Florence 50125 Florence conditions, the PSE was near 0, but varied considerably. For visual stimuli, was smallest (approximately Italy 1) for the smallest Gaussian blobs and largest (approxi-mately 20) for the largest (in line with the results of [6]). Localization accuracy for the auditory click was around Summary 6, falling midway between the various visual stimuli. Note that this poor localization does not necessarily Ventriloquism is the ancient art of making one's voice reflect performance under natural or "free-field" condi-appear to come from elsewhere, an art exploited by tions where all auditory cues (including intensity differ-the Greek and Roman oracles, and possibly earlier ences and monaural cues) are available. Free-field local-[1]. We regularly experience the effect when watching ization of spectrally rich stimuli such as click trains television and movies, where the voices seem to ema-produces discrimination thresholds on the order of 1 nate from the actors' lips rather than from the actual [7, 8]. sound source. Originally, ventriloquism was explained With unimodal thresholds established, we then mea-by performers projecting sound to their puppets by sured localization for a blob and click presented simulta-special techniques [1], but more recently it is assumed neously. Observers were asked to envisage each pre-that ventriloquism results from vision "capturing" sentation as a single event, like a ball hitting the screen, sound [2-5]. In this study we investigate spatial local-and report which of the two presentations was more ization of audiovisual stimuli. When visual localization leftward. For one presentation (randomly first or sec-is good, vision does indeed dominate and capture ond), the visual and auditory stimuli were in conflict, sound. However, for severely blurred visual stimuli with the visual stimulus displaced degrees rightward (that are poorly localized), the reverse holds: sound and the auditory stimulus displaced degrees leftward captures vision. For less blurred stimuli, neither sense (S V S A 2, where S V and S A are the spatial positions dominates and perception follows the mean position. of the visual and auditory stimuli). The average position Precision of bimodal localization is usually better than of this stimulus was always zero, as in previous studies either the visual or the auditory unimodal presentation. [9]. On the other (non-conflict) presentation, the two All the results are well expla...

A comparison of the McGurk effect for spoken and sung syllables

“…A number of researchers p p y p have proposed theories of how auditory and visual input are integrated to create a unified percept. According to optimal integration theory, the relative influence of each modality depends on its perceived reliability in conveying information (Alais & Burr, 2004;Massaro, 2004).…”

Multisensory cue combination after sensory loss: Audio-visual localization in patients with progressive retinal disease.

“…Alais & Burr, 2004;Ernst & Banks, 2002 Overall, the results show two patient groups with progressive sensory loss, one succeeding and one failing at combining cues according to the MLE rule. Why might the group with central loss, in particular, have failed to weight cues by reliability?…”

“…Alais & Burr, 2004;Ernst & Banks, 2002). However, less is known about how human adults adapt to the gradual changes in sensory reliability that occur during ageing or disease.…”

“…Given that adults are able to reweight sensory cues from trial to trial, in line with short-term experimental manipulations to the cue reliabilities, (e.g. Alais & Burr, 2004;Ernst & Banks, 2002), why might they fail to account for longer-term changes?…”

“…For example, there could be a general reliability setting for a particular sensory cue (e.g. a visual cue to location; Alais & Burr, 2004) that is immediately modulated by the specific sensory information on a particular trial, but whose overall setting is more difficult to change. However, in development and ageing there is also the possibility  7 that the cue combination process itself is immature or deficient (e.g.…”