How Visual Cues for when to Listen Aid Selective Auditory Attention

Abstract

Visual cues are known to aid auditory processing when they provide direct information about signal content, as in lip reading. However, some studies hint that visual cues also aid auditory perception by guiding attention to the target in a mixture of similar sounds. The current study directly tests this idea for complex, nonspeech auditory signals, using a visual cue providing only timing information about the target. Listeners were asked to identify a target zebra finch bird song played at a random time within a longer, competing masker. Two different maskers were used: noise and a chorus of competing bird songs. On half of all trials, a visual cue indicated the timing of the target within the masker. For the noise masker, the visual cue did not affect performance when target and masker were from the same location, but improved performance when target and masker were in different locations. In contrast, for the chorus masker, visual cues improved performance only when target and masker were perceived as coming from the same direction. These results suggest that simple visual cues for when to listen improve target identification by enhancing sounds near the threshold of audibility when the target is energetically masked and by enhancing segregation when it is difficult to direct selective attention to the target. Visual cues help little when target and masker already differ in attributes that enable listeners to engage selective auditory attention effectively, including differences in spectrotemporal structure and in perceived location.

Effect of signal-temporal uncertainty in children and adults: Tone detection in noise or a random-frequency masker

“…The present findings are in general agreement with recent studies which have examined the extent to which adults benefit from knowing when to listen for conditions believed to produce substantial informational masking (Best et al, 2007;Varghese et al, 2012). observed a larger effect in a random-frequency, two-tone masker than in a broadband noise for a single group of adult listeners.…”

“…Thus, defining the listening interval either through a visual cue or allowing the listener to initiate the trial appears to result in a similar benefit of knowing when to listen for a 1000-Hz signal embedded in a random-frequency, two-tone masker. A similar pattern of results has also been reported for bird song stimuli (Best et al, 2007;Varghese et al, 2012). Varghese et al (2012) asked listeners to detect a target bird song presented in either a chorus of novel bird songs or in a noise masker which had the same long-term average spectral characteristics.…”

“…The effect of signaltemporal uncertainty appears to be relatively small ( 3 dB) for adults when the signal is embedded in a masker thought to primarily produce energetic masking (e.g., Egan et al, 1961;Green and Weber, 1980;Watson and Nichols, 1976). In contrast to these findings, recent studies indicate a larger effect of signal-temporal uncertainty with maskers thought to produce substantial informational masking (Best et al, 2007;Varghese et al, 2012; for exception see Richards et al, 2011). For example, Best et al (2007) found that listeners could more accurately identify a target birdsong embedded in other birdsongs when they were provided with a cue indicating the time interval that contained the target.…”

“…For example, Best et al (2007) found that listeners could more accurately identify a target birdsong embedded in other birdsongs when they were provided with a cue indicating the time interval that contained the target. A follow-up study by Varghese et al (2012) demonstrated that the benefit of this cue was larger when the masker was comprised of birdsongs (high informational masking) than when it was a spectrally-matched noise (low informational masking). reported a similar pattern of results for tonal stimuli.…”

Attention during social interaction in children with autism: Comparison to specific language impairment, typical development, and links to social cognition

“…We know from the literature on typical development that visual cues from the face have an influence on speech perception. For example, focusing on lip movements can enhance the perception of speech when the quality of the speech is degraded (Varghese, Ozmeral, Best & Shinn-Cunningham, 2012) and when talking on the telephone we need to modify our communication strategy (Doherty-Sneddon & Kent, 1996). Therefore, it may be difficulties in communication (processing of speech and language) that drive a preferential attention bias to the mouth region in autism.…”