Sparse periodicity‐based auditory features explain human performance in a spatial multitalker auditory scene analysis task

Abstract

Human listeners robustly decode speech information from a talker of interest that is embedded in a mixture of spatially distributed interferers. A relevant question is which time-frequency segments of the speech are predominantly used by a listener to solve such a complex Auditory Scene Analysis task. A recent psychoacoustic study investigated the relevance of low signal-to-noise ratio (SNR) components of a target signal on speech intelligibility in a spatial multitalker situation. For this, a three-talker stimulus was manipulated in the spectro-temporal domain such that target speech time-frequency units below a variable SNR threshold (SNR ) were discarded while keeping the interferers unchanged. The psychoacoustic data indicate that only target components at and above a local SNR of about 0 dB contribute to intelligibility. This study applies an auditory scene analysis "glimpsing" model to the same manipulated stimuli. Model data are found to be similar to the human data, supporting the notion of "glimpsing," that is, that salient speech-related information is predominantly used by the auditory system to decode speech embedded in a mixture of sounds, at least for the tested conditions of three overlapping speech signals. This implies that perceptually relevant auditory information is sparse and may be processed with low computational effort, which is relevant for neurophysiological research of scene analysis and novelty processing in the auditory system.

Can You Hear Out the Melody? Testing Musical Scene Perception in Young Normal-Hearing and Older Hearing-Impaired Listeners

“…The timbre task may have even better allowed for successful listening in the dips because it only required to identify the right instrument only from one of the four sequence tones, whereas the pitch sequence task required listeners to extract the full pitch sequence from the mixture. It is possible that yNH listeners require only sparse information comparable to auditory glimpses ( Cooke, 2006 ; Josupeit et al., 2018 ) to identify the instruments present in a mixture—a potential feature of the healthy auditory system that musicians may take for granted when building dense music compositions and productions.…”

Harmonic Cancellation—A Fundamental of Auditory Scene Analysis

“…Discarded pixels are not merely set to zero: they are given zero weight (Cooke et al, 1997). Spectro-temporal glimpsing has been proposed in speech processing applications (Wang & Brown, 2006; Wang, 2008), and to account for human perceptual abilities and derive predictive measures of intelligibility (e.g., Best et al, 2019; Josupeit et al, 2020).…”

Harmonic Cancellation - a Fundamental of Auditory Scene Analysis

“…Spectro-temporal glimpsing has been proposed in speech processing applications (Wang and Brown, 2006), and to account for human perceptual abilities and derive predictive measures of intelligibility (e.g. Best et al, 2019;Josupeit et al, 2020). Discarded patterns are not merely set to zero: they are given zero weight in subsequent processing (Cooke et al, 1997).…”