Modeling speech localization, talker identification, and word recognition in a multi-talker setting

Abstract

This study introduces a model for solving three different auditory tasks in a multi-talker setting: target localization, target identification, and word recognition. The model was used to simulate psychoacoustic data from a call-sign-based listening test involving multiple spatially separated talkers [Brungart and Simpson (2007). Percept. Psychophys. 69(1), 79–91]. The main characteristics of the model are (i) the extraction of salient auditory features (“glimpses”) from the multi-talker signal and (ii) the use of a classification method that finds the best target hypothesis by comparing feature templates from clean target signals to the glimpses derived from the multi-talker mixture. The four features used were periodicity, periodic energy, and periodicity-based interaural time and level differences. The model results widely exceeded probability of chance for all subtasks and conditions, and generally coincided strongly with the subject data. This indicates that, despite their sparsity, glimpses provide sufficient information about a complex auditory scene. This also suggests that complex source superposition models may not be needed for auditory scene analysis. Instead, simple models of clean speech may be sufficient to decode even complex multi-talker scenes.

Sparse periodicity‐based auditory features explain human performance in a spatial multitalker auditory scene analysis task

“…If the model by Josupeit and Hohmann (2017) can predict the SRM, this would support the statement of Schoenmaker and van de Par (2016) that BMLD does not contribute to SRM, at least in the investigated auditory scene with three competing talkers and no additional background noise. Another question of interest is whether the model word recognition scores are similar to those of the subjects.…”

“…The standard deviation depended on the type of auditory feature used. In line with Josupeit and Hohmann (), they were set toσ E = 1 dB, σ T = 25 μs, σ L = 1 dB. For each cns ‐bin and each auditory feature, the kernels were summed up and normalized, so that they formed a probability density function (PDF). In the following, these PDF functions are termed f X ( x , c , n , s | w ), where identifies the feature type, x identifies the function variable, and w identifies the word.…”

“…Several studies indicate that listeners mainly process "glimpses" of information to decode complex auditory scenes (Best et al, 2017;Cooke, 2006;Josupeit & Hohmann, 2017;Schoenmaker & van de Par, 2016). Glimpses are defined as spectro-temporal undisrupted pieces of information (Darwin, 2008), that is a glimpse contains the information from exactly one sound source.…”

“…To further investigate this finding, this study applies an auditory "glimpsing" model (Josupeit & Hohmann, 2017) to the stimuli of the study by Schoenmaker and van de Par (2016). In particular, it is investigated whether the model is able to predict the SRM to be constant in the SNR range from À15 dB to 0 dB.…”

“…To further investigate this question, this study tested the hypothesis that low signal-to-noise ratio target components do not contribute significantly to speech discrimination in a spatial multitalker condition. For this, we apply an auditory "glimpsing" model (Josupeit & Hohmann, 2017) to a speech intelligibility task within a multitalker scene with a manipulated target stimulus (Schoenmaker & van de Par, 2016).…”