A biologically oriented algorithm for spatial sound segregation

Abstract

Listening in an acoustically cluttered scene remains a difficult task for both machines and hearing-impaired listeners. Normal-hearing listeners accomplish this task with relative ease by segregating the scene into its constituent sound sources, then selecting and attending to a target source. An assistive listening device that mimics the biological mechanisms underlying this behavior may provide an effective solution for those with difficulty listening in acoustically cluttered environments (e.g., a cocktail party). Here, we present a binaural sound segregation algorithm based on a hierarchical network model of the auditory system. In the algorithm, binaural sound inputs first drive populations of neurons tuned to specific spatial locations and frequencies. The spiking response of neurons in the output layer are then reconstructed into audible waveforms via a novel reconstruction method. We evaluate the performance of the algorithm with a speech-on-speech intelligibility task in normal-hearing listeners. This two-microphone-input algorithm is shown to provide listeners with perceptual benefit similar to that of a 16-microphone acoustic beamformer. These results demonstrate the promise of this biologically inspired algorithm for enhancing selective listening in challenging multi-talker scenes.

A brain-inspired algorithm improves “cocktail party” listening for individuals with hearing loss

“…29 STOI ranges between 0 and 1 and can be used to predict speech intelligibility when combined with an appropriate mapping function. The time-constant of the alpha function kernel (τ h ), and the scaling factor for DiffMask ( a ) (see Chou et al 14 for details) were chosen by iteratively trying a range of values, quantifying algorithm performance using STOI, then choosing parameter values that produced the highest average STOI. For RatioMask, the beta (ß) parameter value of 1.65 was selected using the genetic algorithm (GA) function in the MATLAB Global Optimization Toolbox with “fitness” defined as the average STOI value.…”

“…The subtractive term acts as a mechanism for sharpening the spatial tuning of output neurons as demonstrated in a previous publication. 14 The scale factor a was adjusted to normalize the mask to a maximum value of 1. In contrast to DiffMask, which employs a subtractive operation, RatioMask implements a multiplicative operation to sharpen the spatial tuning of output neurons in the presence of competing noise.…”

“…11,12 These approaches, however, are computationally expensive and not yet well-suited for low-power real-time applications. 13 We recently developed a biologically oriented sound segregation algorithm 14 (BOSSA), which is designed to separate competing sounds based on differences in spatial location. Taking its inspiration from binaural auditory system, this algorithm requires only two input signals, and does not sacrifice spatial cues.…”

fNIRS Dataset During Complex Scene Analysis

“…CSA can be broken down into several distinct components: determining the spatial location of a target stimulus, segregating the target stimulus from the competing stimuli in the scene, and reconstructing the target stimulus from the mixture forming a perceptual object (Bizley, Maddox and Lee, 2016). Recently, we proposed a brain-inspired algorithm for auditory scene analysis based on a model of cortical neurons (Maddox et al, 2012; Dong, Colburn and Sen, 2016; Chou et al, 2022). This algorithm has the potential to be applied in assistive devices for CSA.…”

fNIRS dataset during complex scene analysis

“…Recently, we proposed a brain-inspired algorithm for auditory scene analysis based on a model of cortical neurons (Maddox et al, 2012 ; Dong et al, 2016 ; Chou et al, 2022 ). This algorithm has the potential to be applied in assistive devices for CSA.…”