A Framework for Speech Activity Detection Using Adaptive Auditory Receptive Fields

Abstract

One of the hallmarks of sound processing in the brain is the ability of the nervous system to adapt to changing behavioral demands and surrounding soundscapes. It can dynamically shift sensory and cognitive resources to focus on relevant sounds. Neurophysiological studies indicate that this ability is supported by adaptively retuning the shapes of cortical spectro-temporal receptive fields (STRFs) to enhance features of target sounds while suppressing those of task-irrelevant distractors. Because an important component of human communication is the ability of a listener to dynamically track speech in noisy environments, the solution obtained by auditory neurophysiology implies a useful adaptation strategy for speech activity detection (SAD). SAD is an important first step in a number of automated speech processing systems, and performance is often reduced in highly noisy environments. In this paper, we describe how task-driven adaptation is induced in an ensemble of neurophysiological STRFs, and show how speech-adapted STRFs reorient themselves to enhance spectro-temporal modulations of speech while suppressing those associated with a variety of nonspeech sounds. We then show how an adapted ensemble of STRFs can better detect speech in unseen noisy environments compared to an unadapted ensemble and a noise-robust baseline. Finally, we use a stimulus reconstruction task to demonstrate how the adapted STRF ensemble better captures the spectrotemporal modulations of attended speech in clean and noisy conditions. Our results suggest that a biologically plausible adaptation framework can be applied to speech processing systems to dynamically adapt feature representations for improving noise robustness.

Modelling auditory attention

“…Top-down task–based adaptations have been incorporated in attention systems by modelling the attentional gain as weights in the classifier to optimize performance based on specific task goals [77,78], or as a separate cognitive model deciding which speaker to attend to among competing sources [79]. A more holistic attention mechanism has instead used the goal-directed adaptation framework of physiological STRFs as a pre-processing stage to speech recognition, by enabling the separation of the target speech stream from the distractor soundscape it is embedded in [80]. The attentional filter provides significant gain to the target speech while being robust to previously unseen noise types.…”

Receptive Field Regularization Techniques for Audio Classification and Tagging with Deep Convolutional Neural Networks

“…Several neurophysiological studies have focused on understanding the ability of humans and animals to tune their cortical Spectro-Temporal Receptive Fields (STRFs) in order to selectively focus on target sounds, while minimizing the irrelevant acoustics and noise background [25]- [28]. Building on such studies, Carlin and Elhilali [25] trained a Gaussian Mixture Model on features obtained from both the initial and adapted STRFs. They showed that an ensemble of adapted STRFs achieves better performance in detecting speech, in the presence of noise.…”