Trainable frontend for robust and far-field keyword spotting

Wang, Yuxuan; Getreuer, Pascal; Hughes, T. A.; Lyon, Richard F.; Saurous, Rif A.

doi:10.1109/icassp.2017.7953242

Cited by 103 publications

(99 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Definition Per-channel energy normalization (PCEN) [70] has recently been proposed as an alternative to the logarithmic transformation of the mel-spectrogram (logmelspec), with the aim of combining dynamic range compression (DRC, also present in logmelspec) and adaptive gain control (AGC) with temporal integration. AGC is a prior stage to DRC involving a low-pass filter φ T of support T , thus yielding…”

Section: Per-channel Energy Normalizationmentioning

confidence: 99%

Robust sound event detection in bioacoustic sensor networks

et al. 2019

View full text Add to dashboard Cite

Bioacoustic sensors, sometimes known as autonomous recording units (ARUs), can record sounds of wildlife over long periods of time in scalable and minimally invasive ways. Deriving per-species abundance estimates from these sensors requires detection, classification, and quantification of animal vocalizations as individual acoustic events. Yet, variability in ambient noise, both over time and across sensors, hinders the reliability of current automated systems for sound event detection (SED), such as convolutional neural networks (CNN) in the time-frequency domain. In this article, we develop, benchmark, and combine several machine listening techniques to improve the generalizability of SED models across heterogeneous acoustic environments. As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise. Starting from a CNN yielding state-of-the-art accuracy on this task, we introduce two noise adaptation techniques, respectively integrating short-term (60 ms) and long-term (30 min) context. First, we apply per-channel energy normalization (PCEN) in the time-frequency domain, which applies short-term automatic gain control to every subband in the mel-frequency spectrogram. Secondly, we replace the last dense layer in the network by a context-adaptive neural network (CA-NN) layer, i.e. an affine layer whose weights are dynamically adapted at prediction time by an auxiliary network taking long-term summary statistics of spectrotemporal features as input. We show that PCEN reduces temporal overfitting across dawn vs. dusk audio clips whereas context adaptation on PCEN-based summary statistics reduces spatial overfitting across sensor locations. Moreover, combining them yields state-of-the-art results that are unmatched by artificial data augmentation alone. We release a pre-trained version of our best performing system under the name of BirdVoxDetect, a ready-to-use detector of avian flight calls in field recordings.

show abstract

Section: Per-channel Energy Normalizationmentioning

confidence: 99%

Robust sound event detection in bioacoustic sensor networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…1. The encoder takes an acoustic feature x[t], t = 1, 2, ..., T , the 40-dimensional Mel-filter bank energies extracted from 16 kHz sampled audio signals with per-channel energy normalization [13] where t is the time frame index, as input and converts it into a hidden representation h[t]. The encoder network consists of a canonical CRNN structure with convolutional and recurrent layers in sequence to capture spectral and temporal characteristics of the acoustic features.…”

Section: Abstractpotting System Descriptionmentioning

confidence: 99%

Orthogonality Constrained Multi-Head Attention for Keyword Spotting

Lee¹,

Lee²,

Jang³

et al. 2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequence into one context vector. However, a naive use of the multi-head attention does not guarantee such richness as the attention heads may have positional and representational redundancy. In this paper, we propose a regularization technique for multi-head attention mechanism in an end-to-end neural keyword spotting system. Augmenting regularization terms which penalize positional and contextual non-orthogonality between the attention heads encourages to output different representations from separate subsequences, which in turn enables leveraging structured information without explicit sequence models such as hidden Markov models. In addition, intra-head contextual non-orthogonality regularization encourages each attention head to have similar representations across keyword examples, which helps classification by reducing feature variability. The experimental results demonstrate that the proposed regularization technique significantly improves the keyword spotting performance for the keyword "Hey Snapdragon".

show abstract

“…Yet, in recent years, the systematic use of machine learning methods has progressively reduced the need for domain-specific knowledge in several other aspects of auditory perception, including melfrequency spectrum [11] and adaptive gain control [12]. It remains to be known whether octave equivalence can, in turn, be discovered by a machine learning algorithm, instead of being engineered ad hoc.…”

Section: Introductionmentioning

confidence: 99%

Learning the Helix Topology of Musical Pitch

Lostanlen

Sridhar

McFee

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

To explain the consonance of octaves, music psychologists represent pitch as a helix where azimuth and axial coordinate correspond to pitch class and pitch height respectively. This article addresses the problem of discovering this helical structure from unlabeled audio data. We measure Pearson correlations in the constant-Q transform (CQT) domain to build a K-nearest neighbor graph between frequency subbands. Then, we run the Isomap manifold learning algorithm to represent this graph in a three-dimensional space in which straight lines approximate graph geodesics. Experiments on isolated musical notes demonstrate that the resulting manifold resembles a helix which makes a full turn at every octave. A circular shape is also found in English speech, but not in urban noise. We discuss the impact of various design choices on the visualization: instrumentarium, loudness mapping function, and number of neighbors K.

show abstract

Trainable frontend for robust and far-field keyword spotting

Cited by 103 publications

References 11 publications

Robust sound event detection in bioacoustic sensor networks

Robust sound event detection in bioacoustic sensor networks

Orthogonality Constrained Multi-Head Attention for Keyword Spotting

Learning the Helix Topology of Musical Pitch

Contact Info

Product

Resources

About