Sparse coding of auditory features for machine hearing in interference

Abstract

A key problem in using the output of an auditory model as the input to a machine-learning system in a machine-hearing application is to find a good feature-extraction layer. For systems such as PAMIR (passive-aggressive model for image retrieval) that work well with a large sparse feature vector, a conversion from auditory images to sparse features is needed. For audio-file ranking and retrieval from text queries, based on stabilized auditory images, we took a multi-scale approach, using vector quantization to choose one sparse feature in each of many overlapping regions of different scales, with the hope that in some regions the features for a sound would be stable even when other interfering sounds were present and affecting other regions. We recently extended our testing of this approach using sound mixtures, and found that the sparse-coded auditory-image features degrade less in interference than vector-quantized MFCC sparse features do. This initial success suggests that our hope of robustness in interference may indeed be realizable, via the general idea of sparse features that are localized in a domain where signal components tend to be localized or stable.

Robust Sound Event Classification Using Deep Neural Networks

“…By default, the nonlinear frequency resolution is 78 and the time lag resolution is 561. In [4], multiple rectangular regions were extracted from each SAI according to a 'start-small and then double' heuristic outlined in [6], starting with an initial window of size D = 16 by B = 32. In total R = 49 rectangular regions were extracted and then downsampled to match the D × B size of the initial window, and the marginals of each region computed as discussed in Section II-A to yield a representative feature vector of size D + B, i.e.…”

“…To further improve efficiency, the large-scale systems [4] perform vector quantisation (VQ) [16] or matching pursuit [34] on each rectangle, and represent the output as a sparse code. In this paper, both VQ and non-VQ results will be presented.…”

“…Where VQ is used, performance tends to increase with codebook size, saturating at a size of around 512. A size of 256 was used in [4].…”

“…16 + 32 = 48 per SAI frame. For the Google-SAI PAMIR-based recall systems [4], the features from each rectangular region are vector quantized using a separate codebook of size 256 for each feature. A sparse representation is then constructed by concatenating the 49 VQ output vectors, forming a super-vector of length 12, 544.…”

“…The SAI features used in this work are derived as defined in [4] using AIM-C [24], extracted over 35 ms windows to yield a real-valued matrix for each analysis frame. By default, the nonlinear frequency resolution is 78 and the time lag resolution is 561.…”