Very deep convolutional neural networks for raw waveforms

Abstract

Learning acoustic models directly from the raw waveform data with minimal processing is challenging. Current waveform-based models have generally used very few (∼2) convolutional layers, which might be insufficient for building high-level discriminative features. In this work, we propose very deep convolutional neural networks (CNNs) that directly use time-domain waveforms as inputs. Our CNNs, with up to 34 weight layers, are efficient to optimize over very long sequences (e.g., vector of size 32000), necessary for processing acoustic waveforms. This is achieved through batch normalization, residual learning, and a careful design of down-sampling in the initial layers. Our networks are fully convolutional, without the use of fully connected layers and dropout, to maximize representation learning. We use a large receptive field in the first convolutional layer to mimic bandpass filters, but very small receptive fields subsequently to control the model capacity. We demonstrate the performance gains with the deeper models. Our evaluation shows that the CNN with 18 weight layers outperform the CNN with 3 weight layers by over 15% in absolute accuracy for an environmental sound recognition task and matches the performance of models using log-mel features.

SampleCNN: End-to-End Deep Convolutional Neural Networks Using Very Small Filters for Music Classification

“…The majority of them used large-sized filters in the first convolutional layer with various size of strides to capture frequency-selective responses which were carefully designed to handle their target problems. We termed this approach as frame-level raw waveform model because the filter and stride sizes of the first convolutional layer were chosen to be comparable to the window and the hop sizes of short-time Fourier transformation, respectively [5][6][7][8][9][10][11].…”

“…These spectral representations have served a similar role to the word embedding in the language model in that the mid-level representation are computed separately from the learning model and they are not particularly optimized for the target task. This issue has been addressed by taking raw waveforms directly as input in different audio tasks, for example, speech recognition [5][6][7], music classification [8][9][10] and acoustic scene classification [11,12].…”

Incorporation of a spectral model in a convolutional neural network for accelerated spectral fitting

“…The CEMD architecture is graphically depicted in Figure . The encoding step in CEMD is a convolutional neural network (CNN) composed of sequential layers of convolution, pooling, and rectification . Each of these layers have parameterized weights that can collectively be referred to as W ; therefore, the CNN with weights W can be conceived of as applying a series of transformations that perform the encoding function defined by .…”

Environmental sound processing and its applications

“…Several techniques for feature extraction based on image processing methods, which handle the time‐frequency data of sound signals as images, have also been proposed . In addition, because deep learning has great potential for handling high‐dimensional data and for jointly optimizing feature extraction and statistical models in end‐to‐end approaches, techniques that directly use the waveform amplitude values of sound signals as input have also been developed .…”