Features Based on Auditory Physiology and Perception

Stern, Richard M.; Morgan, Nelson

doi:10.1002/9781118392683.ch8

Cited by 14 publications

(6 citation statements)

References 114 publications

(105 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, C1 may represent early auditory processing with subband filters (as shown in Figure 1) and half-wave rectified nonlinearity (i.e., ReLU). The feature representation steps involved in this ordering resemble the simplified form of auditory processing in the human ear [6], [12].…”

Section: Analysis Of First Layer C1mentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Deep Auditory Model Using Stack of Convolutional RBMs for Speech Recognition

Sailor¹,

Patil²

2016

Interspeech 2016

View full text Add to dashboard Cite

Recently, we have proposed an unsupervised filterbank learning model based on Convolutional RBM (ConvRBM). This model is able to learn auditory-like subband filters using speech signals as an input. In this paper, we propose two-layer Unsupervised Deep Auditory Model (UDAM) by stacking two ConvRBMs. The first layer ConvRBM learns filterbank from speech signals and hence, it represents early auditory processing. The hidden units' responses of the first layer are pooled as shorttime spectral representation to train another ConvRBM using greedy layer-wise method. The ConvRBM in second layer trained on spectral representation learns Temporal Receptive Field (TRF) which represent temporal properties of the auditory cortex in human brain. To show the effectiveness of the proposed UDAM, speech recognition experiments were conducted on TIMIT and AURORA 4 databases. We have shown that features extracted from second layer when added to filterbank features of first layer performs better than first layer features alone (and their delta features as well). For both databases, our proposed two-layer deep auditory features improve speech recognition performance over Mel filterbank features. Further improvements can be achieved by system-level combination of both UDAM features and Mel filterbank features.

show abstract

Section: Analysis Of First Layer C1mentioning

confidence: 99%

“…We have proposed single layer unsupervised learning model Convolutional Restricted Boltzmann Machine (ConvRBM) to learn filterbanks directly from the speech signals. The computational auditory models for early auditory and auditory cortex are discussed in [12].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Deep Auditory Model Using Stack of Convolutional RBMs for Speech Recognition

Sailor¹,

Patil²

2016

Interspeech 2016

View full text Add to dashboard Cite

show abstract

“…Recent studies of speech enhancement have focused on improving the acoustic waveform and testing the improvement using neural-network-based models for the auditory cortex (Fu et al, 2017; K. Tan & Wang, 2018); however, this method overlooks the role of the physiological nonlinearities in the auditory periphery (Drakopoulos et al, 2022; Zaar & Carney, 2022). Speech recognition systems that include a physiologically realistic model of the auditory periphery have been shown to be more robust in noisy environments than the model that does not include the physiological properties of the auditory system (Stern et al, 2012; Stern & Morgan, 2012).…”

Section: Introductionmentioning

confidence: 99%

Subcortical Auditory Model including Efferent Dynamic Gain Control with Inputs from Cochlear Nucleus and Inferior Colliculus

Farhadi

Jennings

Strickland

et al. 2022

Preprint

View full text Add to dashboard Cite

The focus of most existing auditory models is on the afferent system. The auditory efferent system contains descending projections from several levels of the auditory pathway, from the auditory cortex to the brainstem, that control gain in the cochlea. We developed a model with a time-varying, gain-control signal from the efferent system that includes sub-cortical ascending and descending neural pathways. The medial olivocochlear (MOC) efferent stage of the model receives excitatory projections from both fluctuation-sensitive neurons in the inferior colliculus (IC) and wide-dynamic-range neurons in the cochlear nucleus (CN). The response of the model MOC stage controlled cochlear gain dynamically. Changes in the rates of IC neurons in awake rabbit to long-duration amplitude-modulated (AM) noise were employed to adjust the parameters of the proposed model. In response to AM stimuli, physiological response rates of most IC neurons with band-enhanced (BE) modulation transfer functions (MTFs) increased over a time course consistent with the dynamics of the MOC efferent feedback. The time constant of the MOC model that best matched the IC physiology was compared to available descriptions of the MOC. Responses of the proposed subcortical model to AM noise simulate the trend of increasing rate over time, while the model without the efferent system did not show this trend.

show abstract

“…Mel filterbank is the state-of-the-art auditorybased features for the ESC task. Such handcrafted features rely on the simplified auditory models [15]. There are many approaches that are based on data-driven learning and/or optimization of parameters of auditory models.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Filterbank Learning Using Convolutional Restricted Boltzmann Machine for Environmental Sound Classification

2017

View full text Add to dashboard Cite

In this paper, we propose to use Convolutional Restricted Boltzmann Machine (ConvRBM) to learn filterbank from the raw audio signals. ConvRBM is a generative model trained in an unsupervised way to model the audio signals of arbitrary lengths. ConvRBM is trained using annealed dropout technique and parameters are optimized using Adam optimization. The subband filters of ConvRBM learned from the ESC-50 database resemble Fourier basis in the mid-frequency range while some of the low-frequency subband filters resemble Gammatone basis. The auditory-like filterbank scale is nonlinear w.r.t. the center frequencies of the subband filters and follows the standard auditory scales. We have used our proposed model as a front-end for the Environmental Sound Classification (ESC) task with supervised Convolutional Neural Network (CNN) as a back-end. Using CNN classifier, the ConvRBM filterbank (ConvRBM-BANK) and its score-level fusion with the Mel filterbank energies (FBEs) gave an absolute improvement of 10.65 %, and 18.70 % in the classification accuracy, respectively, over FBEs alone on the ESC-50 database. This shows that the proposed ConvRBM filterbank also contains highly complementary information over the Mel filterbank, which is helpful in the ESC task.

show abstract

Features Based on Auditory Physiology and Perception

Cited by 14 publications

References 114 publications

Unsupervised Deep Auditory Model Using Stack of Convolutional RBMs for Speech Recognition

Unsupervised Deep Auditory Model Using Stack of Convolutional RBMs for Speech Recognition

Subcortical Auditory Model including Efferent Dynamic Gain Control with Inputs from Cochlear Nucleus and Inferior Colliculus

Unsupervised Filterbank Learning Using Convolutional Restricted Boltzmann Machine for Environmental Sound Classification

Contact Info

Product

Resources

About