A convolutional neural-network model of human cochlear mechanics and filter tuning for real-time applications

Abstract

Auditory models are commonly used as feature extractors for automatic speech-recognition systems or as front-ends for robotics, machine-hearing and hearing-aid applications. Although auditory models can capture the biophysical and nonlinear properties of human hearing in great detail, these biophysical models are computationally expensive and cannot be used in real-time applications. We present a hybrid approach where convolutional neural networks are combined with computational neuroscience to yield a real-time end-to-end model for human cochlear mechanics, including level-dependent filter tuning (CoNNear). The CoNNear model was trained on acoustic speech material and its performance and applicability were evaluated using (unseen) sound stimuli commonly employed in cochlear mechanics research. The CoNNear model accurately simulates human cochlear frequency selectivity and its dependence on sound intensity, an essential quality for robust speech intelligibility at negative speech-to-background-noise ratios. The CoNNear architecture is based on parallel and differentiable computations and has the power to achieve real-time human performance. These unique CoNNear features will enable the next generation of human-like machine-hearing applications.

A convolutional neural-network framework for modelling auditory sensory cells and synapses

“…After determining the final CNN model architectures, we compute their runtime benefit over analytical models and investigate the extent to which our methodology is applicable to different existing mechanistic descriptions of the IHC-ANF complex. Lastly, we provide two use cases: one in which IHC-ANF models are connected to a CNN-based cochlear mechanics model (CoNNear cochlea 66 ) to capture the full transformation of acoustic stimuli into IHC receptor potentials and ANF firing rates along the cochlear tonotopy and hearing range, and a second one where we illustrate how backpropagation can be used to modify the CNN model input to restore a pathological output.…”

A convolutional neural-network framework for modelling auditory sensory cells and synapses

“…Three modules that correspond to different stages of the reference analytical auditory periphery model 61 were considered: cochlear processing, IHC transduction and ANF firing. The calibration of the cochlear mechanics module (CoNNear cochlea ) is described elsewhere 66,67 , here we focus on developing the sensory neuron models (i.e., CoNNear IHC and CoNNear ANF ). Fig.…”

“…The number of filters in each layer relate to the non-linear level-and frequency-dependent characteristics of the analytical model and can be selected based on the complexity of the description (e.g., number of ODEs). Since 128 filters per layer were sufficient to describe the properties of the transmission-line cochlear model 66 , a highly complex and non-linear system, this number should prove a good starting point for approximating different non-linear systems. By examining whether the properties of the model are faithfully captured by the trained CNN architecture across a broad range of stimulus levels and frequencies, the filter size can be further optimised.…”

“…CoNNear model of the auditory periphery. Acoustic stimuli can be transformed to IHC receptor potentials and ANF firing rates along the cochlear tonotopy and hearing range, after connecting the CoNNear cochlea 66 , IHC and ANF modules together.…”

“…After determining the final CNN model architectures, we compute their run-time benefit over analytical models and investigate the extent to which our methodology is applicable to different existing mechanistic descriptions of the IHC-ANF complex. Lastly, we provide two use cases: one in which IHC-ANF models are connected to a CNN-based cochlear mechanics model (CoNNear cochlea 66 ) to capture the full transformation of acoustic stimuli into IHC receptor potentials and ANF firing rates along the cochlear tonotopy and hearing range, and a second one where we illustrate how backpropagation can be used to modify the CNN model input to restore a pathological output.…”