An Efficient and Perceptually Motivated Auditory Neural Encoding and Decoding Algorithm for Spiking Neural Networks

Abstract

The auditory front-end is an integral part of a spiking neural network (SNN) when performing auditory cognitive tasks. It encodes the temporal dynamic stimulus, such as speech and audio, into an efficient, effective and reconstructable spike pattern to facilitate the subsequent processing. However, most of the auditory front-ends in current studies have not made use of recent findings in psychoacoustics and physiology concerning human listening. In this paper, we propose a neural encoding and decoding scheme that is optimized for audio processing. The neural encoding scheme, that we call Biologically plausible Auditory Encoding (BAE), emulates the functions of the perceptual components of the human auditory system, that include the cochlear filter bank, the inner hair cells, auditory masking effects from psychoacoustic models, and the spike neural encoding by the auditory nerve. We evaluate the perceptual quality of the BAE scheme using PESQ; the performance of the BAE based on sound classification and speech recognition experiments. Finally, we also built and published two spikeversion of speech datasets: the Spike-TIDIGITS and the Spike-TIMIT, for researchers to use and benchmarking of future SNN research.

Brain-Inspired Learning on Neuromorphic Substrates

“…While the van Rossum distance solves some challenges in the streaming data setting in which labels can only be assigned coarsely in time, a number of important issues remain open. Here, future work, possibly building on aggregate label losses [101], [107] or connectionist temporal classification (CTC) losses [43], provided that these approaches can be made online-capable, may offer possible solutions.…”

Automated Adaptive Threshold-Based Feature Extraction and Learning for Spiking Neural Networks

“…Each word in the TIDIGITS is repeated 450 times, where 2464 (224 × 11) and 2486 (226 × 11) isolated However, in the study of neuromorphic computing, there are other speech datasets such as the N-TIDIGITS [87] that was designed for SNN benchmarking, but it is relatively small compared with the TIDIGITS. Thus, the TIDIGITS dataset is employed as one of the main benchmarks for ASR research [88]- [90]. The RWCP dataset [86] is a non-speech natural sound dataset recorded in an anechoic room composed of sound samples with rich and diverse frequency components.…”

Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition

“…However, the scale of the ASR tasks explored in these studies is relatively small comparing to modern ASR benchmarks due to the limited availability of event-based ASR corpora. Pan et al (Pan et al, 2019) recently proposed an efficient and perceptually motivated auditory neural encoding scheme to encode the large-scale ASR corpora collected by microphone sensors into spiking events. With this encoding scheme, approximately 50% spiking events can be reduced with negligible interference to the perceptual quality of inputs audio signals.…”