Enabling Real-time On-chip Audio Super Resolution for Bone Conduction Microphones

Abstract

Voice communication using the air conduction microphone in noisy environments suffers from the degradation of speech audibility. Bone conduction microphones (BCM) are robust against ambient noises but suffer from limited effective bandwidth due to their sensing mechanism. Although existing audio super resolution algorithms can recover the high frequency loss to achieve high-fidelity audio, they require considerably more computational resources than available in low-power hearable devices. This paper proposes the first-ever real-time on-chip speech audio super resolution system for BCM. To accomplish this, we built and compared a series of lightweight audio super resolution deep learning models. Among all these models, ATS-UNet is the most cost-efficient because the proposed novel Audio Temporal Shift Module (ATSM) reduces the network's dimensionality while maintaining sufficient temporal features from speech audios. Then we quantized and deployed the ATS-UNet to low-end ARM micro-controller units for real-time embedded prototype. Evaluation results show that our system achieved real-time inference speed on Cortex-M7 and higher quality than the baseline audio super resolution method. Finally, we conducted a user study with ten experts and ten amateur listeners to evaluate our method's effectiveness to human ears. Both groups perceived a significantly higher speech quality with our method when compared to the solutions with the original BCM or air conduction microphone with cutting-edge noise reduction algorithms. CCS Concepts: • Computing methodologies → Artificial intelligence; • Computer systems organization → Realtime systems; System on a chip; Embedded software.

Training Strategies for Own Voice Reconstruction in Hearing Protection Devices using an In-ear Microphone

“…In [7] a fully convolutional neural network approach in the time-domain has been proposed to estimate clean broadband speech using two in-ear microphones. Similarly, in [8] it has been proposed to utilize a U-Net architecture to enhance bone-conducted signals in the short-time Fourier transform (STFT) domain.…”