A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech

Valin, Jean‐Marc; Isik, Umut; Phansalkar, Neerad; Giri, Ritwik; Helwani, Karim; Krishnaswamy, Arvindh

doi:10.48550/arxiv.2008.04259

Cited by 6 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While their tendency to impart an unnatural frequency response to the speech of the wearer makes them unattractive as a direct source of voice recording, their isolated perspective provides incredibly potent auxiliary data points. A combination of recent work in audio de-noising, such as RNNoise [40] and PercepNet [41], with the bone conduction methods presented in this paper may enable a new generation of highly effective noise removal with minimal power requirements.…”

Section: Discussionmentioning

confidence: 99%

In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms

Schilk

Polvani

Ronco

et al. 2023

Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms

Schilk

Polvani

Ronco

et al. 2023

Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation

View full text Add to dashboard Cite

“…Higher values indicate better performance. First, we conduct extensive experiments on VoiceBank + DEMAND to compare the proposed methods with several state-of-the-art (SOTA) full-band and super-wideband SE methods, including RNNoise [3], PercepNet [4], DC-CRN [7] (super-wideband version), DeepFilterNet [5] and S-DCCRN [6]. We also conduct ablation study to show the importance of phase recovery for the low-frequency band (i.e., w/o SR-Net), which removes the complex spectral refinement in LF-Net.…”

Section: B Implementation Setupmentioning

confidence: 99%

“…In [3], Bark-scale spectrum with 22dimensional Bark-frequency cepstral coefficients (BFCC) was adopted as input features and 22 ideal critical band gains were mapped, which can reduce the model size and computational complexity simultaneously. More recently, based on the human perception of speech signals, PercepNet developed a perceptual band representation with 32 triangular spectral bands [4], spaced according to the human hearing equivalent rectangular bandwidth (ERB). Obviously, the frequency resolution of the spectrum in Bark scale and that in ERB scale are much lower than the Fourier spectrum, leading to the leakage of information among frequency bands.…”

Section: Introductionmentioning

confidence: 99%

DMF-Net: A decoupling-style multi-band fusion model for full-band speech enhancement

Yu¹,

Guan²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

Full-band speech enhancement based on deep neural networks is still challenging for the difficulty of modeling more frequency bands and real-time implementation. Previous studies usually adopt compressed full-band speech features in Bark and ERB scale with relatively low frequency resolution, leading to degraded performance, especially in the high-frequency region. In this paper, we propose a decoupling-style multi-band fusion model to perform full-band speech denoising and dereverberation. Instead of optimizing the full-band speech by a single network structure, we decompose the full-band target into multi sub bands and then employ a multi-stage chain optimization strategy to estimate clean spectrum stage by stage. Specifically, the low-(0-8 kHz), middle-(8-16 kHz), and high-frequency (16-24 kHz) regions are mapped by three separate sub-networks and are then fused to obtain the full-band clean target STFT spectrum. Comprehensive experiments on two public datasets demonstrate that the proposed method outperforms previous advanced systems and yields promising performance in terms of speech quality and intelligibility in real complex scenarios.

show abstract

“…In recent years, learning-based approaches have shown promising results [4,5,6]. The Deep Noise Suppression (DNS) Challenge organized at INTERSPEECH 2020 showed promising results, while also indicating that we are still about 1.4 Differential Mean Opinion Score (DMOS) from the ideal Mean Opinion Score (MOS) of 5 when tested on the DNS Challenge test set [7,8]. The DNS Challenge is the first contest that we are aware of that used subjective evaluation to benchmark SE methods using a realistic noisy test set [9].…”

Section: Introductionmentioning

confidence: 99%

ICASSP 2021 Deep Noise Suppression Challenge

Reddy

Dubey

Gopal

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The Deep Noise Suppression (DNS) challenge is designed to foster innovation in the area of noise suppression to achieve superior perceptual speech quality. We recently organized a DNS challenge special session at INTERSPEECH 2020 where we open-sourced training and test datasets for researchers to train their noise suppression models. We also open-sourced a subjective evaluation framework and used the tool to evaluate and select the final winners. Many researchers from academia and industry made significant contributions to push the field forward. We also learned that as a research community, we still have a long way to go in achieving excellent speech quality in challenging noisy real-time conditions. In this challenge, we expanded both our training and test datasets. Clean speech in the training set has increased by 200% with the addition of singing voice, emotion data, and non-English languages. The test set has increased by 100% with the addition of singing, emotional, non-English (tonal and non-tonal) languages, and, personalized DNS test clips. There are two tracks with focus on (i) real-time denoising, and (ii) real-time personalized DNS. We present the challenge results at the end.

show abstract

A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speech

Cited by 6 publications

References 0 publications

In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms

In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms

DMF-Net: A decoupling-style multi-band fusion model for full-band speech enhancement

ICASSP 2021 Deep Noise Suppression Challenge

Contact Info

Product

Resources

About