Deepfilternet: A Low Complexity Speech Enhancement Framework for Full-Band Audio Based On Deep Filtering

Schröter, Hendrik; Escalante-B., Alberto N.; Rosenkranz, Tobias; Maier, Andreas

doi:10.1109/icassp43922.2022.9747055

Cited by 47 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For full-band speech, the number of the input features increases markedly when features are extracted on a linear frequency scale, resulting in much greater computational complexity. The number of input features can be reduced by extracting them on a logarithmic frequency scale, or by using ERB-based or Bark-based filter banks to extract input features (Schröter et al, 2022).…”

Section: Deep Learning Methodsmentioning

confidence: 99%

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Zheng,

Zhang,

Liu

et al. 2023

Trends in Hearing

View full text Add to dashboard Cite

Frequency-domain monaural speech enhancement has been extensively studied for over 60 years, and a great number of methods have been proposed and applied to many devices. In the last decade, monaural speech enhancement has made tremendous progress with the advent and development of deep learning, and performance using such methods has been greatly improved relative to traditional methods. This survey paper first provides a comprehensive overview of traditional and deep-learning methods for monaural speech enhancement in the frequency domain. The fundamental assumptions of each approach are then summarized and analyzed to clarify their limitations and advantages. A comprehensive evaluation of some typical methods was conducted using the WSJ + Deep Noise Suppression (DNS) challenge and Voice Bank + DEMAND datasets to give an intuitive and unified comparison. The benefits of monaural speech enhancement methods using objective metrics relevant for normal-hearing and hearing-impaired listeners were evaluated. The objective test results showed that compression of the input features was important for simulated normal-hearing listeners but not for simulated hearing-impaired listeners. Potential future research and development topics in monaural speech enhancement are suggested.

show abstract

Section: Deep Learning Methodsmentioning

confidence: 99%

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Zheng,

Zhang,

Liu

et al. 2023

Trends in Hearing

View full text Add to dashboard Cite

show abstract

“…where For efficient DNN operation, BRnet is based on DeepFilterNet [27], which performs a two-stage denoising process with reference to human auditory properties. The BRnet, which consists of three modules, the Encoder, the ERB Decoder and the Deep Filter (DF) Decoder, is shown in Fig.…”

Section: A the Score Modulementioning

confidence: 99%

“…where q is the look-ahead of the deep filter. In contrast to [27], the proposed BRnet operates at 16 kHz and uses an ERB filterbank with 32 bands. The deep filter with N = 5 is applied only to the lowest 160 frequency bins which covers a 5 kHz bandwidth.…”

Section: A the Score Modulementioning

confidence: 99%

“…As a core element of ACIS-BAT, an array configuration-independent Spatial COherence REpresentation (SCORE) module is developed to extract spatial features robust to array configuration variations. With the extracted spatial features, we propose a Binaural Rendering network (BRnet) based on a DeepfFilterNet [27] to transform the microphone signals into binaural signals. The scalability between signal enhancement and ambience preservation is achieved through a tunable parameter in BRnet injected into a Feature-wise Linear Modulation (FiLM) layer [28].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Model-Matching Principle Applied to the Design of an Array-Based All-Neural Binaural Rendering System for Audio Telepresence

Hsu

Bai

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Audio Telepresence (AT) aims to create an immersive experience of the audio scene at the far end for the user(s) at the near end. The application of AT could encompass scenarios with varying degrees of emphasis on signal enhancement and ambience preservation. It is desirable for an AT system to be scalable between these two extremes. To this end, we propose an array-based Binaural AT (BAT) system using the DeepFilterNet as the backbone to convert the array microphone signals into the Head-Related Transfer Function (HRTF)-filtered signals, with a tunable weighting between signal enhancement and ambience preservation. An array configuration-independent Spatial COherence REpresentation (SCORE) feature is proposed for the model training so that the network remains robust to different array geometries and sensor counts. magnitude-weighted Interaural Phase Difference error (mw-IPDe), magnitude-weighted Interaural Level Difference error (mw-ILDe), and modified Scale-Invariant Signal-to-Distortion Ratio (mSI-SDR) are defined as performance metrics for objective evaluation. Subjective listening tests were also performed to validate the proposed BAT system. The results have shown that the proposed BAT system can achieve superior telepresence performance with the desired balance between signal enhancement and ambience preservation, even when the array configurations are unseen in the training phase.

show abstract

“…These filter banks retain the frequency domain information that is more important to human perception, effectively reducing the dimensionality of the input features and thus the complexity of the neural network model. DeepFilterNet [8], based on PercepNet, first enhances the spectral envelope using ERB-scaled gains and further enhances the periodic part of the preliminarily enhanced spectrum using a DeepFilter [9]. However, compact feature-based work typically uses expert hand-designed filters to derive compact real-valued features, which do not make use of phase information.…”

Section: Introductionmentioning

confidence: 99%

Multi‐stage attention network for monaural speech enhancement

Wang

Liu

et al. 2022

IET Signal Processing

View full text Add to dashboard Cite

In this paper, we propose a two-stage heterogeneous lightweight network for monaural speech enhancement. Specifically, we design a novel two-stage framework consisting of a coarse-grained full-band mask estimation stage and a fine-grained low-frequency refinement stage. Instead of using a hand-designed real-valued filter, we use a novel learnable complex-valued rectangular bandwidth (LCRB) filter bank as an extractor of compact features. Furthermore, considering the respective characteristics of the proposed two-stage task, we used a heterogeneous structure, i.e., a U-shaped subnetwork as the backbone of CoarseNet and a single-scale subnetwork as the backbone of FineNet. We conducted experiments on the VoiceBank + DEMAND and DNS datasets to evaluate the proposed approach. The experimental results show that the proposed method outperforms the current state-of-the-art methods, while maintaining relatively small model size and low computational complexity.

show abstract

Deepfilternet: A Low Complexity Speech Enhancement Framework for Full-Band Audio Based On Deep Filtering

Cited by 47 publications

References 18 publications

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Model-Matching Principle Applied to the Design of an Array-Based All-Neural Binaural Rendering System for Audio Telepresence

Multi‐stage attention network for monaural speech enhancement

Contact Info

Product

Resources

About