Investigating Cross-Domain Losses for Speech Enhancement

Abstract

Recent years have seen a surge in the number of available frameworks for speech enhancement (SE) and recognition. Whether model-based or constructed via deep learning, these frameworks often rely in isolation on either time-domain signals or time-frequency (TF) representations of speech data. In this study, we investigate the advantages of each set of approaches by separately examining their impact on speech intelligibility and quality. Furthermore, we combine the fragmented benefits of time-domain and TF speech representations by introducing two new cross-domain SE frameworks. A quantitative comparative analysis against recent model-based and deep learning SE approaches is performed to illustrate the merit of the proposed frameworks.

CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement

“…Here we normalize the PESQ score to the range [0,1]. Moreover, an additional penalization in the resultant waveform L Time is proven to improve the restored speech quality [20]:…”

“…In the TF-domain, most conventional model-based or DL techniques utilize the magnitude component while ignoring the phase. This is accounted to the unstructured phase component, which imposes challenges to the utilized architectures [19], [20]. To circumvent this challenge, several approaches follow the strategy of enhancing the complex spectrogram (real and imaginary parts), which implicitly enhances both magnitude and phase [21], [22].…”