Learning Bandwidth Expansion Using Perceptually-motivated Loss

Feng, Berthy; Jin, Zeyu; Su, Jiaqi; Finkelstein, Adam

doi:10.1109/icassp.2019.8682367

Cited by 14 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It introduces artifacts specific to SE (speech enhancement), and may also be misaligned. [27]: consists of MOS tests to verify quality of 3 different bandwidth expansion algorithms.…”

Section: Correlation To Mosmentioning

confidence: 99%

A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

Manocha¹,

Finkelstein²,

Zhang³

et al. 2020

Interspeech 2020

Self Cite

View full text Add to dashboard Cite

Assessment of many audio processing tasks relies on subjective evaluation which is time-consuming and expensive. Efforts have been made to create objective metrics but existing ones correlate poorly with human judgment. In this work, we construct a differentiable metric by fitting a deep neural network on a newly collected dataset of just-noticeable differences (JND), in which humans annotate whether a pair of audio clips are identical or not. By varying the type of differences, including noise, reverb, and compression artifacts, we are able to learn a metric that is well-calibrated with human judgments. Furthermore, we evaluate this metric by training a neural network, using the metric as a loss function. We find that simply replacing an existing loss with our metric yields significant improvement in denoising as measured by subjective pairwise comparison.

show abstract

“…It introduces artifacts specific to SE (speech enhancement), and may also be misaligned. [27]: consists of MOS tests to verify quality of 3 different bandwidth expansion algorithms.…”

Section: Correlation To Mosmentioning

confidence: 99%

A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

Manocha¹,

Finkelstein²,

Zhang³

et al. 2020

Interspeech 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…WaveNet [23] and its variants for BWE [24,25] use dilated convolutions to enable large receptive field while preserving the original resolution. Feng et al [6] used FFTNet [26] which resembles the classical FFT process. Ling et al [27] proposed a hierarchical RNN to utilize the waveform structures.…”

Section: Related Workmentioning

confidence: 99%

“…The first claim is not obvious because traditional bandwidth extension (BWE) research has focused on lifting narrow-band signals to 16kHz (from 4-8kHz), primarily for telephony. As far as we are aware, the only previous work that extends to as high as 44kHz (with moderate success) is that of Feng et al [6].…”

Section: Introductionmentioning

confidence: 99%

Bandwidth Extension is All You Need

Wang

Finkelstein

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Speech generation and enhancement have seen recent breakthroughs in quality thanks to deep learning. These methods typically operate at a limited sampling rate of 16-22kHz due to computational complexity and available datasets. This limitation imposes a gap between the output of such methods and that of high-fidelity (≥44kHz) real-world audio applications. This paper proposes a new bandwidth extension (BWE) method that expands 8-16kHz speech signals to 48kHz. The method is based on a feed-forward WaveNet architecture trained with a GAN-based deep feature loss. A mean-opinionscore (MOS) experiment shows significant improvement in quality over state-of-the-art BWE methods. An AB test reveals that our 16to-48kHz BWE is able to achieve fidelity that is typically indistinguishable from real high-fidelity recordings. We use our method to enhance the output of recent speech generation and denoising methods, and experiments demonstrate significant improvement in sound quality over these baselines. We propose this as a general approach to narrow the gap between generated speech and recorded speech, without the need to adapt such methods to higher sampling rates.

show abstract

“…We consider an exhaustive set of 10 different datasets for this evaluation. These datasets span over a variety of well-known speech problems; (1) Speech Synthesis (VoCo [65] and FFTnet [68]), (2) Speech Enhancement (Dereverberation [66], Noizeus [71], HiFi-GAN [67]), (3) Voice Conversion (VCC-2018 [70]), (4) Speech Source Separation (PEASS [69]), (5) Telephony Degradations [72], (6) Bandwidth Extension (BWE [73]), and (7) General Degradation's (Simulated [6]). Please refer to supplementary material for details about these datasets.…”

Section: Subjective Evaluationsmentioning

confidence: 99%

“…Name Simulated [6] FFTnet [68] BWE [73] HiFi-GAN [ Tables 1 and 2 shows correlations with MOS on all 8 datasets. Correlations for full-reference SQA methods (PESQ and CDPAM), and non-intrusive DNSMOS are also shown.…”

Section: Subjective Evaluationsmentioning

confidence: 99%

NORESQA: A Framework for Speech Quality Assessment using Non-Matching References

Manocha,

Xu,

Kumar

2021

Preprint

View full text Add to dashboard Cite

The perceptual task of speech quality assessment (SQA) is a challenging task for machines to do. Objective SQA methods that rely on the availability of the corresponding clean reference have been the primary go-to approaches for SQA. Clearly, these methods fail in real-world scenarios where the ground truth clean references are not available. In recent years, non-intrusive methods that train neural networks to predict ratings or scores have attracted much attention, but they suffer from several shortcomings such as lack of robustness, reliance on labeled data for training and so on. In this work, we propose a new direction for speech quality assessment. Inspired by human's innate ability to compare and assess the quality of speech signals even when they have non-matching contents, we propose a novel framework that predicts a subjective relative quality score for the given speech signal with respect to any provided reference without using any subjective data. We show that neural networks trained using our framework produce scores that correlate well with subjective mean opinion scores (MOS) and are also competitive to methods such as DNSMOS [1], which explicitly relies on MOS from humans for training networks. Moreover, our method also provides a natural way to embed quality-related information in neural networks, which we show is helpful for downstream tasks such as speech enhancement.Preprint. Under review.

show abstract

Learning Bandwidth Expansion Using Perceptually-motivated Loss

Cited by 14 publications

References 16 publications

A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

Bandwidth Extension is All You Need

NORESQA: A Framework for Speech Quality Assessment using Non-Matching References

Contact Info

Product

Resources

About