Speech-in-noise intelligibility improvement based on spectral shaping and dynamic range compression

Zorilă, Tudor-Cătălin; Kandia, Varvara; Stylianou, Yannis

doi:10.21437/interspeech.2012-197

Cited by 73 publications

(35 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To combat environmental degradation, a variety of speech enhancement algorithms have been proposed (see [2] for an overview). There are two main approaches to speech enhancement: 1) the speech signal can be modified prior to degradation (e.g., optimal energy redistribution [4] and dynamic range compression [5]), or 2) the speech signal can be modified after degradation has been introduced (e.g., Wiener filters [6]). The former type of algorithm is referred to as a pre-processing algorithm and the latter as a post-processing algorithm.…”

Section: Introductionmentioning

confidence: 99%

An Evaluation of Intrusive Instrumental Intelligibility Metrics

Kuyk

Kleijn

Hendriks

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Instrumental intelligibility metrics are commonly used as an alternative to listening tests. This paper evaluates 12 monaural intrusive intelligibility metrics: SII, HEGP, CSII, HASPI, NCM, QSTI, STOI, ESTOI, MIKNN, SIMI, SIIB, and sEPSM corr . In addition, this paper investigates the ability of intelligibility metrics to generalize to new types of distortions and analyzes why the top performing metrics have high performance. The intelligibility data were obtained from 11 listening tests described in the literature. The stimuli included Dutch, Danish, and English speech that was distorted by additive noise, reverberation, competing talkers, pre-processing enhancement, and post-processing enhancement. SIIB and HASPI had the highest performance achieving a correlation with listening test scores on average of ρ = 0.92 and ρ = 0.89, respectively. The high performance of SIIB may, in part, be the result of SIIBs developers having access to all the intelligibility data considered in the evaluation. The results show that intelligibility metrics tend to perform poorly on data sets that were not used during their development. By modifying the original implementations of SIIB and STOI, the advantage of reducing statistical dependencies between input features is demonstrated. Additionally, the paper presents a new version of SIIB called SIIB Gauss , which has similar performance to SIIB and HASPI, but takes less time to compute by two orders of magnitude.

show abstract

Section: Introductionmentioning

confidence: 99%

An Evaluation of Intrusive Instrumental Intelligibility Metrics

Kuyk

Kleijn

Hendriks

2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Moreover, we introduce ŝ, the signal example that is enhanced by reference modification algorithms such as SSDRC [3], in the D training process. The loss function is thus extended to Equation (4).…”

Section: Model Description and Training Processmentioning

confidence: 99%

“…The 810 English sentences (90 sentences × 9 SNRs) were used for testing only. To prepare the enhanced signal example ŝ introduced in Section 3.2, we selected three reference algorithms: (1) OptSII [14], a linear filter to maximize the Speech Intelligibility Index (SII), (2) OptMI [15], a linear filter to optimally redistribute energy based on mutual information criterion, and (3) SSDRC [3], a method to integrate spectral shaping and dynamic range compression. Each training sentence was randomly processed by one of these three algorithms to obtain its enhanced example.…”

Section: Model Description and Training Processmentioning

confidence: 99%

“…This task is usually termed near-end listening enhancement (NELE). There are various strategies for NELE tasks, such as spectral tilt flattening [1], formant shifting [2], and dynamic range compression [3,4]. A common idea of the above algorithms is to reallocate the speech energy in the time-frequency (T-F) domain in such a way as to boost the acoustic cues that are perceptually crucial.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

iMetricGAN: Intelligibility Enhancement for Speech-in-Noise Using Generative Adversarial Network-Based Metric Learning

Li¹,

Fu²,

Yu³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

The intelligibility of natural speech is seriously degraded when exposed to adverse noisy environments. In this work, we propose a deep learning-based speech modification method to compensate for the intelligibility loss, with the constraint that the root mean square (RMS) level and duration of the speech signal are maintained before and after modifications. Specifically, we utilize an iMetricGAN approach to optimize the speech intelligibility metrics with generative adversarial networks (GANs). Experimental results show that the proposed iMetricGAN outperforms conventional state-of-the-art algorithms in terms of objective measures, i.e., speech intelligibility in bits (SIIB) and extended short-time objective intelligibility (ESTOI), under a Cafeteria noise condition. In addition, formal listening tests reveal significant intelligibility gains when both noise and reverberation exist.

show abstract

“…In the context of Lombard speech, there have been a number of studies on improving speech intelligibility. Spectral shaping and dynamic range compression, induced statically or adaptively (by optimizing an objective measure of intelligibility), have proven highly effective under various noise conditions [19,20]. We investigate the intelligiblity-enhancing properties of neural-adapted Lombard speech by comparing it to a post-processing strategy based on an intelligibilityenhancing algorithm.…”

Section: Introductionmentioning

confidence: 99%

Whispered and Lombard Neural Speech Synthesis

Bleisch

Petkov

et al. 2021

Preprint

View full text Add to dashboard Cite

It is desirable for a text-to-speech system to take into account the environment where synthetic speech is presented, and provide appropriate context-dependent output to the user. In this paper, we present and compare various approaches for generating different speaking styles, namely, normal, Lombard, and whisper speech, using only limited data. The following systems are proposed and assessed: 1) Pre-training and fine-tuning a model for each style. 2) Lombard and whisper speech conversion through a signal processing based approach. 3) Multi-style generation using a single model based on a speaker verification model. Our mean opinion score and AB preference listening tests show that 1) we can generate high quality speech through the pre-training/fine-tuning approach for all speaking styles. 2) Although our speaker verification (SV) model is not explicitly trained to discriminate different speaking styles, and no Lombard and whisper voice is used for pretrain this system, SV model can be used as style encoder for generating different style embeddings as input for Tacotron system. We also show that the resulting synthetic Lombard speech has a significant positive impact on intelligibility gain.

show abstract

Speech-in-noise intelligibility improvement based on spectral shaping and dynamic range compression

Cited by 73 publications

References 8 publications

An Evaluation of Intrusive Instrumental Intelligibility Metrics

An Evaluation of Intrusive Instrumental Intelligibility Metrics

iMetricGAN: Intelligibility Enhancement for Speech-in-Noise Using Generative Adversarial Network-Based Metric Learning

Whispered and Lombard Neural Speech Synthesis

Contact Info

Product

Resources

About