Predicting speech intelligibility based on a correlation metric in the envelope power spectrum domain

Abstract

A speech intelligibility prediction model is proposed that combines the auditory processing front end of the multi-resolution speech-based envelope power spectrum model [mr-sEPSM; Jørgensen, Ewert, and Dau (2013). J. Acoust. Soc. Am. 134(1), 436–446] with a correlation back end inspired by the short-time objective intelligibility measure [STOI; Taal, Hendriks, Heusdens, and Jensen (2011). IEEE Trans. Audio Speech Lang. Process. 19(7), 2125–2136]. This “hybrid” model, named sEPSMcorr, is shown to account for the effects of stationary and fluctuating additive interferers as well as for the effects of non-linear distortions, such as spectral subtraction, phase jitter, and ideal time frequency segregation (ITFS). The model shows a broader predictive range than both the original mr-sEPSM (which fails in the phase-jitter and ITFS conditions) and STOI (which fails to predict the influence of fluctuating interferers), albeit with lower accuracy than the source models in some individual conditions. Similar to other models that employ a short-term correlation-based back end, including STOI, the proposed model fails to account for the effects of room reverberation on speech intelligibility. Overall, the model might be valuable for evaluating the effects of a large range of interferers and distortions on speech intelligibility, including consequences of hearing impairment and hearing-instrument signal processing.

Predicting the effects of periodicity on the intelligibility of masked speech: An evaluation of different modelling approaches and their limitations

“…A more detailed description, including all equations and the theoretical motivation for each step, is provided in Sec. II in Relaño-Iborra et al (2016).…”

“…Modulation-based models, on the other hand, date back to the speech transmission index (STI; Steeneken and Houtgast, 1980) and also include the more recent modulation filterbank models, for example, the multi-resolution speech-based envelope power spectrum model (mr-sEPSM; Jørgensen et al, 2013). In addition, it is useful to distinguish between models whose decision stage (back end) evaluates energetic differences between the input signals (e.g., ESII and mr-sEPSM) and those that are based on signal correlations, such as the short-time objective intelligibility measure (STOI; Taal et al, 2011) and the correlation-based version of the mr-sEPSM (sEPSM corr ; Relaño-Iborra et al, 2016).…”

“…Four existing speech intelligibility models with different theoretical assumptions were used to predict these behavioural data: ESII (Rhebergen et al, 2006), STOI (Taal et al, 2011), mr-sEPSM (Jørgensen et al, 2013), and sEPSM corr (Relaño-Iborra et al, 2016). Briefly summarised, the ESII compares the energy of the envelopes of speech and the forward-masking corrected noise envelopes in each auditory filter using temporal windows whose durations decrease with increasing filter centre frequencies.…”

An Evaluation of Intrusive Instrumental Intelligibility Metrics

“…The speech-based envelope power spectrum model forms the basis of three intelligibility metrics: sEPSM [24], mr-sEPSM [25], and sEPSM corr [26]. All of the sEPSM metrics use the Hilbert transform and a gammatone filterbank to extract temporal envelopes for different frequency bands.…”

An Instrumental Intelligibility Metric Based on Information Theory

“…Intrusive intelligibility metrics require knowledge of the clean speech and either the communication channel or degraded speech, whereas non-intrusive intelligibility metrics require only the degraded speech. In this paper we develop a new intrusive intelligibility metric based on information theory [2].Existing intrusive intelligibility metrics include the speech intelligibility index (SII) [3], the speech transmission index (STI) [4], the coherence SII (CSII) [5], the extended SII (ESII) [6], the normalized covariance measure (NCM) [7], [8], the hearing-aid speech perception index (HASPI) [9], the shorttime objective intelligibility measure (STOI) [10], the extended STOI (ESTOI) [11], the speech-based envelope power spectrum model (sEPSM) [12]- [14], and the glimpse proportion metric (GP) [15]- [17]. As a group, the above algorithms have been successful at predicting speech intelligibility in a wide-range of conditions including additive noise, filtering, reverberation, and non-linear enhancement.…”