Speech Intelligibility Prediction Based on Mutual Information

Abstract

This paper deals with the problem of predicting the average intelligibility of noisy and potentially processed speech signals, as observed by a group of normal hearing listeners. We propose a model which performs this prediction based on the hypothesis that intelligibility is monotonically related to the mutual information between critical-band amplitude envelopes of the clean signal and the corresponding noisy/processed signal. The resulting intelligibility predictor turns out to be a simple function of the mean-square error (mse) that arises when estimating a clean critical-band amplitude using a minimum mean-square error (mmse) estimator based on the noisy/processed amplitude. The proposed model predicts that speech intelligibility cannot be improved by any processing of noisy critical-band amplitudes. Furthermore, the proposed intelligibility predictor performs well () in predicting the intelligibility of speech signals contaminated by additive noise and potentially non-linearly processed using time-frequency weighting.

An intelligibility metric based on a simple model of speech communication

“…Recently, information theory (IT) has been proposed as a new paradigm for speech intelligibility prediction [13,14,15]. This is a natural approach to take given that the fundamental goal of speech communication is to transfer information from a talker to a listener.…”

“…First, several studies have suggested that IT could provide a unified point of view. In [16] and [17] it was observed that the AI resembles the Shannon capacity of a memoryless Gaussian channel, and in [14] it was shown that STOI is related to the average amount of information shared between the temporal envelopes of clean and distorted speech signals. Second, IT provides a powerful theoretical framework.…”

“…In the literature, two IT-based intelligibility metrics have been proposed [14,15]. Both metrics are based on the hypothesis that intelligibility is monotonically related to the mutual information of the sub-band temporal envelope of a transmitted speech signal and the corresponding received speech signal.…”

“…Both metrics are based on the hypothesis that intelligibility is monotonically related to the mutual information of the sub-band temporal envelope of a transmitted speech signal and the corresponding received speech signal. The main difference between the metrics is that [14] uses a lower bound on the information rate, whereas [15] measures mutual information using non-parametric techniques. Additionally, [14] estimates statistics over short-time segments, whereas [15] uses complete utterances.…”

An image-inspired audio sharpness index

“…Several recent measures estimate in each frequency band the correlation (e.g. Short-Time Objective Intelligibility, STOI [4]) or the mutual information (SIMI [5]) between the clean and the distorted signals.…”