Objective Intelligibility Measures Based on Mutual Information for Speech Subjected to Speech Enhancement Processing

Abstract

We propose a novel method for objective speech intelligibility prediction which can be useful in many application domains such as hearing instruments and forensics. Most objective intelligibility measures available in the literature employ some kind of signal-to-noise ratio (SNR) or a correlation-based comparison between the spectro-temporal representations of clean and processed speech. In this paper, we investigate the speech intelligibility prediction from the viewpoint of information theory and introduce novel objective intelligibility measures based on the estimated mutual information between the temporal envelopes of clean speech and processed speech in the subband domain. Mutual information allows to account for higher order statistics and hence to consider dependencies beyond the conventional second order statistics. Using data from three different listening tests it is shown that the proposed objective intelligibility measures provide promising results for speech intelligibility prediction in different scenarios of speech enhancement where speech is processed by non-linear modification strategies.Index Terms-Mutual information, objective measures, speech intelligibility prediction.

A Simple Model of Speech Communication and its Application to Intelligibility Enhancement

“…A typical scale is the ERB (equivalent rectangular bandwidth) scale, e.g., [18], [19]. It is natural, e.g., [15], to consider the auditory-domain signal to have one independent component signal per ERB. Auditory models provide a manner of deriving such component signals.…”

“…The interpretation process for speech is also noisy: speech signals that are ambiguous in their pronunciation may be interpreted in various ways. Information theoretical concepts have been used in the analysis of human hearing [14] and for the definition of measures of intelligibility [15]. These models do not have the notion of production noise, but the model of [14] considers sensory noise, which corresponds to our interpretation noise.…”

“…These models do not have the notion of production noise, but the model of [14] considers sensory noise, which corresponds to our interpretation noise. The models of [14] and [15] appear not to have been used for optimizing intelligibility.…”

An intelligibility metric based on a simple model of speech communication

“…Recently, information theory (IT) has been proposed as a new paradigm for speech intelligibility prediction [13,14,15]. This is a natural approach to take given that the fundamental goal of speech communication is to transfer information from a talker to a listener.…”

The third ‘CHiME’ speech separation and recognition challenge: Analysis and outcomes

“…The STOI measure is based on the sum of the correlation between the envelopes of the clean speech signal and the corrupted speech measured with 15 1/3-octave frequency bands starting at 150 Hz. More recently, using the same frequency bands, it has been shown that a mutual information-based measure can perform better than STOI (Taghia and Martin, 2014).…”