Intelligibility assessment of cleft lip and palate speech using Gaussian posteriograms based on joint spectro-temporal features

Abstract

Intelligibility is considered as one of the primary measures for speech rehabilitation of individuals with a cleft lip and palate (CLP). Currently, speech processing and machine-learning-based objective methods are gaining more research interest as a way to quantify speech intelligibility. In this work, joint spectro-temporal features computed from a time–frequency representation of speech are explored to derive speech representations based on Gaussian posteriograms. A comparative framework using dynamic time warping (DTW) is used to quantify the intelligibility of child CLP speech. The DTW distance is used to score sentence-level intelligibility and tested for correlation with perceptual intelligibility ratings obtained from expert speech-language pathologists. A baseline DTW system using the conventional Mel-frequency cepstral coefficients (MFCCs) is also developed to compare the performance of the proposed system. Spearman's rank correlation coefficient between the objective intelligibility scores and the perceptual intelligibility rating is studied. A Williams significance test is conducted to assess the statistical significance of the correlation difference between the methods. The results show that the system based on joint spectro-temporal features significantly outperforms the MFCC-based system.

Consonant-Vowel Transition Models Based on Deep Learning for Objective Evaluation of Articulation

“…support vector machine (SVM), Gaussian mixture model (GMM)) that map the features to clinical articulation scores. Most of these ML models [18], [19] are trained on a set of acoustic features extracted from a particular disorder. However, as we see in Fig.…”

“…As an alternative to traditional features based on signal processing, joint spectro-temporal features have been proposed to model CV transition regions in machine learning models. The two-dimensional discrete cosine transform (2D-DCT) is one of the most commonly used approaches to modeling the spectrotemporal dynamics of CV transition regions [19], [22]- [24]. Supervised learning methods using 2D-DCT features as input have been used for classification of place of articulation in stop consonants [23], to evaluate the goodness of /t/ and /k/ productions in children with speech sound disorders [25], to detect stop consonant production errors [24], and to model perceptual intelligibility ratings in children with CP [19].…”

Automation of Speech Quality Assessment in Speech Rehabilitation

“…The obvious shortcomings of the method of assessing the quality of speech restoration, based on GOST R 58040-95 [4], led to the need to develop algorithms for automatic evaluation and their implementation in the framework of automated systems for assessing speech intelligibility. At the moment, there are a number of works on automation of speech intelligibility assessment [5][6][7], but all of them appeared later than the beginning of development and the appearance of the main algorithms of the considered software complex. As part of the research on speech restoration using technical methods, such algorithms were developed and implemented.…”