Utterance Verification-Based Dysarthric Speech Intelligibility Assessment Using Phonetic Posterior Features

Abstract

In the literature, the task of dysarthric speech intelligibility assessment has been approached through development of different low-level feature representations, subspace modeling, phone confidence estimation or measurement of automatic speech recognition system accuracy. This paper proposes a novel approach where the intelligibility is estimated as the percentage of correct words uttered by a speaker with dysarthria by matching and verifying utterances of the speaker with dysarthria against control speakers' utterances in phone posterior feature space and broad phonetic posterior feature space. Experimental validation of the proposed approach on the UA-Speech database, with posterior feature estimators trained on the data from auxiliary domain and language, obtained a best Pearson's correlation coefficient (r) of 0.950 and Spearman's correlation coefficient (ρ) of 0.957. Furthermore, replacing control speakers' speech with speech synthesized by a neural text-to-speech system obtained a best r of 0.937 and ρ of 0.961.

Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature Review

“…M = Male, and F= Female Dataset Repository Speech Disorders Type Instances Languages References AphasiaBank Aphasia 83 Aphasic 180 with and 140 without English, Spanish, German, Italian, Hungarian, Mandarin, Chinese [ 84 ] PC-GITA 85 - Parkinson’s disease 50 with (25 M, 25 F) and 50 without Spanish [ 26 , 55 , 86–90 ] TORGO 78 TORGO 91 Dysarthria 7 With (4 M, 3 F): 2762 utterances, 5980 from healthy speakers English [ 14 , 55 , 86 , 89 , 92–95 ] Finnish 55 PD Parkinson’s disease 35 with (14 male, 21 fe- male), and 32 without (12 male, 20 female) Finnish [ 55 ] UA Speech 79 UASpeech 96 Dysarthria: cerebral palsy 15 with (4 M, 11 F), and 13 without. 765 isolated words per speaker English [ 26 , 89 , 97–103 ] PDSTU 104 …”

“…17 speakers, 5229 words French [ 26 ] ATR 119 - Dysarthria 17 speakers, 5229 words Japanese [ 96 ] TIMIT 120 TIMIT 121 Dysarthria 630 speakers. 3310 sentences English [ 111 ] CMUArctic Festvox 122 Dysarthria 5 M, and 2 F. 1150 utterances English [ 123 ] EMA 97 - Dysarthria 3 (1 M, 2 F), 680 utterances English [ 124 ] IEMOCAP 125 - Dysarthria 1 M and 1 F. 3900 utterances English: USA [ 124 ] Parkinsons 28 Parkinsons 126 Parkinson’s disease 23 with, and 8 without English [ 127 ] [ 70 ] - Parkinson’s disease 20 with and 20 without. 1040 speech signals Turkish [ 70 ] Spanish datasets 13 <...…”

“…In Some Cases, Studies Incorporate More Than One Technique. Following That, the Same Study Was Replicated, Thereby Increasing the Overall Number of Research Studies Method/Technique % Reference MFCCs/derived features from MFCCs 35.4% [ 13 , 15 , 55 , 70 , 72 , 80 , 87 , 94 , 95 , 97–99 , 101 , 112 , 117 , 127 , 131 , 133 , 134 ] Spectro-Temporal of utterances/keywords 26.15% [ 12 , 14 , 15 , 26 , 63 , 72 , 87 , 89 , 90 , 97 , 101–103 , 110 , 112 , 124 , 136 ] Articulation way/Speech timing 18.5% [ 13 , 27 , 58 , 81 , 86–88 , 111 , 113 , 124 , 129 ] Glottal flow 7.7% [ 58 , …”

Dynamic spectral characteristics of defective sounds pronunciation in students with minimal dysarthric disorders

“…In this regard, it can be claimed that specific defects in articulation at minimal dysarthric disorders lead to the appearance of qualitatively different changes in the spectral characteristics of sounds [14][15][16]. Currently, in Russian speech therapy, the linguistic aspect of sound-pronunciation disorders in children with this type of speech dysontogenesis is practically not developed.…”

Investigation of DNN-HMM and Lattice Free Maximum Mutual Information Approaches for Impaired Speech Recognition

“…A bidirectional Deep Recurrent Neural Network (biRNN) based DNN-HMM is used for phoneme recognition [15]. In a recent work [19], the authors used a phonetic posterior feature space for matching and verifying the impaired speech with the control speakers data. Several parameters such as Linear Discriminant Analysis (LDA), context dependent states, Feature space Maximum Liklihood Linear Regression (FMLLR) are used with Teacher-Student network [20] to increase the accuracy.…”