Modelling Errors in Automatic Speech Recognition for Dysarthric Speakers

Abstract

Dysarthria is a motor speech disorder characterized by weakness, paralysis, or poor coordination of the muscles responsible for speech. Although automatic speech recognition (ASR) systems have been developed for disordered speech, factors such as low intelligibility and limited phonemic repertoire decrease speech recognition accuracy, making conventional speaker adaptation algorithms perform poorly on dysarthric speakers. In this work, rather than adapting the acoustic models, we model the errors made by the speaker and attempt to correct them. For this task, two techniques have been developed: (1) a set of "metamodels" that incorporate a model of the speaker's phonetic confusion matrix into the ASR process; (2) a cascade of weighted finite-state transducers at the confusion matrix, word, and language levels. Both techniques attempt to correct the errors made at the phonetic level and make use of a language model to find the best estimate of the correct word sequence. Our experiments show that both techniques outperform standard adaptation techniques.

Improvement Techniques for Automatic Speech Recognition

“…Thus, the objective of the statistical modelling of the phoneme confusion-matrix is to estimate W from P * . This can be accomplished by the following expression [22]:…”

“…Instead, a Hidden Markov model (HMM) can be constructed for each of the phonemes in the phoneme inventory. These HMMs, termed as metamodels [22,24], can be best understood by comparison with a "standard" acoustic HMM: a standard acoustic HMM estimates Pr(O ′ |p j ), where O ′ is a subsequence of the complete sequence of observed acoustic vectors in the utterance, O, and p j is a postulated phoneme in P. A metamodel estimates Pr(P ′ |p j ), whereP ′ is a subsequence of the complete sequence of observed (decoded) phonemes in the utteranceP.…”

“…The architecture of the metamodel of a phoneme is shown in Figure 7 [22,24]. Each state of a metamodel has a discrete probability distribution over the symbols for the set of phonemes, plus an additional symbol labelled DELETION.…”

“…In the work presented by [22], rather than adapting the acoustic models, the errors made by the speaker are modelled to attempt to correct them. For this, the metamodels and WFSTs techniques are applied to correct the errors made at the phonetic level and make use of a language model to find the best estimate of the correct word sequence.…”

Estimation of Phoneme-Specific HMM Topologies for the Automatic Recognition of Dysarthric Speech

“…Also, words with long voiceless stops can be interpreted as two words because of the long silent occlusion phase in the middle of the target word ( before → be for ) [17, 18]. …”