Multi-Stage DNN Training for Automatic Recognition of Dysarthric Speech

Abstract

Incorporating automatic speech recognition (ASR) in individualized speech training applications is becoming more viable thanks to the improved generalization capabilities of neural network-based acoustic models. The main problem in developing applications for dysarthric speech is the relative in-domain data scarcity. Collecting representative amounts of dysarthric speech data is difficult due to rigorous ethical and medical permission requirements, problems in accessing patients who are generally vulnerable and often subject to altering health conditions and, last but not least, the high variability in speech resulting from different pathological conditions. Developing such applications is even more challenging for languages which in general have fewer resources, fewer speakers and, consequently, also fewer patients than English, as in the case of a mid-sized language like Dutch. In this paper, we investigate a multi-stage deep neural network (DNN) training scheme aimed at obtaining better modeling of dysarthric speech by using only a small amount of in-domain training data. The results show that the system employing the proposed training scheme considerably improves the recognition of Dutch dysarthric speech compared to a baseline system with single-stage training only on a large amount of normal speech or a small amount of in-domain data.

Articulatory Features for ASR of Pathological Speech

“…Containing more speakers with more diverse etiologies, performing ASR on this corpus is found to more challenging compared to the CHASING01 dysarthric speech database (c.f. the ASR results in [18] and [21]).…”

Articulatory and bottleneck features for speaker-independent ASR of dysarthric speech

“…In this section, we present the results of the ASR experiments performed using the acoustic models in Section 4.1 trained on several speaker-independent features such as gammatone filterbanks, articulatory features and bottleneck features. Later, we apply the model adaptation approach we described in [29] to explore the further gains that can be obtained using a small amount of training dysarthric data. The baseline systems use DNN, CNN and TFCNN models trained on mel filterbank features and FCNN models trained using the concatenation of mel filterbank features and synthetic AFs.…”

“…In the scope of the CHASING Project, we have been developing a serious game employing ASR to provide additional speech therapy to dysarthric patients [14,27]. Earlier research in this project is reported in [26] and [29] in which we focus on investigating the available resources by using speech data from different varieties of the Dutch language and investigate model adaptation to tune an acoustic model using a small amount of dysarthric speech for training purposes respectively.…”

“…The XRMB data contained 46 speakers, out of which 36 were used for training and 5 each for cross validation and testing. After obtaining the acoustic models and bottleneck systems, we optionally apply model adaptation as described in [29] in explore the performance gains that can be obtained when using different bottleneck feature strategies. The idea behind such adaptation is similar to [61,62].…”

“…This work was conducted in the framework of the CHASING project 1 , in which a serious game employing ASR is being developed to provide additional speech therapy to dysarthric patients [27]. Moreover, we created a 6-hour Dutch dysarthric speech database that had been collected in a previous project (EST) [28] for training purposes and investigate the impact of multi-stage DNN training (henceforth referred to as model adaptation) for pathological speech [29].…”