Improving Aphasic Speech Recognition by Using Novel Semi-Supervised Learning Methods on AphasiaBank for English and Spanish

Abstract

Automatic speech recognition in patients with aphasia is a challenging task for which studies have been published in a few languages. Reasonably, the systems reported in the literature within this field show significantly lower performance than those focused on transcribing non-pathological clean speech. It is mainly due to the difficulty of recognizing a more unintelligible voice, as well as due to the scarcity of annotated aphasic data. This work is mainly focused on applying novel semi-supervised learning methods to the AphasiaBank dataset in order to deal with these two major issues, reporting improvements for the English language and providing the first benchmark for the Spanish language for which less than one hour of transcribed aphasic speech was used for training. In addition, the influence of reinforcing the training and decoding processes with out-of-domain acoustic and text data is described by using different strategies and configurations to fine-tune the hyperparameters and the final recognition systems. The interesting results obtained encourage extending this technological approach to other languages and scenarios where the scarcity of annotated data to train recognition models is a challenging reality.

Special Issue on Applications of Speech and Language Technologies in Healthcare

“…Aphasic speech is considered in two of the papers. Torre et al [7] addressed the difficulty of transcribing very unintelligible speech and the scarcity of annotated data by proposing a new semi-supervised learning method with encouraging results. On the other hand, Cistola al.…”

Learning Co-Speech Gesture for Multimodal Aphasia Type Detection

“…To identify the pathological symptoms shown in a narrative speech of PWA, researchers have focused on linguistic features (e.g., word frequency, Part-of-Speech (Le et al, 2018), word embeddings (Qin et al, 2019a)), and acoustic features (e.g., filler words, pauses, the number of phones per word (Le and Provost, 2016;Qin et al, 2019b)). With the advances in automated speech recognition (ASR) that can make the transcription of aphasic speech into text (Radford et al, 2022;Baevski et al, 2020), there have been end-toend approaches that do not require explicit feature extraction in assessing patients with aphasia (Chatzoudis et al, 2022;Torre et al, 2021). While these works reveal valuable insight into detecting aphasia, little attention has been paid to identifying the types of aphasia, which can be crucial for proper treatment procedures.…”

Digital Assessment of Acquired Alexia and Agraphia: On the Relevance of Written Discourse

“…Since a large amount of data is needed as a basis for standardization studies, automated analyses and the recognition of language peculiarities, creating an underlying database is correspondingly complex and will take time. Nevertheless, this approach to digital diagnostics is very promising and should be pursued further (e.g., Kohlschein et al, 2018;Torre et al, 2021).…”