Improving Detection of Alzheimer’s Disease Using Automatic Speech Recognition to Identify High-Quality Segments for More Robust Feature Extraction

Abstract

Speech and language based automatic dementia detection is of interest due to it being non-invasive, low-cost and potentially able to aid diagnosis accuracy. The collected data are mostly audio recordings of spoken language and these can be used directly for acoustic-based analysis. To extract linguistic-based information, an automatic speech recognition (ASR) system is used to generate transcriptions. However, the extraction of reliable acoustic features is difficult when the acoustic quality of the data is poor as is the case with DementiaBank, the largest opensource dataset for Alzheimer's Disease classification. In this paper, we explore how to improve the robustness of the acoustic feature extraction by using time alignment information and confidence scores from the ASR system to identify audio segments of good quality. In addition, we design rhythminspired features and combine them with acoustic features. By classifying the combined features with a bidirectional-LSTM attention network, the F-measure improves from 62.15% to 70.75% when only the high-quality segments are used. Finally, we apply the same approach to our previously proposed hierarchical-based network using linguistic-based features and show improvement from 74.37% to 77.25%. By combining the acoustic and linguistic systems, a state-of-the-art 78.34% F-measure is achieved on the DementiaBank task.

Using the Outputs of Different Automatic Speech Recognition Paradigms for Acoustic- and BERT-Based Alzheimer’s Dementia Detection Through Spontaneous Speech

“…(2). Using mul-tiple ASR hypotheses and confidence scores as an input to the BERT system improves the performance compared to using just the best hypothesis as in previously proposed approached [12]. (3).…”

“…Disfluencies and unclear pronunciation can decrease the performance of ASR systems, whilst at the same time be beneficial if the related information is used to inform the classification [5,12]. In [5], the pause and disfluency annotation was used for punctuation generation providing important linguistic information in addition to the manual transcripts.…”

“…To overcome some of these challenges two types of ASR paradigms are explored: i) the traditional ASR systems, based on a pipeline comprising of acoustic, language and lexical models, and ii) end-to-end systems, which directly map a sequence of input acoustic waveforms into a sequence of graphemes or words with an integral structure. For the traditional ASR system, time delay neural network (TDNN) has been used in our previous research and got an outstanding result on dementia-related speech recognition [12][13][14].…”

“…An ASR lattice provides time alignment, recognised words and confidence scores for different hypotheses. Usually, only the most likely hypothesis is selected for the subsequent AD detection [4,12]. Here, we explore both the use of multiple hypotheses (can be seen as a form of data augmentation) as well as using the hypothesis with the highest word confidence scores (High-ConfHyp).…”

Alzheimer’s Disease Detection from Spontaneous Speech Through Combining Linguistic Complexity and (Dis)Fluency Features with Pretrained Language Models

“…(3) Filled pauses -Next to the number and total duration of silent pauses, we derived frequency counts per sentence for two filled pause type, uh and um, that had been shown to discriminate between AD patients and controls in previous studies [9]. (4) Pronunciation -As the known symptoms of AD patients include mispronunciation [14], we calculated average word level confidence scores as a proxy of pronunciation quality, which have been employed for the speech pattern detection in the context of detection of Alzheimer's Disease [15]. All measures were calculated at utterance level.…”