Alzheimer Disease Recognition Using Speech-Based Embeddings From Pre-Trained Models

Gauder, Lara; Pepino, Leonardo; Ferrer, Luciana; Riera, Pablo

doi:10.21437/interspeech.2021-753

Cited by 22 publications

(19 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text‐based features exhibit superior performance compared with acoustic‐based features [15, 16]. Specifically, in [22], the use of text‐based embedding vectors with pretrained BERT achieved the highest accuracy reported to date at 84.5%, whereas the accuracy using acoustic features remained at 74.6%.…”

Section: Related Workmentioning

confidence: 99%

“…Among these studies, the 2020 ADReSS Challenge [11] used both speech signals and transcribed text, while the 2021 ADReSSo Challenge [12] focused on speech signals, and the 2023 ADReSS-M Challenge [13] considered multilingual speech. In this context, the ADReSSo challenges, which are of interest in this study, are primarily aimed at detecting dementia using acoustic features extracted from audio [15][16][17] or word-embedding representations [9,17,18,22] obtained from transcribed text acquired through ASR.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Alzheimer's disease recognition from spontaneous speech using large language models

Bang,

Han,

Kang

2024

ETRI Journal

View full text Add to dashboard Cite

We propose a method to automatically predict Alzheimer's disease from speech data using the ChatGPT large language model. Alzheimer's disease patients often exhibit distinctive characteristics when describing images, such as difficulties in recalling words, grammar errors, repetitive language, and incoherent narratives. For prediction, we initially employ a speech recognition system to transcribe participants' speech into text. We then gather opinions by inputting the transcribed text into ChatGPT as well as a prompt designed to solicit fluency evaluations. Subsequently, we extract embeddings from the speech, text, and opinions by the pretrained models. Finally, we use a classifier consisting of transformer blocks and linear layers to identify participants with this type of dementia. Experiments are conducted using the extensively used ADReSSo dataset. The results yield a maximum accuracy of 87.3% when speech, text, and opinions are used in conjunction. This finding suggests the potential of leveraging evaluation feedback from language models to address challenges in Alzheimer's disease recognition.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Alzheimer's disease recognition from spontaneous speech using large language models

Bang,

Han,

Kang

2024

ETRI Journal

View full text Add to dashboard Cite

show abstract

“…Pretrained speech representations for impaired speech is a relatively unexplored field. Positive results were obtained for Alzheimer's detection through speech with Wav2Vec [18,19]. However, limited studies have evaluated the effectiveness of speech representations for dysarthric speech recognition.…”

Section: Related Workmentioning

confidence: 99%

Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition

Hernandez¹,

Pérez-Toro²,

Nöth³

et al. 2022

Preprint

View full text Add to dashboard Cite

State-of-the-art automatic speech recognition (ASR) systems perform well on healthy speech. However, the performance on impaired speech still remains an issue. The current study explores the usefulness of using Wav2Vec self-supervised speech representations as features for training an ASR system for dysarthric speech. Dysarthric speech recognition is particularly difficult as several aspects of speech such as articulation, prosody and phonation can be impaired. Specifically, we train an acoustic model with features extracted from Wav2Vec, Hubert, and the cross-lingual XLSR model. Results suggest that speech representations pretrained on large unlabelled data can improve word error rate (WER) performance. In particular, features from the multilingual model led to lower WERs than filterbanks (Fbank) or models trained on a single language. Improvements were observed in English speakers with cerebral palsy caused dysarthria (UASpeech corpus), Spanish speakers with Parkinsonian dysarthria (PC-GITA corpus) and Italian speakers with paralysis-based dysarthria (EasyCall corpus). Compared to using Fbank features, XLSR-based features reduced WERs by 6.8%, 22.0%, and 7.0% for the UASpeech, PC-GITA, and EasyCall corpus, respectively.

show abstract

“…There is also information related to phones in the learned speech representations from the lower layers of the network. Therefore, the pre-trained models can be used to extract features in various speech-related tasks, such as in classification of stuttering and Alzheimer's disease, and in speaker and language identification [49,50,51,63,64].…”

Section: Feature Extraction Based On Pre-trained Modelsmentioning

confidence: 99%

Convolutional Neural Networks for Classification of Voice Qualities from Speech and Neck Surface Accelerometer Signals

Kadiri¹,

Javanmardi²,

Alku³

2022

Interspeech 2022

View full text Add to dashboard Cite

Prior studies in the automatic classification of voice quality have mainly studied support vector machine (SVM) classifiers using the acoustic speech signal as input. Recently, one voice quality classification study was published using neck surface accelerometer (NSA) and speech signals as inputs and using SVMs with hand-crafted glottal source features. The present study examines simultaneously recorded NSA and speech signals in the classification of three voice qualities (breathy, modal, and pressed) using convolutional neural networks (CNNs) as classifier. The study has two goals: (1) to investigate which of the two signals (NSA vs. speech) is more useful in the classification task, and (2) to compare whether deep learning -based CNN classifiers with spectrogram and mel-spectrogram features are able to improve the classification accuracy compared to SVM classifiers using hand-crafted glottal source features. The results indicated that the NSA signal showed better classification of the voice qualities compared to the speech signal, and that the CNN classifier outperformed the SVM classifiers with large margins. The best mean classification accuracy was achieved with mel-spectrogram as input to the CNN classifier (93.8% for NSA and 90.6% for speech).

show abstract

Alzheimer Disease Recognition Using Speech-Based Embeddings From Pre-Trained Models

Cited by 22 publications

References 9 publications

Alzheimer's disease recognition from spontaneous speech using large language models

Alzheimer's disease recognition from spontaneous speech using large language models

Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition

Convolutional Neural Networks for Classification of Voice Qualities from Speech and Neck Surface Accelerometer Signals

Contact Info

Product

Resources

About