Fantastic Features and Where to Find Them: Detecting Cognitive Impairment with a Subsequence Classification Guided Approach

Eyre, Benjamin; Balagopalan, Aparna; Novikova, Jekaterina

doi:10.18653/v1/2020.wnut-1.25

Cited by 5 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While we extracted a large number of linguistic and acoustic features to capture a wide range of linguistic and acoustic changes in speech associated with AD, based on a survey of prior literature (Yancheva et al, 2015;Fraser et al, 2016; Pou- Prom and Rudzicz, 2018;Zhu et al, 2019), we are also interested in identifying the most differentiating features between AD and non-AD speech. In order to study statistically significant differences in linguistic/acoustic phenomena, we perform independent t-tests between feature means for each class in the ADReSS training set, following the methodology followed by Eyre et al (2020). 87 features are significantly different between the two groups at p < 0.05.…”

Section: Feature Differentiation Analysismentioning

confidence: 99%

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech

Balagopalan

Eyre²,

Robin³

et al. 2021

Front. Aging Neurosci.

Self Cite

View full text Add to dashboard Cite

Introduction: Research related to the automatic detection of Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing, and machine learning provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms. In this study, we compare and contrast the performance of two common approaches for automatic AD detection from speech on the same, well-matched dataset, to determine the advantages of using domain knowledge vs. pre-trained transfer models.Methods: Audio recordings and corresponding manually-transcribed speech transcripts of a picture description task administered to 156 demographically matched older adults, 78 with Alzheimer's Disease (AD) and 78 cognitively intact (healthy) were classified using machine learning and natural language processing as “AD” or “non-AD.” The audio was acoustically-enhanced, and post-processed to improve quality of the speech recording as well control for variation caused by recording conditions. Two approaches were used for classification of these speech samples: (1) using domain knowledge: extracting an extensive set of clinically relevant linguistic and acoustic features derived from speech and transcripts based on prior literature, and (2) using transfer-learning and leveraging large pre-trained machine learning models: using transcript-representations that are automatically derived from state-of-the-art pre-trained language models, by fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models.Results: We compared the utility of speech transcript representations obtained from recent natural language processing models (i.e., BERT) to more clinically-interpretable language feature-based methods. Both the feature-based approaches and fine-tuned BERT models significantly outperformed the baseline linguistic model using a small set of linguistic features, demonstrating the importance of extensive linguistic information for detecting cognitive impairments relating to AD. We observed that fine-tuned BERT models numerically outperformed feature-based approaches on the AD detection task, but the difference was not statistically significant. Our main contribution is the observation that when tested on the same, demographically balanced dataset and tested on independent, unseen data, both domain knowledge and pretrained linguistic models have good predictive performance for detecting AD based on speech. It is notable that linguistic information alone is capable of achieving comparable, and even numerically better, performance than models including both acoustic and linguistic features here. We also try to shed light on the inner workings of the more black-box natural language processing model by performing an interpretability analysis, and find that attention weights reveal interesting patterns such as higher attribution to more important information content units in the picture description task, as well as pauses and filler words.Conclusion: This approach supports the value of well-performing machine learning and linguistically-focussed processing techniques to detect AD from speech and highlights the need to compare model performance on carefully balanced datasets, using consistent same training parameters and independent test datasets in order to determine the best performing predictive model.

show abstract

Section: Feature Differentiation Analysismentioning

confidence: 99%

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech

Balagopalan

Eyre²,

Robin³

et al. 2021

Front. Aging Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Previous literature highlights the importance of pauses in Alzheimer's disease detection from speech (Calley et al, 2010;Mack et al, 2013;Seifart et al, 2018). Several authors report increases in AD detection performance by extracting acoustic features such as filled pause counts (Eyre et al, 2020;Tóth et al, 2015Tóth et al, , 2018Pistono et al, 2016). Removal of such information should make it more difficult for a model to accurately detect AD-related samples of text.…”

Section: Perturbation Approachesmentioning

confidence: 99%

Robustness and Sensitivity of BERT Models Predicting Alzheimer’s Disease from Text

Novikova¹

2021

Proceedings of the Seventh Workshop on Noisy User-Generated Text (W-Nut 2021)

Self Cite

View full text Add to dashboard Cite

Understanding robustness and sensitivity of BERT models predicting Alzheimer's disease from text is important for both developing better classification models and for understanding their capabilities and limitations. In this paper, we analyze how a controlled amount of desired and undesired text alterations impacts performance of BERT. We show that BERT is robust to natural linguistic variations in text. On the other hand, we show that BERT is not sensitive to removing clinically important information from text.

show abstract

“…While we extracted a large number of linguistic and acoustic features to capture a wide range of linguistic and acoustic changes in speech associated with AD, based on a survey of prior literature ; Pou-Prom and Zhu et al, 2019), we are also interested in identifying the most differentiating features between AD and non-AD speech. In order to study statistically significant differences in linguistic/acoustic phenomena, we perform independent t-tests between feature means for each class in the ADReSS training set, following the methodology followed by Eyre et al (2020). 87 features are significantly different between the two groups at p < 0.05.…”

Section: Feature Differentiation Analysismentioning

confidence: 99%

Alzheimer's Dementia Recognition Through Spontaneous Speech

2021

Frontiers Research Topics

View full text Add to dashboard Cite

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals. Frontiers Journal SeriesThe Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too. Dedication to QualityEach Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public -and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews. Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

show abstract

Fantastic Features and Where to Find Them: Detecting Cognitive Impairment with a Subsequence Classification Guided Approach

Cited by 5 publications

References 33 publications

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech

Robustness and Sensitivity of BERT Models Predicting Alzheimer’s Disease from Text

Alzheimer's Dementia Recognition Through Spontaneous Speech

Contact Info

Product

Resources

About