Spoken language biomarkers for detecting cognitive impairment

Abstract

In this study we developed an automated system that evaluates speech and language features from audio recordings of neuropsychological examinations of 92 subjects in the Framingham Heart Study. A total of 265 features were used in an elastic-net regularized binomial logistic regression model to classify the presence of cognitive impairment, and to select the most predictive features. We compared performance with a demographic model from 6,258 subjects in the greater study cohort (0.79 AUC), and found that a system that incorporated both audio and text features performed the best (0.92 AUC), with a True Positive Rate of 29% (at 0% False Positive Rate) and a good model fit (Hosmer-Lemeshow test > 0.05). We also found that decreasing pitch and jitter, shorter segments of speech, and responses phrased as questions were positively associated with cognitive impairment.

Detecting Signs of Dementia Using Word Vector Representations

“…Systems have used a combination of feature types extracted from the acoustic signal (such as duration, pauses and general voice quality parameters), as well as from the text (such as those based on part of speech tags, phonetic and word identity). Promising results have been reported on a variety of data with systems initially developed for manual transcripts [10,11,12,13] and later on ASR transcripts with WERs often in the high 40s [14,15,16]. All of the text/transcript features are somewhat shallow in nature though, and effectively modelling semantic, lexical and linguistic characteristics without much notion of word co-occurrence except at a relatively simple level such as Wankerl et al 's use of n-grams [17].…”

How technology is reshaping cognitive assessment: Lessons from the Framingham Heart Study.

“…The value of having digitally stored data is that as automated algorithms for extracting new features are created, they can be easily applied to all available recordings, virtually instantly, creating additional longitudinal metrics. In collaboration with the researchers from MIT, analysis of a subset of recordings has been conducted in which 256 prosodic and speech-to-text features were extracted and used to predict those with incident cognitive impairment (Alhanai, Au, & Glass, under review). Some of the features measured included number of words, vocabulary, pitch, speaker turn taking, hesitations, and speaking rate.…”

Scalable diagnostic screening of mild cognitive impairment using AI dialogue agent

“…Future studies may look to deploy Wizard of Oz evaluation on top of the RL pipeline to quantify the difference between human and AI delivery of questions 38 . Additionally, another promising direction include the immersion of our text-based approach with current state-of-the-art audio-based approaches 11,12 as well as existing biomarkers. This step is beyond the scope of this prototype study, but it can improve the generalizability of our dialogue models and provide interpretability of the discovered features.…”

“…Fortunately, recent studies have shown that simple linguistic markers such as word choice, phrasing (i.e., "utterance") and short speech patterns possess predictive power in assessing MCI status in the elderly population 9 . Note that this is quite different from "speech markers" that involve auditory changes in pronunciations [10][11][12] which reflect early symptomatic changes in speech generation. Behavior and social markers such as language, speech and conversational behaviors reflect cognitive changes that may precede physiological changes and offer a much more cost-effective option for preclinical MCI detection 13,14 , especially if they can be extracted from a non-clinical setting.…”