The Joint Effects of Acoustic and Linguistic Markers for Early Identification of Mild Cognitive Impairment

Abstract

In recent years, behavioral markers such as spoken language and lexical preferences have been studied in the early detection of mild cognitive impairment (MCI) using conversations. While the combination of linguistic and acoustic signals have been shown to be effective in detecting MCI, they have generally been restricted to structured conversations in which the interviewee responds to fixed prompts. In this study, we show that linguistic and acoustic features can be combined synergistically to identify MCI in semi-structured conversations. Using conversational data from an on-going clinical trial (Clinicaltrials.gov: NCT02871921), we find that the combination of linguistic and acoustic features on semi-structured conversations achieves a mean AUC of 82.7, significantly (p < 0.01) out-performing linguistic-only (74.9 mean AUC) or acoustic-only (65.0 mean AUC) detections on hold-out data. Additionally, features (linguistic, acoustic and combination) obtained from semi-structured conversations outperform their counterparts obtained from structured weekly conversations in identifying MCI. Some linguistic categories are significantly better at predicting MCI status (e.g., death, home) than others.

Detection of Mild Cognitive Impairment from Language Markers with Crossmodal Augmentation

“…The early MCI detection from language data is formulated as a classification problem, 12 using language markers from a subject to predict the subject’s clinical label. The proposed solution leverages the entire data available for training.…”

“…Early detection of MCI is of great clinical importance and predictive models are built from a variety of data types, such as clinical information, 16 neuroimaging, 4 , 17 and, more recently, digital biomarkers. 12 Neuroimaging captures the structural information of the brain, and therefore imaging markers, especially from structural MRI, 17 have shown great sensitivity. Besides being non-intrusive, the cost of imaging markers is still prohibitive for large-scale screening of early dementia.…”

“…Recent advances in digital biomarkers, 18 such as language markers, have provided an accessible and affordable alternative. 12 From the spontaneous speech, we can extract linguistic features (e.g., word preference, syntactic features, semantic features, data-driven word embedding) and acoustic information (e.g., MFCC). 11 , 12 It has been recently shown that combining acoustic features and linguistic features delivers an improved prediction performance.…”

“… 12 From the spontaneous speech, we can extract linguistic features (e.g., word preference, syntactic features, semantic features, data-driven word embedding) and acoustic information (e.g., MFCC). 11 , 12 It has been recently shown that combining acoustic features and linguistic features delivers an improved prediction performance. 12 , 13 The development of language markers is still in the very early phase, with limited data available for modeling.…”

“… 11 , 12 It has been recently shown that combining acoustic features and linguistic features delivers an improved prediction performance. 12 , 13 The development of language markers is still in the very early phase, with limited data available for modeling. The analysis can benefit from more data from different data sources to deliver high predictive performance.…”