Predicting MCI Status From Multimodal Language Data Using Cascaded Classifiers

Abstract

Recent work has indicated the potential utility of automated language analysis for the detection of mild cognitive impairment (MCI). Most studies combining language processing and machine learning for the prediction of MCI focus on a single language task; here, we consider a cascaded approach to combine data from multiple language tasks. A cohort of 26 MCI participants and 29 healthy controls completed three language tasks: picture description, reading silently, and reading aloud. Information from each task is captured through different modes (audio, text, eye-tracking, and comprehension questions). Features are extracted from each mode, and used to train a series of cascaded classifiers which output predictions at the level of features, modes, tasks, and finally at the overall session level. The best classification result is achieved through combining the data at the task level (AUC = 0.88, accuracy = 0.83). This outperforms a classifier trained on neuropsychological test scores (AUC = 0.75, accuracy = 0.65) as well as the “early fusion” approach to multimodal classification (AUC = 0.79, accuracy = 0.70). By combining the predictions from the multimodal language classifier and the neuropsychological classifier, this result can be further improved to AUC = 0.90 and accuracy = 0.84. In a correlation analysis, language classifier predictions are found to be moderately correlated (ρ = 0.42) with participant scores on the Rey Auditory Verbal Learning Test (RAVLT). The cascaded approach for multimodal classification improves both system performance and interpretability. This modular architecture can be easily generalized to incorporate different types of classifiers as well as other heterogeneous sources of data (imaging, metabolic, etc.).

Artificial Intelligence, Speech, and Language Processing Approaches to Monitoring Alzheimer’s Disease: A Systematic Review

“…[ 94 ], although it is not included in their analysis. Other analyzed modalities include neuroimaging data, such as MRI [ 67 ] and fNIRS [ 64 ], eye-tracking [ 7, 94 ], or gait information [ 71 ].…”

“…Only 39% (20) of the reviewed studies present class balance, that is, the number of participants is evenly distributed across the two, three, or four diagnostic categories [ 7, 8, 11, 50, 60, 62–64, 75, 78, 82, 84, 86, 88–91, 94, 95, 98 ]. Among these 20 studies, one reports only between-class age and gender balance [ 94 ]; another one reports class balance, within-class gender balance, and between-class age and gender balance [ 11 ].…”

“…As expected, a number of studies (41%) were conducted through English. However, there is a fair amount of papers using data in a variety of languages, including: Italian [ 91 ], Portuguese [ 57, 90 ], Chinese and Taiwanese [ 82 ], French [ 50, 69, 77, 96, 102 ], Hungarian [ 62, 63, 99 ], Spanish [ 83, 89, 100 ], Swedish [ 7, 59, 87 ], Japanese [ 64, 71, 94 ], Turkish [ 84 ], Persian [ 65 ], Greek [ 61, 88 ], German [ 68, 74 ], or reported as multilingual [ 60, 79, 80 ].…”

“…[ 90 ] and Fraser et al. [ 7 ] report their data being accessible upon request, and Chien et al. [ 82 ] works with data available from DementiaBank.…”

“…Among the remaining papers, a few set specific results from a comparable work as their baseline [ 6, 65 ] or from their own previous work [ 75 ]. Others calculate their baseline by training a classifier with all the generated features, that is, before attempting to reduce the feature set with either selection or extraction methods [ 83, 95, 99 ], with cognitive scores only [ 7, 44, 70, 71 ] or by training a classifier with demographic scores only [ 63 ]. Some baseline classifiers are also trained with a set of speech-based features that excludes the feature targeted by the research question.…”