Combining Multimodal Behavioral Data of Gait, Speech, and Drawing for Classification of Alzheimer’s Disease and Mild Cognitive Impairment

Abstract

Background: Gait, speech, and drawing behaviors have been shown to be sensitive to the diagnosis of Alzheimer’s disease (AD) and mild cognitive impairment (MCI). However, previous studies focused on only analyzing individual behavioral modalities, although these studies suggested that each of these modalities may capture different profiles of cognitive impairments associated with AD. Objective: We aimed to investigate if combining behavioral data of gait, speech, and drawing can improve classification performance compared with the use of individual modality and if each of these behavioral data can be associated with different cognitive and clinical measures for the diagnosis of AD and MCI. Methods: Behavioral data of gait, speech, and drawing were acquired from 118 AD, MCI, and cognitively normal (CN) participants. Results: Combining all three behavioral modalities achieved 93.0%accuracy for classifying AD, MCI, and CN, and only 81.9%when using the best individual behavioral modality. Each of these behavioral modalities was statistically significantly associated with different cognitive and clinical measures for diagnosing AD and MCI. Conclusion: Our findings indicate that these behaviors provide different and complementary information about cognitive impairments such that classification of AD and MCI is superior to using either in isolation.

Automated Analysis of Drawing Process to Estimate Global Cognition in Older Adults: Preliminary International Validation on the US and Japan Data Sets

“…Drawing tests have been widely used for screening cognitive impairments and dementia (eg, trail making [13] and clock drawing [14]), and automated analysis of the drawing process has shown that features characterizing the drawing process are sensitive to cognitive impairments and diagnoses of dementia [15][16][17][18]. For example, reduction in the drawing speed and increases in its variability, as well as increased pauses between drawing motions, have been reported as statistically significant features for assessment of impaired global cognition [19,20], as well as for detecting Alzheimer disease (AD) and mild cognitive impairment (MCI) [21][22][23][24]. Machine learning models based on these drawing features have succeeded in estimating measures of global cognition [25,26] and classifying AD, MCI, and control individuals [23][24][25]27].…”

“…For example, reduction in the drawing speed and increases in its variability, as well as increased pauses between drawing motions, have been reported as statistically significant features for assessment of impaired global cognition [19,20], as well as for detecting Alzheimer disease (AD) and mild cognitive impairment (MCI) [21][22][23][24]. Machine learning models based on these drawing features have succeeded in estimating measures of global cognition [25,26] and classifying AD, MCI, and control individuals [23][24][25]27]. However, there has been little evidence of the capability of automated analysis of the drawing process for assessment of cognitive performance across different populations, even though applicability across the intended populations is a requirement for machine learning-based health care tools, including those for screening of dementia [1,28,29].…”

Neural Mechanisms of Motor Dysfunction in Mild Cognitive Impairment and Alzheimer’s Disease: A Systematic Review

“…The pattern of widespread motor behavioral deficits in MCI and AD implies that gait should not be the single (or even the primary) outcome in motor behavioral studies in MCI and AD. In fact, previous work has shown that combining motor behavioral measures more strongly predicts [ 94 ] or better classifies [ 95 ] AD than single measures do. Future studies should therefore combine assessments addressing the breadth of motor domains when studying neural mechanisms underlying motor dysfunction in MCI and AD if we want to obtain an integral explanation and if we want to understand which neural deficits in MCI and AD are responsible for either cognitive deficits, motor deficits, or both.…”

Automatic Assessment of Loneliness in Older Adults Using Speech Analysis on Responses to Daily Life Questions

“…One of our contributions lies in providing the first empirical evidence showing the feasibility of using the automatic analysis of speech for detecting changes due to loneliness in older adults. In addition, many recent studies have explored the use of speech data for healthcare applications for monitoring various types of health statuses in older adults, for example, for detecting cognitive impairments ( 31 , 82 – 84 ) and Alzheimer's disease ( 26 , 28 , 29 , 32 , 85 – 90 ), for detecting depression ( 38 , 91 , 92 ), and for predicting driving risks ( 30 ). Together with these previous studies, our results may help future efforts toward developing applications using speech data for automatically and simultaneously monitoring various types of health statuses including loneliness.…”