DCTclock: Clinically-Interpretable and Automated Artificial Intelligence Analysis of Drawing Behavior for Capturing Cognition

Abstract

Developing tools for efficiently measuring cognitive change specifically and brain health generally—whether for clinical use or as endpoints in clinical trials—is a major challenge, particularly for conditions such as Alzheimer's disease. Technology such as connected devices and advances in artificial intelligence offer the possibility of creating and deploying clinical-grade tools with high sensitivity, rapidly, cheaply, and non-intrusively. Starting from a widely-used paper and pencil cognitive status test—The Clock Drawing Test—we combined a digital input device to capture time-stamped drawing coordinates with a machine learning analysis of drawing behavior to create DCTclock™, an automated analysis of nuances in cognitive performance beyond successful task completion. Development and validation was conducted on a dataset of 1,833 presumed cognitively unimpaired and clinically diagnosed cognitively impaired individuals with varied neurological conditions. We benchmarked DCTclock against existing clock scoring systems and the Mini-Mental Status Examination, a widely-used but lengthier cognitive test, and showed that DCTclock offered a significant improvement in the detection of early cognitive impairment and the ability to characterize individuals along the Alzheimer's disease trajectory. This offers an example of a robust framework for creating digital biomarkers that can be used clinically and in research for assessing neurological function.

Digital Cognitive Biomarker for Mild Cognitive Impairments and Dementia: A Systematic Review

“…Table S5: Diagnostic performance of digital cognitive biomarkers for cognitive impairments (CI). Table S6: Quality assessment based on The Newcastle–Ottawa Scale (NOS) [ 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 ]. Figure S1: Between-group mean difference in cognitive outcomes measured by computerized tests in mild cognitive impairments (MCI) and normal controls.…”

Aging in the Digital Age: Using Technology to Increase the Reach of the Clinician Expert and Close the Gap Between Health Span and Life Span

“…One example is DCTclock TM (22,23), a digital version of the Clock Drawing Test, used for over 50 years to screen for cognitive impairment. Participants are asked to draw a pair of clocks, and DCTclock captures and automatically analyzes the assessment using machine learning algorithms, thus reducing bias from subjective judgment.…”

“…The automated scoring models have significantly outperformed traditional clock drawing test, producing (area under the curve) AUC scores ranging from 0.89 to 0.93 for screening, compared against published AUC scores from existing clinician scoring systems of 0.66-0.79. DCTclock has also been shown to outperform traditional neuropsychological tests like the MMSE in detecting early cognitive impairment and demonstrated the ability to characterize individuals along the AD trajectory (23). In a push to detect the earliest cognitive changes associated with AD, two validation studies of DCTclock on patients with preclinical AD showed significant association of DCTclock measures with both amyloid and tau PET imaging markers; the first study also showed significant association with the Preclinical Alzheimer's Cognitive Composite (24), while the second demonstrated the ability to further tailor the DCTclock algorithm to improve detection of Alzheimer's pathology (25).…”

Variational autoencoder provides proof of concept that compressing CDT to extremely low-dimensional space retains its ability of distinguishing dementia

“…This technology, therefore provides additional benefits compared to the traditional clock drawing test because it can analyze the process by which the drawing was created rather than solely relying on the final created image. Automated scoring models developed on the dCDT in recent years 18 – 20 have significantly outperformed traditional pen and paper clock drawing test. Their AUROC scores range from 0.89 to 0.93, compared against published AUROC scores from existing clinician scoring systems of 0.66–0.79 21 .…”