Modeling Users' Cognitive Performance Using Digital Pen Features

Abstract

Digital pen features model characteristics of sketches and user behavior, and can be used for various supervised machine learning (ML) applications, such as multi-stroke sketch recognition and user modeling. In this work, we use a state-of-the-art set of more than 170 digital pen features, which we implement and make publicly available. The feature set is evaluated in the use case of analyzing paper-pencil-based neurocognitive assessments in the medical domain. Most cognitive assessments, for dementia screening for example, are conducted with a pen on normal paper. We record these tests with a digital pen as part of a new interactive cognitive assessment tool with automatic analysis of pen input. The physician can, first, observe the sketching process in real-time on a mobile tablet, e.g., in telemedicine settings or to follow Covid-19 distancing regulations. Second, the results of an automatic test analysis are presented to the physician in real-time, thereby reducing manual scoring effort and producing objective reports. As part of our evaluation we examine how accurately different feature-based, supervised ML models can automatically score cognitive tests, with and without semantic content analysis. A series of ML-based sketch recognition experiments is conducted, evaluating 10 modern off-the-shelf ML classifiers (i.e., SVMs, Deep Learning, etc.) on a sketch data set which we recorded with 40 subjects from a geriatrics daycare clinic. In addition, an automated ML approach (AutoML) is explored for fine-tuning and optimizing classification performance on the data set, achieving superior recognition accuracies. Using standard ML techniques our feature set outperforms all previous approaches on the cognitive tests considered, i.e., the Clock Drawing Test, the Rey-Osterrieth Complex Figure Test, and the Trail Making Test, by automatically scoring cognitive tests with up to 87.5% accuracy in a binary classification task.

Application of machine learning in measurement of ageing and geriatric diseases: A systematic review

“…Among neurodegenerative disorders, another part of disease is the dementia (23,24) and cognitive disfunctions. Machine learning was used to diagnose predict the cognitive dysfunction mostly using the population based data(25-29), mostly using regression models from supervised ML, another type of studies have used biomarker variables (30), digital device features (31,32) and hospital records (33) to analyse the risk factors of cognitive dysfunction. Similarly, for dementia, most of the studies used population based surveys (34)(35)(36)(37) and clinical datasets(38) using classi cation and deep learning methods of ML.…”

Application of machine learning in measurement of ageing and geriatric diseases: a systematic review

“…ML models- Support vector regression, Lasso linear regression Machine learning regressors are able to extract information from speech features and perform above baseline in predicting anxiety, apathy, and depression scores. Different NPS seem to be characterized by distinct speech features, which are easily extractable automatically from short vocal tasks NA 54 Prange and Sonntag [ 69 ] 2022 To use digital pen features, such as geometrical, spacial, temporal and pressure characteristics to model user's cognitive performance (binary classification) 40 subjects from a geriatric daycare clinic Neurodegenrative Disease Traditional approach—content analysis of drawn features Current approach- digital cognitive assessment ML models- SVM, LR, nearest neighbors, naïve bayes, DT, RF, AdaBoost, Gradient boosted trees, deep learning ML techniques our feature set outperforms all previous approaches on the cognitive tests considered, i.e., the Clock Drawing Test, the Rey-Osterrieth Complex Figure Test, and the Trail Making Test in a binary classification task Accuracy, F1 score, Log loss, Precision, Recall, AUC was calculated for feature subsets 55 Younan et al [ 70 ] 2020 1) to examine whether PM2.5 (particulate matter) affects the episodic memory decline, 2) to explore the potential mediating role of increased neuroanatomic risk of Alzheimer’s disease associated with exposure 531 older females from Women's Health Initiative Study of Cognitive Ageing & the Women's Health Initiative Memory Study of Magnetic Resonance Imaging (1999–2010) Neurodegenrative Disease Subjects were assigned Alzheimer's disease pattern similarity scores through brain MRI. Method applied- Structural Equation Modeling (SEM) The continuum of PM2.5 neurotoxicity that contributes to early decline of immediate free recall/new learning at the preclinical stage, which is mediated by progressive atrophy of grey matter indicative of increased Alzheimer’s disease risk, independent of cerebrovascular damage NA 56 Aschwanden et al [ 71 ] 2020 To estimate the relative importance of selected predictors in forecasting cognitive impairement and dementia in a large scale population representative sample 9,979 older adults from HRS Neurodegenrative Disease Combined methodology Estimatinf relative importance- RF and survival analysis estimate effect size for imp vars- Cox proportional hazard model ...…”

Digital ink and differentiated subjective ratings for cognitive load measurement in middle childhood

“…Smartpens record three basic features: x and y coordinates, pressure on the pen tip and time stamps. By means of these, a large set of sophisticated metrics of pen use can be calculated (for an overview, see Prange & Sonntag, 2022). For our purpose, only selected indicators for velocity and pressure of the normalized sketches were considered that had already yielded promising results in previous studies regarding cognitive load measurement.…”

“…The identification of reliable smartpen measures for cognitive load also contributes to using smartpens as helpful tool for people with special needs. Smartpens measures have been used for various diagnostic purposes: they were found to indicate developmental coordination disorders (Rosenblum & Livneh-Zirinski, 2008), dysgraphia (Rosenblum & Dror, 2016), autism spectrum disorder (Rosenblum et al, 2019), Parkinson's disease (Drotár et al, 2016) and cognitive impairment (Prange & Sonntag, 2022). Since cognitive (over)load is closely linked to cognitive impairment, future research should bring together findings on smartpen-based cognitive load indicators and cognitive deficits.…”