Towards a Multimodal Multisensory Cognitive Assessment Framework

Niemann, Mira; Prange, Alexander; Sonntag, Daniel

doi:10.1109/cbms.2018.00012

Cited by 16 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This includes verbal interview like tests, in which the physician takes notes of the patient's answers, as well as tests in which the patient is asked to write or sketch. In this work, a selection of paper-pencil cognitive assessments is considered based on feedback from domain experts and a recent market analysis of existing, widely used, cognitive assessments conducted by Niemann et al ( 2018 ). These assessments were successfully digitalized during the Interakt project (Interactive Cognitive Assessment Tool) (Sonntag, 2017 ), and are summarized in Table 1 , namely Age-Concentration (AKT) (Gatterer et al, 1989 ), Clock Drawing Test (CDT) (Freedman et al, 1994 ), CERAD Neuropsychological Battery (Morris et al, 1988 ), Dementia Detection (DemTect) (Kalbe et al, 2004 ), Mini-Mental State Examination (MMSE) (Folstein et al, 1975 ), Montreal Cognitive Assessment (MoCA) (Nasreddine et al, 2005 ), Rey-Osterrieth Complex Figure (ROCF) (Duley et al, 1993 ), and Trail Making Test (TMT) (Reitan, 1986 ).…”

Section: Related Workmentioning

confidence: 99%

“…Several automated scoring systems based on multi-stroke sketch recognition and machine learning have been published previously (Canham et al, 2000 ; Davis et al, 2015 ; Niemann et al, 2018 ; Prange and Sonntag, 2019 ; Prange et al, 2019 ). Such systems typically score assessments based on their scoring schemes, thereby simulating the work of a clinician, and need to include task dependent knowledge for each type of assessment.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling Users' Cognitive Performance Using Digital Pen Features

Prange

Sonntag

2022

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Digital pen features model characteristics of sketches and user behavior, and can be used for various supervised machine learning (ML) applications, such as multi-stroke sketch recognition and user modeling. In this work, we use a state-of-the-art set of more than 170 digital pen features, which we implement and make publicly available. The feature set is evaluated in the use case of analyzing paper-pencil-based neurocognitive assessments in the medical domain. Most cognitive assessments, for dementia screening for example, are conducted with a pen on normal paper. We record these tests with a digital pen as part of a new interactive cognitive assessment tool with automatic analysis of pen input. The physician can, first, observe the sketching process in real-time on a mobile tablet, e.g., in telemedicine settings or to follow Covid-19 distancing regulations. Second, the results of an automatic test analysis are presented to the physician in real-time, thereby reducing manual scoring effort and producing objective reports. As part of our evaluation we examine how accurately different feature-based, supervised ML models can automatically score cognitive tests, with and without semantic content analysis. A series of ML-based sketch recognition experiments is conducted, evaluating 10 modern off-the-shelf ML classifiers (i.e., SVMs, Deep Learning, etc.) on a sketch data set which we recorded with 40 subjects from a geriatrics daycare clinic. In addition, an automated ML approach (AutoML) is explored for fine-tuning and optimizing classification performance on the data set, achieving superior recognition accuracies. Using standard ML techniques our feature set outperforms all previous approaches on the cognitive tests considered, i.e., the Clock Drawing Test, the Rey-Osterrieth Complex Figure Test, and the Trail Making Test, by automatically scoring cognitive tests with up to 87.5% accuracy in a binary classification task.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Modeling Users' Cognitive Performance Using Digital Pen Features

Prange

Sonntag

2022

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Experiments with children deal with one-year-old children to seek changes in the attentional processes [40] and treat children with Autism Spectrum Disorder (ASD) in two different game-play settings [41], [42]. Technology evaluated in the papers varies between games for entertainment [33], [43], and technology aids for diagnosis [44].…”

Section: Related Workmentioning

confidence: 99%

Towards Classifying Cognitive Performance by Sensing Electrodermal Activity in Children With Specific Learning Disorders

et al. 2020

View full text Add to dashboard Cite

When children suffer from cognitive disorders, school performance and social environment are affected. Measuring changes in cognitive progress is essential for assessing the clinical follow-up of the patient's cognitive abilities. This process is considered as a challenge in ambulatory settings, where follow-ups should be non-invasive and continuous. Psychophysiological measures are an objective and unobtrusive evaluation alternative for recognizing cognitive changes. This paper aims to validate the relationship between cognition and the changes in physiological signals of children suffering from Specific Learning Disorders (SLD). This validation was carried out in an eHealth rehabilitation context (with the HapHop-Physio game). Electrodermal activity (EDA) signals were collected, processed, and analyzed through a machine learning approach. Obtained results were: a dataset built from wearable physiological data and a supervised classification model. The classification model can identify the children's cognitive performance (class) from the features of the tonic component of the EDA signal (attributes) with an accuracy of 79.95%. The presented results evidence that psychophysiological measures could allow for a highly objective follow-up for patients. They can also lead to creating a basis for further improvement of rehabilitation environments and developing neurofeedback applications.

show abstract

“…Bio-physiological signals such as driver-facing sensors and relay features such as the hands, fingers, head, eye gaze, feet, facial expression, voice tone, brain activity, heart rate, skin conductance, steering-wheel grip force, muscle activity and skin/body temperature are other signals which could be used for cognitive load estimation [15]. Other methods focus on the brain activity through an electroencephalogram (EEG) by identifying frequency bands which are likely to capture the cognitive load and brain locations related to it [16]; in contrast, methods based on an electrocardiogram (ECG) assume that heart rhythms, controlled by the autonomic nervous system, can fluctuate with cognitive load [17] or on the electrodermal activity (EDA) [18].…”

Section: Virtual Wallmentioning

confidence: 99%

Heatmap-Based Method for Estimating Drivers’ Cognitive Distraction

Musabini

Chetitah

2020

2020 IEEE 19th International Conference on Cognitive Informatics &Amp; Cognitive Computing (ICCI*CC)

View full text Add to dashboard Cite

In order to increase road safety, among the visual and manual distractions, modern intelligent vehicles need also to detect cognitive distracted driving (i.e., the drivers mind wandering). In this study, the influence of cognitive processes on the drivers gaze behavior is explored. A novel image-based representation of the driver's eye-gaze dispersion is proposed to estimate cognitive distraction. Data are collected on open highway roads, with a tailored protocol to create cognitive distraction. The visual difference of created shapes shows that a driver explores a wider area in neutral driving compared to distracted driving. Thus, support vector machine (SVM)based classifiers are trained, and 85.2% of accuracy is achieved for a two-class problem, even with a small dataset. Thus, the proposed method has the discriminative power to recognize cognitive distraction using gaze information. Finally, this work details how this image-based representation could be useful for other cases of distracted driving detection.

show abstract

Towards a Multimodal Multisensory Cognitive Assessment Framework

Cited by 16 publications

References 18 publications

Modeling Users' Cognitive Performance Using Digital Pen Features

Modeling Users' Cognitive Performance Using Digital Pen Features

Towards Classifying Cognitive Performance by Sensing Electrodermal Activity in Children With Specific Learning Disorders

Heatmap-Based Method for Estimating Drivers’ Cognitive Distraction

Contact Info

Product

Resources

About