Deep convolutional neural networks for automated scoring of pentagon copying test results

Abstract

This study aims to investigate the accuracy of a fine-tuned deep convolutional neural network (CNN) for evaluating responses to the pentagon copying test (PCT). To develop a CNN that could classify PCT images, we fine-tuned and compared the pre-trained CNNs (GoogLeNet, VGG-16, ResNet-50, Inception-v3). To collate our training dataset, we collected 1006 correct PCT images and 758 incorrect PCT images drawn on a test sheet by dementia suspected patients at the Osaka City Kosaiin Hospital between April 2009 and December 2012. For a validation dataset, we collected PCT images from consecutive patients treated at the facility in April 2020. We examined the ability of the CNN to detect correct PCT images using a validation dataset. For a validation dataset, we collected PCT images (correct, 41; incorrect, 16) from 57 patients. In the validation testing for an ability to detect correct PCT images, the fine-tuned GoogLeNet CNN achieved an area under the receiver operating characteristic curve of 0.931 (95% confidence interval 0.853–1.000). These findings indicate that our fine-tuned CNN is a useful method for automatically evaluating PCT images. The use of CNN-based automatic scoring of PCT can potentially reduce the burden on assessors in screening for dementia.

Clock-drawing Test and Cube-copying Test to Quickly Screen Dementia: In Combination with the Mini-mental State Examination Scores

“…Maruta et al reported that the sensitivity of PCT was low at 29.4%; however, when pre-trained artifical intelligence (AI) models (fine-tuned GoogLeNet CNN) were used to distinguish the difference between pentagons drawn by dementia patients and those drawn by non-dementia patients, AI was able to improve both the sensitivity and specificity with a high level of accuracy, namely approximately 79 to 93% ( 12 ). Administering PCT and CCT to patients with Parkinson's disease dementia, Alty JE reported that the sensitivity of PCT was 26% and CCT was 74%, showing that CCT was superior to PCT in sensitivity ( 13 ).…”

Interpretable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons in older adults

“…Automated scoring methods that utilize machine learning methods offer the potential to address some or all of these weaknesses. For example, deep-learning techniques have demonstrated promising performance in automating ratings for the PDT 7,8 , Rey Complex Figure Test 9,10 , and Clock Drawing Test 11,12 . However, the primary objective of the previous automated scoring is to reproduce human-based conventional ratings and few machine learning approaches directly predict cognitive performance from drawings 13 .…”

Explainable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons

“…Automated scoring methods that utilize machine learning methods offer the potential to address some or all of these weaknesses. For example, deep-learning techniques have demonstrated promising performance in automating ratings for the PDT 7 , 8 , Rey Complex Figure Test 9 , 10 , and Clock Drawing Test 11 , 12 . However, the primary objective of the previous automated scoring is to reproduce human-based conventional ratings, and few machine learning approaches directly predict cognitive performance from drawings 13 .…”