The Accuracy of Speech and Linguistic Analysis in Early Diagnostics of Neurocognitive Disorders in a Memory Clinic Setting

Abstract

Objective To investigate whether automatic analysis of the Semantic Verbal Fluency test (SVF) is reliable and can extract additional information that is of value for identifying neurocognitive disorders. In addition, the associations between the automatically derived speech and linguistic features and other cognitive domains were explored. Method We included 135 participants from the memory clinic of the Maastricht University Medical Center+ (with Subjective Cognitive Decline [SCD; N = 69] and Mild Cognitive Impairment [MCI]/dementia [N = 66]). The SVF task (one minute, category animals) was recorded and processed via a mobile application, and speech and linguistic features were automatically extracted. The diagnostic performance of the automatically derived features was investigated by training machine learning classifiers to differentiate SCD and MCI/dementia participants. Results The intraclass correlation for interrater reliability between the clinical total score (golden standard) and automatically derived total word count was 0.84. The full model including the total word count and the automatically derived speech and linguistic features had an Area Under the Curve (AUC) of 0.85 for differentiating between people with SCD and MCI/dementia. The model with total word count only and the model with total word count corrected for age showed an AUC of 0.75 and 0.81, respectively. Semantic switching correlated moderately with memory as well as executive functioning. Conclusion The one-minute SVF task with automatically derived speech and linguistic features was as reliable as the manual scoring and differentiated well between SCD and MCI/dementia. This can be considered as a valuable addition in the screening of neurocognitive disorders and in clinical practice.

The Reliability and Clinical Validation of Automatically-Derived Verbal Memory Features of the Verbal Learning Test in Early Diagnostics of Cognitive Impairment

“…The application used the iPad’s standard internal microphone, which was placed in front of the participant. After speech responses were recorded, they were sent to the backend of ki:elements for preprocessing (such as cutting recordings into relevant parts and audio transformation), automatic speech recognition, and feature extraction [ 31 ]. This resulted in two different measurements of both the total immediate recall and delayed recall: the automatically derived ASR score, and the clinician’s independent score.…”

Validation of an Automated Speech Analysis of Cognitive Tasks within a Semiautomated Phone Assessment

“…As part of the DeepSpA (Deep Speech Analysis for Cognitive Assessment in Clinical Trials) project, 140 participants were included via the BioBank Alzheimer Centre Limburg (BBACL) study [ 11 ]. Out of the 140 participants, 94 (56 SCD, 38 MCI) completed the semiautomated phone assessment (see Table 1 for participant characteristics).…”

“…At baseline, each participant underwent a face-to-face NPA at the hospital as part of a clinical routine [ 11 ]. After 6 months, participants underwent a semiautomated phone assessment, in which a well-trained test leader guided the participant through the phone assessment.…”

“…The total word count of both the VLT and SVF and the speech and linguistic features were automatically extracted. Examples of speech and linguistic features of the VLT are “primacy count,” “serial clustering,” and “recency count” and for the SVF, examples are “semantic clustering,” “temporal clustering,” and “mean word frequency” [ 8 , 11 , 12 , 24 , 25 ] (for the full list of the speech and linguistic features, see online suppl. Supplement 1; for all online suppl.…”

“…Moreover, ASR may be helpful in differentiating people with subjective cognitive decline (SCD) from MCI/dementia in a clinical face-to-face setting (Possemis et al, unpubl. data) [ 11 , 12 ]. In addition, comparing manual and automatic scoring in a face-to-face setting showed a high level of agreement, demonstrating that ASR can provide accurate results.…”