Cross-lingual detection of mild cognitive impairment based on temporal parameters of spontaneous speech

Abstract

Mild Cognitive Impairment (MCI) is a heterogeneous clinical syndrome, often considered as the prodromal stage of dementia. It is characterized by the subtle deterioration of cognitive functions, including memory, executive functions and language. Mainly due to the tenuous nature of these impairments, a high percentage of MCI cases remain undetected. There is evidence that language changes in MCI are present even before the manifestation of other distinctive cognitive symptoms, which offers a chance for early recognition. A cheap non-invasive way of early screening could be the use of automatic speech analysis. Earlier, our research team developed a set of speech temporal parameters, and demonstrated its applicability for MCI detection. For the automatic extraction of these attributes, a Hungarianlanguage ASR system was employed to match the native language of the MCI and healthy control (HC) subjects. In practical applications, however, it would be convenient to use exactly the same tool, regardless of the language spoken by the subjects. In this study we show that our temporal parameter set, consisting of articulation rate, speech tempo and various other attributes describing the hesitation of the subject, can indeed be reliably extracted regardless of the language of the ASR system used. For this purpose, we performed experiments both on English-speaking and on Hungarian-speaking MCI patients and healthy control subjects, using English and Hungarian ASR systems in both cases. Our experimental results indicate that the language on which the ASR system was trained only slightly affects the MCI classification performance, because we got quite similar scores (67-92%) as we did in the monolingual cases (67-92% as well). As our last investigation, we compared the proposed attribute values for the same utterances, utilizing both the English and the Hungarian ASR models. We found that the articulation rate and speech tempo values calculated based on the two ASR models were highly correlated, and so were the attributes corresponding to silent pauses; however, noticeable differences were found regarding the filled pauses (still, these attributes remained indicative for both languages). Our further analysis revealed that this is probably due to a difference regarding the annotation of the English and the Hungarian ASR training utterances.

Temporal Speech Parameters Indicate Early Cognitive Decline in Elderly Patients With Type 2 Diabetes Mellitus

“…This ASR model performed phone-level recognition, with labeling of the input signal (including filled pauses, treated as a special “phoneme”) and the output of a phonetic segmentation. Based on the raw parameters from the ASR output, 15 temporal speech parameters were extracted using simple calculations established in previous works of our research group 16,20. The calculations and definitions of the parameters are available as supplements (Supplemental Digital Content 1, http://links.lww.com/WAD/A379).…”

“…Pauses were defined as the interruption of speech by either complete silence (silent pause) or by filler words like “um” or “er” (filled pause) lasting longer than 30 ms. The acoustic model was trained on a subset of the BEA audio corpus 29 that consisted of spontaneous speech, as this type of speech is expected to contain filled pauses (for the training of the ASR system, see Gosztolya et al 20 ). For training, the speech of 116 speakers was utilized, which amounted to ~44 hours of recordings.…”

“…individuals with healthy cognition (HC), both for Hungarian [15][16][17][18][19] and for English native speakers. 20 The objective of the present study was (1) to explore whether elderly HC individuals with and without T2DM differ in temporal speech characteristics, which may reflect subtle differences in cognition as well; and (2) to also understand how the same temporal speech characteristics compare between MCI patients with and without T2DM.…”

“…In the present study, an automated speech analysis method, the Speech-Gap Test (S-GAP Test) was applied on speech recordings of T2DM participants. This method, built on automatic speech recognition (ASR) techniques, was sensitive to distinguish between MCI patients and elderly individuals with healthy cognition (HC), both for Hungarian15–19 and for English native speakers 20…”

“…This method, built on automatic speech recognition (ASR) techniques, was sensitive to distinguish between MCI patients and elderly individuals with healthy cognition (HC), both for Hungarian 15 – 19 and for English native speakers. 20 …”