Fully automated assessment of the severity of Parkinson's disease from speech

Abstract

For several decades now, there has been sporadic interest in automatically characterizing the speech impairment due to Parkinson’s disease (PD). Most early studies were confined to quantifying a few speech features that were easy to compute. More recent studies have adopted a machine learning approach where a large number of potential features are extracted and the models are learned automatically from the data. In the same vein, here we characterize the disease using a relatively large cohort of 168 subjects, collected from multiple (three) clinics. We elicited speech using three tasks – the sustained phonation task, the diadochokinetic task and a reading task, all within a time budget of 4 minutes, prompted by a portable device. From these recordings, we extracted 1582 features for each subject using openSMILE, a standard feature extraction tool. We compared the effectiveness of three strategies for learning a regularized regression and find that ridge regression performs better than lasso and support vector regression for our task. We refine the feature extraction to capture pitch-related cues, including jitter and shimmer, more accurately using a time-varying harmonic model of speech. Our results show that the severity of the disease can be inferred from speech with a mean absolute error of about 5.5, explaining 61% of the variance and consistently well-above chance across all clinics. Of the three speech elicitation tasks, we find that the reading task is significantly better at capturing cues than diadochokinetic or sustained phonation task. In all, we have demonstrated that the data collection and inference can be fully automated, and the results show that speech-based assessment has promising practical application in PD. The techniques reported here are more widely applicable to other paralinguistic tasks in clinical domain.

Automatic measurement of affective valence and arousal in speech

“…Per-utterance features are based on F0 jitter, shimmer, harmonic to noise ratio (HNR), and the energy ratio of the first and second harmonics (H12). In previous work, we found that these features detect voiced segments and estimate F0 more accurately than other algorithms and that they are useful in rating the severity of a subjects Parkinsons disease [16]. Jitter and shimmer (and, of course, F0) have been used previously for emotion detection (e.g., [17]); however, few if any studies on emotion detection have used the harmonic model or have used it for estimating F0, jitter, or shimmer.…”

Inferring clinical depression from speech and spoken utterances

“…In previous work, we found that these features detect voiced segments and estimate pitch frequency more accurately than other algorithms and that they are useful in rating the severity of subjects' Parkinsons dis ease [25]. work [23].…”

Inferring social contexts from audio recordings using deep neural networks

“…Moreover, given the estimate of unknown parameters, we reconstruct noise-free signal and compute other pitch-related features such as Harmonic-to-Noise Ratio (HNR), shimmer, and jitter. We refer the reader to our recent works [16] [17] for more detail on extracting aforementioned features from harmonic model.…”