Acoustic Feature Extraction with Interpretable Deep Neural Network for Neurodegenerative Related Disorder Classification

Abstract

Speech-based automatic approaches for detecting neurodegenerative disorders (ND) and mild cognitive impairment (MCI) have received more attention recently due to being non-invasive and potentially more sensitive than current pen-and-paper tests. The performance of such systems is highly dependent on the choice of features in the classification pipeline. In particular for acoustic features, arriving at a consensus for a best feature set has proven challenging. This paper explores using deep neural network for extracting features directly from the speech signal as a solution to this. Compared with hand-crafted features, more information is present in the raw waveform, but the feature extraction process becomes more complex and less interpretable which is often undesirable in medical domains. Using a Sinc-Net as a first layer allows for some analysis of learned features. We propose and evaluate the Sinc-CLA (with SincNet, Convolutional, Long Short-Term Memory and Attention layers) as a task-driven acoustic feature extractor for classifying MCI, ND and healthy controls (HC). Experiments are carried out on an inhouse dataset. Compared with the popular hand-crafted feature sets, the learned task-driven features achieve a superior classification accuracy. The filters of the SincNet is inspected and acoustic differences between HC, MCI and ND are found.

An Interpretable Deep Learning Model for Automatic Sound Classification

“…For example, it has been shown that the first layers of end-to-end convolutional neural networks that learn representations from raw audio data extract features that are similar to the spectrogram or energies in mel-frequency bands [53][54][55]. Additionally, some works have addressed the design of the first layers of these networks to tailor the feature extraction stage using parametric filters [56][57][58] or trainable hand-crafted kernels [59,60]. Attention mechanisms have been used to bring interpretability to neural networks in speech and music emotion recognition [61,62] and in music auto-tagging [63].…”

Multi-Task Estimation of Age and Cognitive Decline from Speech

“…Previous studies [16] have shown that Sinc-CLA architecture has a good performance and interpretability in classifying recordings from people living with mild cognitive impairment, neurodegenerative disorders, or healthy controls. The multi-task Sinc-CLA system introduced in this paper is shown in Figure 3.…”

“…The SincNet Layer and CNN layers are shared by the two tasks, but the bi-directional LSTM and its following layers are separately trained with a specific target (age or MMSE). The detailed description of each functional layers can be found in Section 3.4 of this paper and in [16].…”

“…For the pipeline system, speaker embeddings like x-vector or i-vectors have achieved excellent performances for both age or MMSE estimation. On the other hand, the efficiency and interpretability of an end-to-end system named Sinc-CLA for neurodegenerative related disorder classification has been demostrated in [16]. In this paper, both of the two mainstream structures are demonstrated to be efficient for multi-task learning.…”

“…It has been found that in addition to paralinguistic acoustic feature sets [13], x-vector and ivector are also efficient for pathological speech detection [14,15]. In addition, inspired by the outstanding performance of deep neural networks used in numerous speech-based research areas, [16] proposed using a deep neural network for task-specific feature extractor learning and achieved superior performance.…”