Automated depression analysis using convolutional neural networks from speech

Abstract

To help clinicians to efficiently diagnose the severity of a person's depression, the affective computing community and the artificial intelligence field have shown a growing interest in designing automated systems. The speech features have useful information for the diagnosis of depression. However, manually designing and domain knowledge are still important for the selection of the feature, which makes the process labor consuming and subjective. In recent years, deep-learned features based on neural networks have shown superior performance to hand-crafted features in various areas. In this paper, to overcome the difficulties mentioned above, we propose a combination of hand-crafted and deep-learned features which can effectively measure the severity of depression from speech. In the proposed method, Deep Convolutional Neural Networks (DCNN) are firstly built to learn deep-learned features from spectrograms and raw speech waveforms. Then we manually extract the state-of-the-art texture descriptors named median robust extended local binary patterns (MRELBP) from spectrograms. To capture the complementary information within the hand-crafted features and deep-learned features, we propose joint fine-tuning layers to combine the raw and spectrogram DCNN to boost the depression recognition performance. Moreover, to address the problems with small samples, a data augmentation method was proposed. Experiments conducted on AVEC2013 and AVEC2014 depression databases show that our approach is robust and effective for the diagnosis of depression when compared to state-of-the-art audio-based methods.

Automatic Depression Level Detection via ℓp-Norm Pooling

“…They divided the speech into many fixed length segments, then 2268 features were extracted from these segments by open-source Emotion and Affect Recognition (openEAR) toolkit [19]. However, only using these hand-crafted low-level features might lost other information associated with depression [12]. In addition, they aggregated the features extracted from segments to generate the representation of the speech through average-pooling, which is a special case of p-norm pooling [20] and not necessarily optimal for the depression detection.…”

“…However, the number of Gaussian components was not adapted to the depression detection task, which affected the accuracy of prediction [22]. He et al [12] proposed a four-stream CNN to detect an individual depression level. Although, CNN is good at capturing spatial structure [23], it can not well explore the impact of temporal changes on depression detection.…”

“…However, there are some limitations in previous methods. Firstly, most of them [8,10] predict individual depression level through hand-crafted features, which lose some useful patterns related to depression [12]. Secondly, some methods [13,14] only use spatial structure or temporal changes to describe the spectrograms and MFCC.…”

Deep learning-based automated speech detection as a marker of social functioning in late-life depression

“…Most of the studies to date measure speech in controlled settings (e.g. recording participants reading passages aloud in quiet rooms) and focus on detecting specific features of speech (He & Cao, 2018;Jiang et al, 2018;Li et al, 2018). An alternative approach would be to use wearable devices to objectively detect how much speech participants encounter and produce in their natural environment.…”

Learning Voice Source Related Information for Depression Detection

“…e-mail: (see http://www.idiap.ch/en/people/directory). have used statistics of features, called low level descriptors (LLD), that are related to both the vocal-source and vocal-tract to improve the systems [12,4,14]; however not all the statistical properties contribute to the improvements. Despite these advances, there seem to be no concurred set of features for detecting depression from speech signals; and moreover, the performances of all these systems may be limited by the choice of features and their statistics.…”