Detecting Depression using Vocal, Facial and Semantic Communication Cues

Williamson, James R.; Godoy, Elizabeth; Cha, Miriam; Schwarzentruber, Adrianne; Khorrami, Pooya; Gwon, Youngjune; Kung, H. T.; Dagli, Charlie K.; Quatieri, Thomas F.

doi:10.1145/2988257.2988263

Cited by 151 publications

(74 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also outperformed the development set depression F1-score of 0.50 obtained using vocal tract correlation features [27], which are well known to capture depression information in speech [13,27] (Note that, the test set scores were not given in [27]). The VL-Formants also outperformed the i-vector paradigm depression F1-scores on the development and test sets (0.57 and 0.48, as presented in [28]).…”

Section: Resultsmentioning

confidence: 87%

See 1 more Smart Citation

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

et al. 2017

View full text Add to dashboard Cite

Whilst studies on emotion recognition show that genderdependent analysis can improve emotion classification performance, the potential differences in the manifestation of depression between male and female speech have yet to be fully explored. This paper presents a qualitative analysis of phonetically aligned acoustic features to highlight differences in the manifestation of depression. Gender-dependent analysis with phonetically aligned gender-dependent features are used for speech-based depression recognition. The presented experimental study reveals gender differences in the effect of depression on vowel-level features. Considering the experimental study, we also show that a small set of knowledge-driven gender-dependent vowel-level features can outperform state-of-the-art turn-level acoustic features when performing a binary depressed speech recognition task. A combination of these preselected gender-dependent vowel-level features with turn-level standardised openSMILE features results in additional improvement for depression recognition.

show abstract

Section: Resultsmentioning

confidence: 87%

“…These include: VTC features [27]; the i-vector paradigm [28]; and a deep neural network which combined both convolutional and Long Short Term Memory (LSTM) layers [29].…”

Section: Depression Corpusmentioning

confidence: 99%

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The cross-cultural and cross-linguistic characteristics in depressed speech using vocal biomarkers is studied in [1]. In [43], authors study the neurocognitive changes influencing the dialogue delivery and semantics. Semantic features are encoded using sparse lexical embedding space and context is drawn from subject's past clinical history.…”

Section: Depression Detection From Speechmentioning

confidence: 99%

Multi-level Attention Network using Text, Audio and Video for Depression Prediction

Ray

Kumar

Reddy

et al. 2019

Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop

View full text Add to dashboard Cite

Depression has been the leading cause of mental-health illness worldwide. Major depressive disorder (MDD), is a common mental health disorder that affects both psychologically as well as physically which could lead to loss of lives. Due to the lack of diagnostic tests and subjectivity involved in detecting depression, there is a growing interest in using behavioural cues to automate depression diagnosis and stage prediction. The absence of labelled behavioural datasets for such problems and the huge amount of variations possible in behaviour makes the problem more challenging. This paper presents a novel multi-level attention based network for multimodal depression prediction that fuses features from audio, video and text modalities while learning the intra and inter modality relevance. The multi-level attention reinforces overall learning by selecting the most influential features within each modality for the decision making. We perform exhaustive experimentation to create different regression models for audio, video and text modalities. Several fusions models with different configurations are constructed to understand the impact of each feature and modality. We outperform the current baseline by 17.52% in terms of root mean squared error.

show abstract

“…In [14] and [13], the text is analyzed on a subject level and audio/video features are separately extracted and then fused with semantic features, i.e., topic modeling is not used in these approaches. In [21], the authors conduct a question/answer extraction (which is similar to topic extraction) before text analysis. However, the question/answer extraction is only applied to text analysis.…”

Section: Related Workmentioning

confidence: 99%

Topic Modeling Based Multi-modal Depression Detection

Gong

Poellabauer

2017

Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge

105

View full text Add to dashboard Cite

Major depressive disorder is a common mental disorder that affects almost 7% of the adult U.S. population. e 2017 Audio/Visual Emotion Challenge (AVEC) asks participants to build a model to predict depression levels based on the audio, video, and text of an interview ranging between 7-33 minutes. Since averaging features over the entire interview will lose most temporal information, how to discover, capture, and preserve useful temporal details for such a long interview are significant challenges. erefore, we propose a novel topic modeling based approach to perform context-aware analysis of the recording. Our experiments show that the proposed approach outperforms context-unaware methods and the challenge baselines for all metrics.

show abstract

Detecting Depression using Vocal, Facial and Semantic Communication Cues

Cited by 151 publications

References 18 publications

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

Multi-level Attention Network using Text, Audio and Video for Depression Prediction

Topic Modeling Based Multi-modal Depression Detection

Contact Info

Product

Resources

About