Multimodal and Multiresolution Depression Detection from Speech and Facial Landmark Features

Nasir, .; Jati, Arindam; Shivakumar, Prashanth Gurunath; Chakravarthula, Sandeep Nallan; Georgiou, Panayiotis G.

doi:10.1145/2988257.2988261

Cited by 100 publications

(52 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The VL-Formants also outperformed the i-vector paradigm depression F1-scores on the development and test sets (0.57 and 0.48, as presented in [28]). Impressively, they match performance with the DepAudioNet system presented in [29].…”

Section: Resultsmentioning

confidence: 81%

See 1 more Smart Citation

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

et al. 2017

View full text Add to dashboard Cite

Whilst studies on emotion recognition show that genderdependent analysis can improve emotion classification performance, the potential differences in the manifestation of depression between male and female speech have yet to be fully explored. This paper presents a qualitative analysis of phonetically aligned acoustic features to highlight differences in the manifestation of depression. Gender-dependent analysis with phonetically aligned gender-dependent features are used for speech-based depression recognition. The presented experimental study reveals gender differences in the effect of depression on vowel-level features. Considering the experimental study, we also show that a small set of knowledge-driven gender-dependent vowel-level features can outperform state-of-the-art turn-level acoustic features when performing a binary depressed speech recognition task. A combination of these preselected gender-dependent vowel-level features with turn-level standardised openSMILE features results in additional improvement for depression recognition.

show abstract

Section: Resultsmentioning

confidence: 81%

“…These include: VTC features [27]; the i-vector paradigm [28]; and a deep neural network which combined both convolutional and Long Short Term Memory (LSTM) layers [29].…”

Section: Depression Corpusmentioning

confidence: 99%

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The second approach to modeling depression attempts to exploit global and/or time varying statistics, independent of the question that prompted the response. Williamson et al utilized correlations of formants and spectral information across different time scales [11], Syed et al developed audio and video features to capture temporal variations [12], while Pampouchidou et al and Nasir et al fused low and high-level features [13,14]. Utilizing emerging techniques, Ma et al used audio to model depression by allowing deep neural networks to learn such associations rather than perform feature engineering [15].…”

Section: Introductionmentioning

confidence: 99%

“…Their strengths lie in their ability to represent information through non-linear transforms, at varying spatial and temporal resolution, and from multiple modalities [17,18]. While work in the domain of detecting depression has looked at fusing features from multiple modalities together [9,13,14,19], and utilizing neural networks to model single sequences [10,15], there remains to explore the sequence modeling of depression that utilizes deep learning approaches.…”

Section: Introductionmentioning

confidence: 99%

Detecting Depression with Audio/Text Sequence Modeling of Interviews

Alhanai¹,

2018

View full text Add to dashboard Cite

Medical professionals diagnose depression by interpreting the responses of individuals to a variety of questions, probing lifestyle changes and ongoing thoughts. Like professionals, an effective automated agent must understand that responses to queries have varying prognostic value. In this study we demonstrate an automated depression-detection algorithm that models interviews between an individual and agent and learns from sequences of questions and answers without the need to perform explicit topic modeling of the content. We utilized data of 142 individuals undergoing depression screening, and modeled the interactions with audio and text features in a Long-Short Term Memory (LSTM) neural network model to detect depression. Our results were comparable to methods that explicitly modeled the topics of the questions and answers which suggests that depression can be detected through sequential modeling of an interaction, with minimal information on the structure of the interview.

show abstract

“…These challenges hint at the potential benefits of involving automatic behavior annotation, in which data-driven machine learning techniques are employed to automatically extract behavioral information directly from data, rather than relying on time-consuming and expensive annotations from human experts. Such behavior analysis work has been shown to be effective at identifying behaviors during interactions in domains such as couple therapy [7,8,9], depression [10,11,12] and suicide risk assessment [13,14,15,16]. However, due to potential domain mismatch, obtaining accurate performance in one domain by utilizing well-trained behavior analysis systems from a different domain is not straightforward.…”

Section: Introductionmentioning

confidence: 99%

Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions Using Speech and Language

Chakravarthula

Tseng

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Cancer impacts the quality of life of those diagnosed as well as their spouse caregivers, in addition to potentially influencing their day-to-day behaviors. There is evidence that effective communication between spouses can improve well-being related to cancer but it is difficult to efficiently evaluate the quality of daily life interactions using manual annotation frameworks. Automated recognition of behaviors based on the interaction cues of speakers can help analyze interactions in such couples and identify behaviors which are beneficial for effective communication. In this paper, we present and detail a dataset of dyadic interactions in 85 real-life cancer-afflicted couples and a set of observational behavior codes pertaining to interpersonal communication attributes. We describe and employ neural network-based systems for classifying these behaviors based on turn-level acoustic and lexical speech patterns. Furthermore, we investigate the effect of controlling for factors such as gender, patient/caregiver role and conversation content on behavior classification. Analysis of our preliminary results indicates the challenges in this task due to the nature of the targeted behaviors and suggests that techniques incorporating contextual processing might be better suited to tackle this problem.

show abstract

Multimodal and Multiresolution Depression Detection from Speech and Facial Landmark Features

Cited by 100 publications

References 38 publications

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

Implementing Gender-Dependent Vowel-Level Analysis for Boosting Speech-Based Depression Recognition

Detecting Depression with Audio/Text Sequence Modeling of Interviews

Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions Using Speech and Language

Contact Info

Product

Resources

About