An End-to-End Model for Detection and Assessment of Depression Levels using Speech

Srimadhur, N.S.; Lalitha, S.

doi:10.1016/j.procs.2020.04.003

Cited by 34 publications

(19 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We show that the privacy-preserving FL models perform robustly with only 4-6% accuracy lost compared to a centralized approach. More importantly, these models achieve better accuracy than the best-performing models in prior work using DAIC-WOZ (e.g., 87% (FL) compared to 74.64% [8]). These findings establish the feasibility of using privacy-preserving FL models for depression assessment.…”

Section: Discussionmentioning

confidence: 83%

“…• The FL models achieve significantly better accuracy compared to the best-performing models in prior work using the DAIC-WOZ dataset (e.g., 87% accuracy for the combined dataset compared to 74.64% in [8]).…”

Section: Introductionmentioning

confidence: 84%

“…Hanai et al [12] developed an LSTM model using audio features with an accuracy of 0.59. Srimadhur et al [8] used an end-to-end network for classification from audio data with an accuracy of 74.64%. However, these previous studies do not focus on privacy-preserving audio analysis.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression

BN,

Abdullah

2022

Preprint

View full text Add to dashboard Cite

Recent studies have used speech signals to assess depression. However, speech features can lead to serious privacy concerns. To address these concerns, prior work has used privacy-preserving speech features. However, using a subset of features can lead to information loss and, consequently, non-optimal model performance. Furthermore, prior work relies on a centralized approach to support continuous model updates, posing privacy risks. This paper proposes to use Federated Learning (FL) to enable decentralized, privacypreserving speech analysis to assess depression. Using an existing dataset (DAIC-WOZ), we show that FL models enable a robust assessment of depression with only 4-6% accuracy loss compared to a centralized approach. These models also outperform prior work using the same dataset. Furthermore, the FL models have short inference latency and small memory footprints while being energy-efficient. These models, thus, can be deployed on mobile devices for real-time, continuous, and privacy-preserving depression assessment at scale.

show abstract

Section: Discussionmentioning

confidence: 83%

Section: Introductionmentioning

confidence: 84%

See 1 more Smart Citation

Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression

BN,

Abdullah

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Acoustic features of speech can be predictive of distress irrespective of the speech content [38], [39]. For example, Ozdas et al [38] assessed the risk of suicide by detecting the fluctuations in the fundamental frequency of people's speech.…”

Section: Audio Modalitymentioning

confidence: 99%

Looking at the Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Lin

Orton

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Psychological distress is a significant and growing issue in society. In particular, depression and anxiety are leading causes of disability that often go undetected or late-diagnosed. Automatic detection, assessment, and analysis of behavioural markers of psychological distress can help improve identification and support prevention and early intervention efforts. Compared to modalities such as face, head, and vocal, research investigating the use of the body modality for these tasks is relatively sparse, which is partly due to the limited available datasets and difficulty in automatically extracting useful body features. To enable our research, we have collected and analyzed a new dataset containing full body videos for interviews and self-reported distress labels. We propose a novel approach to automatically detect self-adaptors and fidgeting, a subset of self-adaptors that has been shown to correlate with psychological distress. We perform analysis on statistical body gestures and fidgeting features to explore how distress levels affect behaviors. We then propose a multi-modal approach that combines different feature representations using Multi-modal Deep Denoising Auto-Encoders and Improved Fisher Vector Encoding. We demonstrate that our proposed model, combining audio-visual features with detected fidgeting behavioral cues, can successfully predict depression and anxiety in the dataset.

show abstract

“…Acoustic features of speech can be predictive of distress irrespective of the speech content [29], [30]. For example, Ozdas et al [29] assessed the risk of suicide by detecting the fluctuations in the fundamental frequency of people's speech.…”

Section: Audio Modalitymentioning

confidence: 99%

Looking At The Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Lin

Orton

et al. 2020

Preprint

View full text Add to dashboard Cite

Psychological distress is a significant and growing issue in society. Automatic detection, assessment, and analysis of such distress is an active area of research. Compared to modalities such as face, head, and vocal, research investigating the use of the body modality for these tasks is relatively sparse. This is, in part, due to the limited available datasets and difficulty in automatically extracting useful body features. Recent advances in pose estimation and deep learning have enabled new approaches to this modality and domain. To enable this research, we have collected and analyzed a new dataset containing full body videos for short interviews and self-reported distress labels. We propose a novel method to automatically detect self-adaptors and fidgeting, a subset of self-adaptors that has been shown to be correlated with psychological distress. We perform analysis on statistical body gestures and fidgeting features to explore how distress levels affect participants' behaviors. We then propose a multi-modal approach that combines different feature representations using Multi-modal Deep Denoising Auto-Encoders and Improved Fisher Vector Encoding. We demonstrate that our proposed model, combining audio-visual features with automatically detected fidgeting behavioral cues, can successfully predict distress levels in a dataset labeled with self-reported anxiety and depression levels.

show abstract

An End-to-End Model for Detection and Assessment of Depression Levels using Speech

Cited by 34 publications

References 14 publications

Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression

Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression

Looking at the Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Looking At The Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

Contact Info

Product

Resources

About