Cross-cultural detection of depression from nonverbal behaviour

Abstract

Millions of people worldwide suffer from depression. Do commonalities exist in their nonverbal behavior that would enable cross-culturally viable screening and assessment of severity? We investigated the generalisability of an approach to detect depression severity cross-culturally using video-recorded clinical interviews from Australia, the USA and Germany. The material varied in type of interview, subtypes of depression and inclusion healthy control subjects, cultural background, and recording environment. The analysis focussed on temporal features of participants‘ eye gaze and head pose. Several approaches to training and testing within and between datasets were evaluated. The strongest results were found for training across all datasets and testing across datasets using leave-one-subject-out cross-validation. In contrast, generalisability was attenuated when training on only one or two of the three datasets and testing on subjects from the dataset(s) not used in training. These findings highlight the importance of using training data exhibiting the expected range of variability.

Quantitative comparison of motion history image variants for video-based depression assessment

“…They used the Local Gabor Binary Patterns in Three Orthogonal Planes, provided by the challenge organizers [10], achieving 82% accuracy. Similar results were obtained in the crosscultural study of Alghowinem et al [15] who focused on geometrical features derived from eye activity to achieve 81.3% classification accuracy utilizing one subset of the AVEC'13 dataset, among other datasets. Pampouchidou et al [16] reported 74.5% accuracy with the Local Curvelet Binary Patterns in Pairwise Orthogonal Planes, while Pampouchidou et al [17] employed geometrical features to achieve an F1 score of 58.6% in the single-modality (i.e., visual) approach and 72.8% when taking into account both audio and visual features.…”

A Linguistically-Informed Fusion Approach for Multimodal Depression Detection

“…Researchers have found early fusion, although simple, to be a successful technique to combine modalities for depression, noting improvements over unimodal systems (Alghowinem et al, 2015;Morales and Levitan, 2016;Morales et al, 2017b;Scherer et al, 2013b). However, a drawback of the early fusion approach is the high dimensionality of the combined feature vector.…”

“…However, it is a fairly new research interest and as a result only a few studies have compared techniques for fusing features from different modalities (Alghowinem et al, 2015). In the few studies that have investigated fusion techniques, the canonical fusion techniques have been considered, including early, late, and hybrid fusion.…”

“…In Alghowinem et al (2015) a hybrid fusion approach was investigated, which involved concatenating results from individual modalities to the the early fusion feature vector. A majority voting method was used.…”

“…Initial studies on depression detection from multimodal features have shown performance gains can be achieved by combining information from various modalities (Morales and Levitan, 2016;. However, few studies have investigated fusion approaches for depression detection (Alghowinem et al, 2015). In this paper, we present a novel linguistically motivated approach to fusion: syntaxinformed fusion.…”