Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Jan, Asim; Meng, Hongying; Gaus, Yona Falinie Binti A.; Zhang, Fan

doi:10.1109/tcds.2017.2721552

Cited by 158 publications

(85 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once again, the proposed method outperforms the methods proposed by Zhu et al [6], Jazaery et al [8] and Melo et al [7]. In [5], the authors proposed a deep learning model to explore facial frames and employed feature dynamic history histogram (FDHH) to capture variations in the features. Our method achieves better results in terms of MAE than the method in [5], while this later is better in terms of RMSE.…”

Section: Experimental Analysismentioning

confidence: 93%

“…In [5], the authors proposed a deep learning model to explore facial frames and employed feature dynamic history histogram (FDHH) to capture variations in the features. Our method achieves better results in terms of MAE than the method in [5], while this later is better in terms of RMSE. This indicates that the exploration of temporal redundancies can also contribute to depression detection.…”

Section: Experimental Analysismentioning

confidence: 99%

“…For the visual information, face and body carry important cues, such as facial expressions, eye blinks, and head pose and movement [4]. For example, Jan et al [5] combined features extracted from facial expressions using a deep learning model, and vocal expressions using regression techniques.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Depression Detection Based on Deep Distribution Learning

Melo

Granger

Hadid

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Major depressive disorder is among the most common and harmful mental health problems. Several deep learning architectures have been proposed for video-based detection of depression based on the facial expressions of subjects. To predict the depression level, these architectures are often modeled for regression with Euclidean loss. Consequently, they do not leverage the data distribution, nor explore the ordinal relationship between facial images and depression levels, and have limited robustness to noisy and uncertain labeling. This paper introduces a deep learning architecture for accurately predicting depression levels through distribution learning. It relies on a new expectation loss function that allows to estimate the underlying data distribution over depression levels, where expected values of the distribution are optimized to approach the ground-truth levels. The proposed approach can produce accurate predictions of depression levels even under label uncertainty. Extensive experiments on the AVEC2013 and AVEC2014 datasets indicate that the proposed architecture represents an effective approach that can outperform state-of-the-art techniques.

show abstract

Section: Experimental Analysismentioning

confidence: 93%

Section: Experimental Analysismentioning

confidence: 99%

See 1 more Smart Citation

Depression Detection Based on Deep Distribution Learning

Melo

Granger

Hadid

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…Deep learning models for video-based depression detection often cascade a 2D CNN with an RNN [17]. Most notably, Jan et al [38] propose deep learning techniques to extract features from facial frames and employ feature dynamic history histogram (FDHH) to capture variations in the features. In addition, other authors [10] proposed a two-channel CNN where one channel inputs full facial regions, whereas the second inputs facial flows, with two fully connected layers performing the fusion of the features.…”

Section: Related Workmentioning

confidence: 99%

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

Melo

Granger

Hadid

2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

Deep learning architectures have been successfully applied in video-based health monitoring, to recognize distinctive variations in the facial appearance of subjects. To detect patterns of variation linked to depressive behavior, deep neural networks (NNs) typically exploit spatial and temporal information separately by, e.g., cascading a 2D convolutional NN (CNN) with a recurrent NN (RNN), although the intrinsic spatio-temporal relationships can deteriorate. With the recent advent of 3D CNNs like the convolutional 3D (C3D) network, these spatio-temporal relationships can be modeled to improve performance. However, the accuracy of C3D networks remain an issue when applied to depression detection. In this paper, the fusion of diverse C3D predictions are proposed to improve accuracy, where spatio-temporal features are extracted from global (full-face) and local (eyes) regions of subject. This allows to increasingly focus on a local facial region that is highly relevant for analyzing depression. Additionally, the proposed network integrates 3D Global Average Pooling in order to efficiently summarize spatio-temporal features without using fully-connected layers, and thereby reduce the number of model parameters and potential over-fitting. Experimental results on the Audio Visual Emotion Challenge (AVEC 2013 and AVEC 2014) depression datasets indicates that combining the responses of global and local C3D networks achieves a higher level of accuracy than state-of-the-art systems.

show abstract

“…For example, flexible displays can be applied to smart electronics such as Internet of Things (IoT), augmented reality (AR), and wearable applications to exchange customized and bilateral information via visual communication protocols. The field of flexible electronics is being converged with artificial intelligence (AI) by providing users' biological and behavioral signals collected in wearable biodevices, offering intelligent services based on big data cloud computing and machine learning …”

Section: Introductionmentioning

confidence: 99%

Novel Electronics for Flexible and Neuromorphic Computing

Lee

Park

Kim

et al. 2018

Adv Funct Materials

109

View full text Add to dashboard Cite

REVIEWbeing converged with artificial intelligence (AI) by providing users' biological and behavioral signals collected in wearable biodevices, [21][22][23][24][25][26][27][28][29][30][31] offering intelligent services based on big data cloud computing and machine learning. [32][33][34][35][36][37][38][39][40] A number of research groups have demonstrated flexible memories, thin film transistors (TFTs), and integrated circuits (ICs) as key technology for data processing, information storage, and communication. [41][42][43][44][45][46][47][48][49] Since Kim et al. [50] reported functional flexible resistive random access memory (RRAM), various chalcogenidebased phase change memories (PCMs) [51][52][53][54] as well as RRAMs using inorganic (e.g., WO 3 , Al 2 O 3 , HfO 2 , TiO 2 ), [45,49,50,[55][56][57] carbon (graphene, carbon nanotubes (CNTs)), [58][59][60][61][62][63][64] and organic materials [65][66][67][68] have been fabricated on polymer films. In addition, an ultrathin TFT and Si-based large-scale integration (LSI) were demonstrated for high-density flexible electronics. [8,69,70] Beyond the front-end device level, flexible packaging has been developed to interconnect core and peripheral modules to realize fully operational system-on-plastic (SoP). [71][72][73][74][75] Neuromorphic computing systems (brain-inspired model of parallel neuron network) are considered as a promising technology for AI applications, overcoming the limitation of von Neumann architecture (serial and iterative processing) for intelligent data analysis and low power consumption. [76][77][78][79][80][81] With the rapid advancement in the electronics (e.g., memristor, PCM, and TFT) on plastics or any type of surface, [82][83][84][85][86][87][88][89][90] emulation of biological synapses (adaptive synaptic weight, [91][92][93][94] and spike-timing-dependent plasticity (STDP) [95][96][97][98][99] of neurons) is being demonstrated on flexible substrate, which realizes an era of merged electronics toward cognitive IoT, physiological sensor, wearable computer, and autonomous driving system. [92,100] Here, we present recent progress in the field of electronics for flexible and neuromorphic applications that can be classified into four main categories: i) various devices (e.g., resistive memory, PCM, TFT, and IC) on plastic for computing, ii) flexible electronic systems using large-scale interconnection and packaging, iii) electronics for neuromorphic engineering, and iv) promising research areas of flexible synaptic applications. In addition, we have organized the main features of the studies in each section as a table to clearly compare their performance and challenges. The new electronic concept of a flexible and Emerging classes of flexible electronic systems that can be attached to a wide range of surfaces from wearable clothes to internal organs have driven significant advances in communication protocols (e.g., Internet of Things, augmented reality) and clinical research, shifting today's personal computing paradigm. The field of "system ...

show abstract

Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions

Cited by 158 publications

References 55 publications

Depression Detection Based on Deep Distribution Learning

Depression Detection Based on Deep Distribution Learning

Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions

Novel Electronics for Flexible and Neuromorphic Computing

Contact Info

Product

Resources

About