Classifying Major Depressive Disorder and Response to Deep Brain Stimulation Over Time by Analyzing Facial Expressions

Jiang, Zifan; Harati, Sahar; Crowell, Andrea; Mayberg, Helen S.; Nemati, Shamim; Clifford, Gari D.

doi:10.1109/tbme.2020.3010472

Cited by 38 publications

(21 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although these tools have been shown utility for screening for MCI [5], it is challenging to scale these types of assessments, mainly because they need to be administered by trained personnel. In recent work we have studied passive activity of the face from webcams on consumer hardware in a psychiatric population [6]. Here we extend that work to study the nature of emotional expression in cognitively impaired subjects.…”

Section: Introductionmentioning

confidence: 82%

Automated analysis of facial emotions in subjects with cognitive impairment

et al. 2022

Self Cite

View full text Add to dashboard Cite

Differences in expressing facial emotions are broadly observed in people with cognitive impairment. However, these differences have been difficult to objectively quantify and systematically evaluate among people with cognitive impairment across disease etiologies and severity. Therefore, a computer vision-based deep learning model for facial emotion recognition trained on 400.000 faces was utilized to analyze facial emotions expressed during a passive viewing memory test. In addition, this study was conducted on a large number of individuals (n = 493), including healthy controls and individuals with cognitive impairment due to diverse underlying etiologies and across different disease stages. Diagnoses included subjective cognitive impairment, Mild Cognitive Impairment (MCI) due to AD, MCI due to other etiologies, dementia due to Alzheimer’s diseases (AD), and dementia due to other etiologies (e.g., Vascular Dementia, Frontotemporal Dementia, Lewy Body Dementia, etc.). The Montreal Cognitive Assessment (MoCA) was used to evaluate cognitive performance across all participants. A participant with a score of less than or equal to 24 was considered cognitively impaired (CI). Compared to cognitively unimpaired (CU) participants, CI participants expressed significantly less positive emotions, more negative emotions, and higher facial expressiveness during the test. In addition, classification analysis revealed that facial emotions expressed during the test allowed effective differentiation of CI from CU participants, largely independent of sex, race, age, education level, mood, and eye movements (derived from an eye-tracking-based digital biomarker for cognitive impairment). No screening methods reliably differentiated the underlying etiology of the cognitive impairment. The findings provide quantitative and comprehensive evidence that the expression of facial emotions is significantly different in people with cognitive impairment, and suggests this may be a useful tool for passive screening of cognitive impairment.

show abstract

Section: Introductionmentioning

confidence: 82%

Automated analysis of facial emotions in subjects with cognitive impairment

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…So, the existing prediction system takes the input sample from either in the form of online available datasets as discussed in the next section or data that needs to be captured through the device. The author's [10], [14], [15] and [16] proposed the technique where online datasets need to be loaded in the system whereas [6], [17], [18] and [19] used the data which were generated through the capturing device at stage one for the prediction based approach. The author's [6], [17], [18] and [19] uses the concept where the certain types of questions -answers, showing movies / videos /audios and the regular activities recorded through the camera then the frames were extracted.…”

Section: Figure 3 Block Diagram Of Prediction Systemmentioning

confidence: 99%

“…The author's [10], [14], [15] and [16] proposed the technique where online datasets need to be loaded in the system whereas [6], [17], [18] and [19] used the data which were generated through the capturing device at stage one for the prediction based approach. The author's [6], [17], [18] and [19] uses the concept where the certain types of questions -answers, showing movies / videos /audios and the regular activities recorded through the camera then the frames were extracted. After extracting the data these data need to be process hence it is given to the preprocessing block where data cleaning, data integration, data reduction, and data transformation will take place.…”

Section: Figure 3 Block Diagram Of Prediction Systemmentioning

confidence: 99%

“…This section has stated different approaches of predicting mental disorders with the available online datasets and along with the use of capturing devices which were generated by conducting the interviews / observing the activities etc. The dataset has been approved by Emory University Institutional Review Board which has been generated by authors [17] and proposed the novel approach for predicting the depression by evaluating 365 videos from a group of 12 depressed patients with before and after deep brain stimulation (DBS) treatment. Using the Convolution Neural Network, seven different emotions were extracted from the video frame and segmentation, classification was performed.…”

Section: Analysis Of Prediction Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Study of Mental Disorder Prediction Model approaches its issues and Challenges

Bhuva¹,

Mishra²

2021

JUSST

View full text Add to dashboard Cite

Mental disorder is becoming one of the major health issues in society today. (WHO) depression will be the leading mental disorder all over the world by 2030. The timely prediction of such disorders is very essential for maintaining the health of human beings. These mental disorders are associated with different symptoms, some of these symptoms are visible in the form of facial expressions, gestures, change in voice modulation, etc. and some of them even cannot be noticed by naked eyes. Therefore, it is highly important to collect appropriate verbal and non-verbal symptom details about any individual along with using best-suited algorithms for accurate prediction using information technology. This paper explores different types of such symptoms associated with different types of mental disorders, their causes, and existing prediction-based solutions. The paper further presents the critical analysis of these aspects and proposes usages of biometrics-based traits for building better prediction systems for mental disorders.

show abstract

“…Temporal information, on the other hand, can describe relevant dynamic information among consecutive frames. Jiang et al [25] proposed a novel deep neural network framework to assess major depressive disorder. Zhou et al [26] proposed an end-to-end pain intensity regression framework based on AAM-warped facial images and recurrent convolutional neural networks to predict pain intensity.…”

Section: Introductionmentioning

confidence: 99%

A Personalized Spatial-Temporal Cold Pain Intensity Estimation Model Based on Facial Expression

Guo

Wang

Xiao

et al. 2021

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

Objective: Pain assessment is of great importance in both clinical research and patient care. Facial expression analysis is becoming a key part of pain detection because it is convenient, automatic, and real-time. The aim of this study is to present a cold pain intensity estimation experiment, investigate the importance of the spatial-temporal information on facial expression based cold pain, and study the performance of the personalized model as well as the generalized model. Methods: A cold pain experiment was carried out and facial expressions from 29 subjects were extracted. Three different architectures (Inception V3, VGG-LSTM, and Convolutional LSTM) were used to estimate three intensities of cold pain: No pain, Moderate pain, and Severe Pain. Architectures with Sequential information were compared with single-frame architecture, showing the importance of spatial-temporal information on pain estimation. The performances of the personalized model and the generalized model were also compared. Results: A mean F1 score of 79.48% was achieved using Convolutional LSTM based on the personalized model. Conclusion: This study demonstrates the potential for the estimation of cold pain intensity from facial expression analysis and shows that the personalized spatial-temporal framework has better performance in cold pain intensity estimation. Significance: This cold pain intensity estimator could allow convenient, automatic, and real-time use to provide continuous objective pain intensity estimations of subjects and patients.

show abstract

Classifying Major Depressive Disorder and Response to Deep Brain Stimulation Over Time by Analyzing Facial Expressions

Cited by 38 publications

References 39 publications

Automated analysis of facial emotions in subjects with cognitive impairment

Automated analysis of facial emotions in subjects with cognitive impairment

Study of Mental Disorder Prediction Model approaches its issues and Challenges

A Personalized Spatial-Temporal Cold Pain Intensity Estimation Model Based on Facial Expression

Contact Info

Product

Resources

About