Decoding depressive disorder using computer vision

Singh, Jaiteg; Goyal, Gaurav

doi:10.1007/s11042-020-10128-9

Cited by 15 publications

(4 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data from 50 subjects were used to train the model and the result showed that their framework improved depression prediction performance with 80%, 78% and 72% accuracies on the EGG, speech, and facial data, respectively. Singh and Goyal [ 24 ] attempted to decode depressive disorder using computer vision. A questionnaire [ 33 ] on Attention Deficit Hyperactivity Disorder (ADHD) was administered to a total of 401 volunteers.…”

Section: Related Workmentioning

confidence: 99%

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

Aina,

Akinniyi,

Rahman

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Mental illness has grown to become a prevalent and global health concern that affects individuals across various demographics. Timely detection and accurate diagnosis of mental disorders are crucial for effective treatment and support as late diagnosis could result in suicidal, harmful behaviors and ultimately death. To this end, the present study introduces a novel pipeline for the analysis of facial expressions, leveraging both the AffectNet and 2013 Facial Emotion Recognition (FER) datasets. Consequently, this research goes beyond traditional diagnostic methods by contributing a system capable of generating a comprehensive mental disorder dataset and concurrently predicting mental disorders based on facial emotional cues. Particularly, we introduce a hybrid architecture for mental disorder detection leveraging the state-of-the-art object detection algorithm, YOLOv8 to detect and classify visual cues associated with specific mental disorders. To achieve accurate predictions, an integrated learning architecture based on the fusion of Convolution Neural Networks (CNNs) and Visual Transformer (ViT) models is developed to form an ensemble classifier that predicts the presence of mental illness (e.g., depression, anxiety, and other mental disorder). The overall accuracy is improved to about 81% using the proposed ensemble technique. To ensure transparency and interpretability, we integrate techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) and saliency maps to highlight the regions in the input image that significantly contribute to the model’s predictions thus providing healthcare professionals with a clear understanding of the features influencing the system’s decisions thereby enhancing trust and more informed diagnostic process.

show abstract

Section: Related Workmentioning

confidence: 99%

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

Aina,

Akinniyi,

Rahman

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In addition to being used for the education domain, AEMS can also be used in many other domains like entertainment (Wang, S., & Ji,Q. 2015), healthcare (Singh & Goyal, 2021), shopping (Yolcu et al, 2020) and more. Since AEMS can be used in various fields, each field needs to redesign a different set of contextual features according to the engagement dimensions to obtain better predictions.…”

Section: Significance Of Aemsmentioning

confidence: 99%

An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition

Pabba

Kumar

2021

Expert Systems

View full text Add to dashboard Cite

Students' disengagement problem has become critical in the modern scenario due to various distractions and lack of student-teacher interactions. This problem is exacerbated with large offline classrooms, where it becomes challenging for teachers to monitor students' engagement and maintain the right-level of interactions. Traditional ways of monitoring students' engagement rely on self-reporting or using physical devices, which have limitations for offline classroom use. Student's academic affective states (e.g., moods and emotions) analysis has potential for creating intelligent classrooms, which can autonomously monitor and analyse students' engagement and behaviours in real-time. In recent literature, a few computer vision based methods have been proposed, but they either work only in the e-learning domain or have limitations in real-time processing and scalability for large offline classes. This paper presents a real-time system for student group engagement monitoring by analysing their facial expressions and recognizing academic affective states: 'boredom,' 'confuse,' 'focus,' 'frustrated,' 'yawning,' and 'sleepy,' which are pertinent in the learning environment. The methodology includes certain pre-processing steps like face detection, a convolutional neural network (CNN) based facial expression recognition model, and post-processing steps like frame-wise group engagement estimation. For training the CNN model, we created a dataset of the aforementioned facial expressions from classroom lecture videos and added related samples from three publicly available datasets, BAUM-1, DAiSEE, and YawDD, to generalize the model predictions. The trained model has achieved train and test accuracy of 78.70% and 76.90%, respectively. The proposed methodology gave promising results when compared with self-reported engagement levels by students.

show abstract

“…• Speech: para-verbals features-e.g., speed, silences, pauses [3,5,[8][9][10]-as well as non-verbal features [17][18][19][20] in read and spontaneous speeches; • Handwriting and drawing [4,6,8] mainly focusing on the shape of the drawn lines; • Video analysis: face expressions [21], eye movements [22]; • Content of written and spoken words [23][24][25][26]; • Electroencephalogram (EEG) [27][28][29][30][31]; • Multimodality: more than a source of data is used and combined to improve detection performance [32,33].…”

Section: Related Workmentioning

confidence: 99%

A Privacy-Oriented Approach for Depression Signs Detection Based on Speech Analysis

et al. 2021

View full text Add to dashboard Cite

Currently, AI-based assistive technologies, particularly those involving sensitive data, such as systems for detecting mental illness and emotional disorders, are full of confidentiality, integrity, and security compromises. In the aforesaid context, this work proposes an algorithm for detecting depressive states based on only three never utilized speech markers. This reduced number of markers offers a valuable protection of personal (sensitive) data by not allowing for the retrieval of the speaker’s identity. The proposed speech markers are derived from the analysis of pitch variations measured in speech data obtained through a tale reading task performed by typical and depressed subjects. A sample of 22 subjects (11 depressed and 11 healthy, according to both psychiatric diagnosis and BDI classification) were involved. The reading wave files were listened to and split into a sequence of intervals, each lasting two seconds. For each subject’s reading and each reading interval, the average pitch, the pitch variation (T), the average pitch variation (A), and the inversion percentage (also called the oscillation percentage O) were automatically computed. The values of the triplet (Ti, Ai, Oi) for the i-th subject provide, all together, a 100% correct discrimination between the speech produced by typical and depressed individuals, while requiring a very low computational cost and offering a valuable protection of personal data.

show abstract

Decoding depressive disorder using computer vision

Cited by 15 publications

References 82 publications

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

A Hybrid Learning-Architecture for Mental Disorder Detection Using Emotion Recognition

An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition

A Privacy-Oriented Approach for Depression Signs Detection Based on Speech Analysis

Contact Info

Product

Resources

About