Realtime Emotional Reflective User Interface Based on Deep Convolutional Neural Networks and Generative Adversarial Networks

Abstract

It is becoming increasingly apparent that a significant amount of the population suffers from mental health problems, such as stress, depression, and anxiety. These issues are a result of a vast range of factors, such as genetic conditions, social circumstances, and lifestyle influences. A key cause, or contributor, for many people is their work; poor mental state can be exacerbated by jobs and a person’s working environment. Additionally, as the information age continues to burgeon, people are increasingly sedentary in their working lives, spending more of their days seated, and less time moving around. It is a well-known fact that a decrease in physical activity is detrimental to mental well-being. Therefore, the need for innovative research and development to combat negativity early is required. Implementing solutions using Artificial Intelligence has great potential in this field of research. This work proposes a solution to this problem domain, utilising two concepts of Artificial Intelligence, namely, Convolutional Neural Networks and Generative Adversarial Networks. A CNN is trained to accurately predict when an individual is experiencing negative emotions, achieving a top accuracy of 80.38% with a loss of 0.42. A GAN is trained to synthesise images from an input domain that can be attributed to evoking position emotions. A Graphical User Interface is created to display the generated media to users in order to boost mood and reduce feelings of stress. The work demonstrates the capability for using Deep Learning to identify stress and negative mood, and the strategies that can be implemented to reduce them.

A Lightweight Facial Emotion Recognition System Using Partial Transfer Learning for Visually Impaired People

“…Burrows, et al [7] propose three different CNN and Generative Adversarial Networks (GANs) based models that classify facial expressions.The authors created a combined dataset from six different datasets to have around 41K images used for testing and training. The first model, classifies the images into seven emotions; sad, fear, happy, angry, neutral, disgust, and surprise.…”

“…However, such solutions may be unsuitable for VIP. Most of the current FER systems proposed in the literature were implemented on desktop [3], [7], [13], [17], [19], [23], [30], [31] or other powerful devices [29]. Furthermore, some of them produce a huge overhead [13], [19], [26], [31], [34].…”

“…These challenges pose a great limitations on the model predictive capability [8], [11]. Some works in the literature [3], [7], [23], solved this problem by minimizing the number of classes or combining these classes under one class. Some samples from EFER2013 and CK+ datasets with the actual and predicted labels are shown in Figure 11.…”

Social Media Posts as a Window into Mental Health: A Machine Learning Approach

“…Another study employed a combination of natural language processing and machine learning techniques to classify depression, anxiety, and stress in social media users, achieving an accuracy of 72.5% [16]. Other studies have employed decision tree algorithms [39] convolutional neural networks [40][41], and deep learning approaches [42] [43] to classify mental health conditions in social media data. Other studies have focused on predicting suicidal ideation on social media platforms.…”

Anthropologically oriented strategies of interaction in the Human-Computer system

“…Burrows et al consider the issue of a mental model of HCI based on deep learning using convolutional neural networks and generative adversarial networks, offer solutions for solving the problems of stress and negative mood detection, defining strategies to counteract these negative phenomena [6]. The authors update attention to the problems of emotional intelligence, realtime design of HCI.…”