Mediating individual affective experience through the emotional photo frame

Abstract

A photograph is considered a medium with emotional legibility and a means of expressing and exchanging emotional experience. This research presents the interactive emotional photo frame system focusing on mediating individual affective experience among close relationships. This emotional photo frame system dynamically controls the photograph based on a user's affective states and user-specified emotion reaction rules. It is designed for helping users recognize and experience the same emotions with others. The system consists of an emotion recognition system, an emotion share server and an emotional digital picture frame. The emotion recognition system analyzes photoplethysmography, skin temperature and galvanic skin response signals to recognize user's emotional states and transmits the results to the server in real-time. The server stores user emotional data with individual emotion reaction rules that define how the picture frame should affect the photo in response to individual affective states. The emotional picture frame renders personalized visual and aural elements on the photograph according to the feedback of emotions. This paper presents an empirical exploration of the effectiveness of this system. The results revealed that most participants were influenced by their partner's emotion presented in this system. There was a strong relation of emotion sharing between the partners.

Real-Time Mobile Emotional Content Player Using Smartphone Camera-Based PPG Measurement

“…ECG, PPG, and skin temperature (SKT) signals are mainly used in automatic nervous system responses. A study was conducted on acquiring emotion information in real time by measuring the biological signals of a PPG, SKT, and galvanic skin response (GSR), and a real-time emotion information sharing system was developed [10][11][12][13][20][21][22][23][24][25][26][27][28][29][30][31][32]. However, to measure ECG signals, a sensor has to be attached close to the heart, and accordingly, there are disadvantages in that the user may feel repulsion and experience a limitation in their activities [32].…”

“…For example, one study reported that fear-related content induces the emotional responses of fear, arousal, and displeasure, and relaxationrelated content induces the emotional response of relaxation [1][2][3][4][5][6][7][8][9]. Currently, however, users want to use multimedia content provided according to their circumstances or emotional state rather than uniform content that does not consider their emotional responses [10,11]. In other words, recent studies have focused on methods to induce the emotional responses of users and reflect those responses within the content, just as the content can induce emotional responses in the users.…”

The Design of CNN Architectures for Optimal Six Basic Emotion Classification Using Multiple Physiological Signals

“…There have also been studies using deep-learning methods, such as convolutional neural networks (CNNs) [17,18], Deep Belief Network (DBN) [19], and Sparse AE [20], or models integrating machine learning and deep learning [21,22]. In addition to the simple categorization of emotions, studies that apply classified models to various fields have been actively conducted [23][24][25][26]. Even the conventional classifiers used in the past are not inferior in performance to state-of-the-art classifiers, and the results may vary depending on the classifier parameters and the characteristics of input data.…”