A Minimalist Approach for Identifying Affective States for Mobile Interaction Design

Tikadar, Subrata; Kazipeta, Sharath; Ganji, Chandrakanth; Bhattacharya, Surajit

doi:10.1007/978-3-319-67744-6_1

Cited by 6 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Table 1, we compare different touch-based emotion detection explorations using smartphone in terms of (a) type of touch interaction (b) usage of additional sensors (c) dependency on specific application and (d) number of detected emotion states. Gao et al (2012); Tikadar et al (2017) explored touch interactions during gameplay to detect emotion states. Lee et al (2012) used sensor details and typing details in a specific application (Twitter) to determine emotion.…”

Section: Related Workmentioning

confidence: 99%

Emotion detection from touch interactions during text entry on smartphones

Ghosh

Hiware

Ganguly

et al. 2019

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

There are different modes of interaction with a software keyboard on a smartphone, such as typing and swyping. Patterns of such touch interactions on a keyboard may reflect emotions of a user. Since users may switch between different touch modalities while using a keyboard, therefore, automatic detection of emotion from touch patterns must consider both modalities in combination to detect the pattern. In this paper, we focus on identifying different features of touch interactions with a smartphone keyboard that lead to a personalized model for inferring user emotion. Since distinguishing typing and swyping activity is important to record the correct features, we designed a technique to correctly identify the modality. The ground truth labels for user emotion are collected directly from the user by periodically collecting self-reports. We jointly model typing and swyping features and correlate them with user provided self-reports to build a personalized machine learning model, which detects four emotion states (happy, sad, stressed, relaxed). We combine these design choices into an Android application TouchSense and evaluate the same in a 3-week in-the-wild study involving 22 participants. Our key evaluation results and post-study participant assessment demonstrate that it is possible to predict these emotion states with an average accuracy (AUCROC) of 73% (std dev. 6%, maximum 87%) combining these two touch interactions only.

show abstract

Section: Related Workmentioning

confidence: 99%

Emotion detection from touch interactions during text entry on smartphones

Ghosh

Hiware

Ganguly

et al. 2019

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

show abstract

“…Among the experiments presented in the reviewed papers, two major types may be distinguished. The first type are laboratory experiments performed in controlled environments and usually involving a user in a single session consisting of several stages [37][38][39][40][41][42][43][44][45][46][47][48][49]. The other type is in-the-wild experiments carried in uncontrolled conditions, usually during users' everyday phone usage.…”

Section: Experiments Designmentioning

confidence: 99%

“…Various methods were used to bring about stress, e.g., time limits given for a task [38,56,69], sounds and vibrations [56,69], and unexpected behaviours of the device [56]. It is also possible to design a game which makes the player always lose or win [44] or has a dull and a fascinating mode which is designed to induce negative and positive states, respectively [46]. The experiment protocol usually consists of several stages ending with self-reports with a calming stage between them, which may incorporate relaxation music [69] or video [42,43,49].…”

Section: Experiments Designmentioning

confidence: 99%

“…In the case of laboratory experiments, a self-report is presented to a participant at specified points of the session, usually at the beginning, end, and after each step or task if a session consists of several of them [38,42,68,69]. Sometimes the labels are assigned according to the emotions elicited, assuming the induction was successful [44,46].…”

Section: Data Labellingmentioning

confidence: 99%

“…These data are also processed to extract features, which may be useful for emotion recognition. Feature values are calculated on the basis of single strokes [99] or may be averaged over a series of them from a predefined time window [44] or session or a predefined number of consecutive strokes [41].…”

Section: Touch Dynamicsmentioning

confidence: 99%

See 2 more Smart Citations

A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors

Kołakowska

Szwoch

2020

Sensors

View full text Add to dashboard Cite

In recent years, emotion recognition algorithms have achieved high efficiency, allowing the development of various affective and affect-aware applications. This advancement has taken place mainly in the environment of personal computers offering the appropriate hardware and sufficient power to process complex data from video, audio, and other channels. However, the increase in computing and communication capabilities of smartphones, the variety of their built-in sensors, as well as the availability of cloud computing services have made them an environment in which the task of recognising emotions can be performed at least as effectively. This is possible and particularly important due to the fact that smartphones and other mobile devices have become the main computer devices used by most people. This article provides a systematic overview of publications from the last 10 years related to emotion recognition methods using smartphone sensors. The characteristics of the most important sensors in this respect are presented, and the methods applied to extract informative features on the basis of data read from these input channels. Then, various machine learning approaches implemented to recognise emotional states are described.

show abstract

A Novel Method to Build and Validate an Affective State Prediction Model from Touch-Typing

Tikadar

Bhattacharya

2019

Human-Computer Interaction – INTERACT 2019

Self Cite

View full text Add to dashboard Cite

Affective systems are supposed to improve user satisfaction and hence usability by identifying and complementing the affective state of a user at the time of interaction. The first and most important challenge for building such systems is to identify the affective state in a systematic way. This is generally done based on computational models. Building such models requires affective data. In spite of the extensive growth in this research area, there are a number of challenges in affect induction method for collecting the affective data as well as for building models for real-time prediction of affective states. In this article, we have reported a novel method for inducing particular affective states to unobtrusively collect the affective data as well as a minimalist model to predict the affective states of a user from her/his typing pattern on a touchscreen of a smartphone. The prediction accuracy for our model was 86.60%. The method for inducing the specific affective states and the model to predict these states are validated through empirical studies comprising EEG signals of twenty two participants.

show abstract

A Minimalist Approach for Identifying Affective States for Mobile Interaction Design

Cited by 6 publications

References 34 publications

Emotion detection from touch interactions during text entry on smartphones

Emotion detection from touch interactions during text entry on smartphones

A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors

A Novel Method to Build and Validate an Affective State Prediction Model from Touch-Typing

Contact Info

Product

Resources

About