&lt;italic&gt;SmartMood&lt;/italic&gt;: Toward Pervasive Mood Tracking and Analysis for Manic Episode Detection

Lam, Kam-Yiu; Wang, Jiantao; Ng, Joseph Kee‐Yin; Han, Song; Li-mei, Zheng; Kam, Calvin Ho Chuen; Zhu, Chun Jiang

doi:10.1109/thms.2014.2360469

Cited by 11 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple studies have aimed to determine emotion states based on smartphone activities (like SMS, call pattern, voice data), additional sensor readings (like wrist sensor, skin conductor) and contextual details (like location, proximity) (Bogomolov et al, 2014;2013;Lam et al, 2015;LiKamWa et al, 2013;Lu et al, 2012;Pielot et al, 2015;Rachuri et al, 2010;Roshanaei et al, 2017;Sano and Picard, 2013;Politou et al, 2017). This has led to the development of different interactive recommender application based on the emotion responses (Andjelkovic et al, 2018;McGaugh, 2016).…”

Section: Related Workmentioning

confidence: 99%

Emotion detection from touch interactions during text entry on smartphones

Ghosh

Hiware

Ganguly

et al. 2019

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

There are different modes of interaction with a software keyboard on a smartphone, such as typing and swyping. Patterns of such touch interactions on a keyboard may reflect emotions of a user. Since users may switch between different touch modalities while using a keyboard, therefore, automatic detection of emotion from touch patterns must consider both modalities in combination to detect the pattern. In this paper, we focus on identifying different features of touch interactions with a smartphone keyboard that lead to a personalized model for inferring user emotion. Since distinguishing typing and swyping activity is important to record the correct features, we designed a technique to correctly identify the modality. The ground truth labels for user emotion are collected directly from the user by periodically collecting self-reports. We jointly model typing and swyping features and correlate them with user provided self-reports to build a personalized machine learning model, which detects four emotion states (happy, sad, stressed, relaxed). We combine these design choices into an Android application TouchSense and evaluate the same in a 3-week in-the-wild study involving 22 participants. Our key evaluation results and post-study participant assessment demonstrate that it is possible to predict these emotion states with an average accuracy (AUCROC) of 73% (std dev. 6%, maximum 87%) combining these two touch interactions only.

show abstract

Section: Related Workmentioning

confidence: 99%

Emotion detection from touch interactions during text entry on smartphones

Ghosh

Hiware

Ganguly

et al. 2019

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

show abstract

“…Many studies [32][33][34][35] have considered affect sensing from voice during phone conversation or from other day-to-day conversation, but studies that considered affect sensing on phone from facial expression are rare. In StudentLife 35 authors process audio on the fly for audio and speech/conversation detection.…”

Section: Landscape Of Opportunistic and Context-aware Sensingmentioning

confidence: 99%

“…However, of the many studies discussed to date, only one 24 has engaged mental health patients in experiments. In order to demonstrate importance of a particular algorithm/method for mental health patients, it is necessary to conduct such experiments with them 33,51 .…”

Section: Robust Classification Algorithmsmentioning

confidence: 99%

Opportunistic and Context-Aware Affect Sensing on Smartphones

Rana

Hume

Reilly

et al. 2016

IEEE Pervasive Comput.

View full text Add to dashboard Cite

Opportunistic affect sensing offers unprecedented potential for capturing spontaneous affect, eliminating biases inherent in the controlled setting. Facial expression and voice are two major affective displays, however most affect sensing systems on smartphone avoid them due to extensive power requirements. Encouragingly, due to the recent advent of low-power DSP (Digital Signal Processing) co-processor and GPU (Graphics Processing Unit) technology, audio and video sensing are becoming more feasible on smartphone. To utilize opportunistically captured facial expression and voice, gathering contextual information about the dynamic audio-visual stimuli is also important. This paper discusses recent advances of affect sensing on the smartphone and identifies the key barriers and potential solutions for implementing opportunistic and contextaware affect sensing on smartphone platforms. In addition to exploring the technical challenges (privacy, battery life and robust algorithms), the challenges of recruiting and retention of mental health patients have also been considered; as experimentation with mental health patients is difficult but crucial to showcase the importance/effectiveness of the smartphone centred affect sensing technology. Affect SensingAffect sensing on smartphones is relatively new. Based on the sensing modalities, we group the existing studies into five categories: (1) phone usage (2) phone interaction (3) sensor feed (4) hybrid and (5) self-reporting.Phone Usage: includes communication patterns, i.e., statistics of call and text messages. It also includes the frequency of using social media, application usage patterns (for security reason no longer available to a non-system app), and switching between applications.

show abstract

“…Studies have also shown that the pause, intonation and emotional tension during speech could help to detect whether a BD patient is in manic state. Smartphones have been used for detecting different mood states and mood changes of BD by analyzing patient’s physiological activities such as Heart Rate Variability (HRV), Electro Dermal Response (EDR) [ 14 , 15 ] and behavior activities (such as geospatial information and phone call activities) [ 16 ]. Low-level speech features correlated with BD mood states, depressive and manic states can be detected using smartphones, although the detection accuracy was moderate [ 17 , 18 ].…”

Section: Introductionmentioning

confidence: 99%

Detecting Manic State of Bipolar Disorder Based on Support Vector Machine and Gaussian Mixture Model Using Spontaneous Speech

Pan¹,

Gui²,

Zhang³

et al. 2018

Psychiatry Investig

View full text Add to dashboard Cite

ObjectiveThis study was aimed to compare the accuracy of Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) in the detection of manic state of bipolar disorders (BD) of single patients and multiple patients. Methods21 hospitalized BD patients (14 females, average age 34.5±15.3) were recruited after admission. Spontaneous speech was collected through a preloaded smartphone. Firstly, speech features [pitch, formants, mel-frequency cepstrum coefficients (MFCC), linear prediction cepstral coefficient (LPCC), gamma-tone frequency cepstral coefficients (GFCC) etc.] were preprocessed and extracted. Then, speech features were selected using the features of between-class variance and within-class variance. The manic state of patients was then detected by SVM and GMM methods. ResultsLPCC demonstrated the best discrimination efficiency. The accuracy of manic state detection for single patients was much better using SVM method than GMM method. The detection accuracy for multiple patients was higher using GMM method than SVM method. ConclusionSVM provided an appropriate tool for detecting manic state for single patients, whereas GMM worked better for multiple patients’ manic state detection. Both of them could help doctors and patients for better diagnosis and mood state monitoring in different situations.

show abstract

<italic>SmartMood</italic>: Toward Pervasive Mood Tracking and Analysis for Manic Episode Detection

Cited by 11 publications

References 9 publications

Emotion detection from touch interactions during text entry on smartphones

Emotion detection from touch interactions during text entry on smartphones

Opportunistic and Context-Aware Affect Sensing on Smartphones

Detecting Manic State of Bipolar Disorder Based on Support Vector Machine and Gaussian Mixture Model Using Spontaneous Speech

Contact Info

Product

Resources

About