WiStress

Ha, Unsoo; Madani, Sohrab; Adib, Fadel

doi:10.1145/3478121

Cited by 30 publications

(15 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We designed ML models to infer EMA receptivity and affect. A wide variety of ML algorithms are used in affect and receptivity prediction including random forest (RF) [ 31 , 32 , 39 ], support vector machine [ 33 , 34 , 39 ], logistic regression, k -nearest neighbors [ 30 ], neural network (NN; long short-term memory, recurrent NN, convolutional NN, etc) [ 31 , 39 ], and naive Bayes [ 39 , 41 ]. On the basis of our sensor data, initial tests, and drawing inspiration from previous studies, especially those by Mishra et al [ 10 , 11 ].…”

Section: Methodsmentioning

confidence: 99%

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

King,

Yu,

Vaessen

et al. 2024

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

Background As mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants’ responses when implementing new survey delivery methods. Objective This study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants’ reported emotional state. We examine the factors that affect participants’ receptivity to EMAs in a 10-day wearable and EMA–based emotional state–sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs. Methods We collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity. Results Our findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions. Conclusions Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.

show abstract

Section: Methodsmentioning

confidence: 99%

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

King,

Yu,

Vaessen

et al. 2024

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

show abstract

“…• Different mental states can affect human learning ability, such as drowsiness and stress [42], [43], [71].…”

Section: Erudite: Human-in-the-loop Iot Learning Systemmentioning

confidence: 99%

ERUDITE: Human-in-the-Loop IoT for an Adaptive Personalized Learning System

Taherisadr¹,

Faruque²,

Elmalaki³

2023

Preprint

View full text Add to dashboard Cite

Thanks to the rapid growth in wearable technologies and recent advancement in machine learning and signal processing, monitoring complex human contexts becomes feasible, paving the way to develop human-in-the-loop IoT systems that naturally evolve to adapt to the human and environment state autonomously. Nevertheless, a central challenge in designing many of these IoT systems arises from the requirement to infer the human mental state, such as intention, stress, cognition load, or learning ability. While different human contexts can be inferred from the fusion of different sensor modalities that can correlate to a particular mental state, the human brain provides a richer sensor modality that gives us more insights into the required human context. This paper proposes ERUDITE, a human-in-the-loop IoT system for the learning environment that exploits recent wearable neurotechnology to decode brain signals.Through insights from concept learning theory, ERUDITE can infer the human state of learning and understand when human learning increases or declines. By quantifying human learning as an input sensory signal, ERUDITE can provide adequate personalized feedback to humans in a learning environment to enhance their learning experience. ERUDITE is evaluated across 15 participants and showed that by using the brain signals as a sensor modality to infer the human learning state and providing personalized adaptation to the learning environment, the participants' learning performance increased on average by 26%. Furthermore, we showed that ERUDITE can be deployed on an edge-based prototype to evaluate its practicality and scalability.

show abstract

“…There have been many wearable signals utilized for emotion recognition: Electrocardiography (ECG) [33,34,35,36], Photoplethysmogram (PPG) [37], Galvanic Skin Response (GSR) [38,37,39,40,41], Electroencephalography (EEG), Respiration (RESP) [42,34], Body Temperature (BT) [38], Accelerometer (ACC) [42,38,43], and phone data (GPS, ACC, Activity, Call, and Text logs) [44]. Although ACC on its own is not necessarily a physiological sensor since it does not measure body functions.…”

Section: Mobile Sensor-base Affect Inferencementioning

confidence: 99%

“…The types of emotions related to a signal and the viability of recognition vary from sensor to sensor. Negative affect [36], Positive affect [36], general emotion [34], stress [33,38,3,40], and happiness/sadness/fear/anger [41]. There is still no accepted best sensor suite for affect inference, but most researchers would agree that multimodal analysis is the best route for inferring emotional state [45].…”

Section: Mobile Sensor-base Affect Inferencementioning

confidence: 99%

“…Modeling techniques utilized by these researchers also vary, although some sensors, particularly those collected via phone data, would be considered contextual data rather than time-series data. Machine learning algorithms employed by researchers include Random Forest (RF) [33,38,41], Support Vector Machine (SVM) [34,35,41], Logistic Regression (LR), K-nearest neighbors (kNN) [38], Neural Network (LSTM, RNN, CNN, etc.) [38,41], Naive Bayes (NB) [41], and many more [46].…”

Section: Mobile Sensor-base Affect Inferencementioning

confidence: 99%

See 1 more Smart Citation

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study (Preprint)

King¹,

Yu²,

Vaessen³

et al. 2023

Preprint

View full text Add to dashboard Cite

BACKGROUND As mobile health (mHealth) studies become increasingly productive due to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. OBJECTIVE We examine the factors that affect participants’ responsiveness to ecological momentary assessments (EMA) in a 10-day wearable and EMA-based affect sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the two constructs. METHODS We collected the data from 45 healthy participants wearing two devices measuring electrodermal activity, acceleration, electrocardiography, and skin temperature while answering 10 EMAs a day containing questions related to perceived mood. Due to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during a response. Therefore, we utilized unsupervised and supervised learning methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to find the relationship between physiological relationships and responsiveness then inferred the emotional state during non-responses. For the supervised learning method, we primarily used Random Forest (RF) and Neural Networks (NN) to predict affect of unlabeled data points. RESULTS Based on our findings we showed that using a receptivity model to trigger EMAs will decrease the reported negative affect by more than 3 points or 0.29 standard deviation using our psychological instrument scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during non-responses. CONCLUSIONS Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of a mHealth study, particularly those studies that employ a learning algorithm to trigger EMAs. Therefore, we propose a smart trigger that promotes EMA and JITI receptivity without influencing affect during sampled time points as future work.

show abstract

WiStress

Cited by 30 publications

References 83 publications

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

ERUDITE: Human-in-the-Loop IoT for an Adaptive Personalized Learning System

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study (Preprint)

Contact Info

Product

Resources

About