Deep Learning based Emotion Classification with Temporal Pupillometry Sequences

Rafique, Sidra; Kanwal, Nadia; Ansari, Mohd. Samar; Asghar, Mamoona Naveed; Akhtar, Zuhair

doi:10.1109/icecet52533.2021.9698663

Cited by 3 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sampling change will need to be addressed. For the offline model, the LSTM model will need to be further finetuned [26].…”

Section: Future Workmentioning

confidence: 99%

Thelxinoë: Recognizing Human Emotions Using Pupillometry and Machine Learning

Barker,

Levkowitz

2024

MLAIJ

View full text Add to dashboard Cite

In this study, we present a method for emotion recognition in Virtual Reality (VR) using pupillometry. We analyze pupil diameter responses to both visual and auditory stimuli via a VR headset and focus on extracting key features in the time-domain, frequency-domain, and time-frequency domain from VR-generated data. Our approach utilizes feature selection to identify the most impactful features using Maximum Relevance Minimum Redundancy (mRMR). By applying a Gradient Boosting model, an ensemble learning technique using stacked decision trees, we achieve an accuracy of 98.8% with feature engineering, compared to 84.9% without it. This research contributes significantly to the Thelxinoë framework, aiming to enhance VR experiences by integrating multiple sensor data for realistic and emotionally resonant touch interactions. Our findings open new avenues for developing more immersive and interactive VR environments, paving the way for future advancements in virtual touch technology.

show abstract

“…The sampling change will need to be addressed. For the offline model, the LSTM model will need to be further finetuned [26].…”

Section: Future Workmentioning

confidence: 99%

Thelxinoë: Recognizing Human Emotions Using Pupillometry and Machine Learning

Barker,

Levkowitz

2024

MLAIJ

View full text Add to dashboard Cite

show abstract

“…Using a valence-arousal model we can classify a human's emotional response from their physiological data. Physiological data include EEG, ECG, heart rate, Galvanic Skin Response, blood pressure, and pupil response (Dzedzickis et al 2020;Rafique et al 2021a). Sensors are usually required to be physically attached to the human body to get these physiological responses which make people uncomfortable and hence cannot be used on a large scale.…”

Section: Assessing Meaningful User Interactionmentioning

confidence: 99%

“…Pupil responses are not discrete and therefore, temporal data analysis is required to assess the response accurately. Therefore, machine learning and deep learning methods can be employed to capture the constriction and dilation patterns (Rafique et al 2021a). This paper proposes that the pupil responses can also be used to assess the QofE from the technology's perspective.…”

Section: Pupillometry As a Metric For Qofementioning

confidence: 99%

Using physiological signals to measure the Quality-of-Experience of Health Care Professionals when interacting with a clinical guideline mobile app

Mitchell¹,

Kanwal²

2022

Electronic Workshops in Computing

View full text Add to dashboard Cite

Digital content adaptation and personalisation is a crucial component in increasing user engagement, and becoming of interest to designers/developers in areas related to clinical information delivery. In order to achieve this, new data-intensive methods are required that go beyond traditional user-centred design approaches. In this position paper, we discuss how although user-centred design has shown to be useful for generating generalised design guidelines (predominantly driven by qualitative data collection techniques), more quantitative methods and the use of measures such as Quality of Experience, could not only augment standard user research methods but also provide data to inform the adaption and personalisation of interfaces. In this paper we propose a solution-by-design to gather personal preferences through users' physiological data (using pupillometry) and how it would be useful for applications such as mobile apps for clinical guidelines, where access to in-situ data collection is increasingly more challenging.

show abstract