A Recurrent Neural Network for Attenuating Non-cognitive Components of Pupil Dynamics

Abstract

There is increasing interest in how the pupil dynamics of the eye reflect underlying cognitive processes and brain states. Problematic, however, is that pupil changes can be due to non-cognitive factors, for example luminance changes in the environment, accommodation and movement. In this paper we consider how by modeling the response of the pupil in real-world environments we can capture the non-cognitive related changes and remove these to extract a residual signal which is a better index of cognition and performance. Specifically, we utilize sequence measures such as fixation position, duration, saccades, and blink-related information as inputs to a deep recurrent neural network (RNN) model for predicting subsequent pupil diameter. We build and evaluate the model for a task where subjects are watching educational videos and subsequently asked questions based on the content. Compared to commonly-used models for this task, the RNN had the lowest errors rates in predicting subsequent pupil dilation given sequence data. Most importantly was how the model output related to subjects' cognitive performance as assessed by a post-viewing test. Consistent with our hypothesis that the model captures non-cognitive pupil dynamics, we found (1) the model's root-mean square error was less for lower performing subjects than for those having better performance on the post-viewing test, (2) the residuals of the RNN (LSTM) model had the highest correlation with subject post-viewing test scores and (3) the residuals had the highest discriminability (assessed via area under the ROC curve, AUC) for classifying high and low test performers, compared to the true pupil size or the RNN model predictions. This suggests that deep learning sequence models may be good for separating components of pupil responses that are linked to luminance and accommodation from those that are linked to cognition and arousal.

Attenuation of the dynamic pupil light response during screen viewing for arousal assessment

“…Neural networks, which are entirely data-driven, have also been successfully used to model different biological processes including pupil responses. One such example is the work of Koorathota et al (2021) who trained an LSTM model to predict the pupil size based on gaze data (fixations, saccades, etc.) and past pupil size.…”

Convolution Neural Network Based Prediction for Eye Gaze Estimation

“…Fig 8 : Capturing frontal view.From figure[6][7][8] is a representation for eye gaze estimation captured at different angle (right, left, frontal) views of the eye ball movement.4. ConclusionThe project is designed using Deep convolutional neural network and is efficient for providing the desired outcomes.…”