Predicting Confusion from Eye-Tracking Data with Recurrent Neural Networks

Abstract

Encouraged by the success of deep learning in a variety of domains, we investigate the suitability and effectiveness of Recurrent Neural Networks (RNNs) in a domain where deep learning has not yet been used; namely detecting confusion from eye-tracking data. Through experiments with a dataset of user interactions with ValueChart (an interactive visualization tool), we found that RNNs learn a feature representation from the raw data that allows for a more powerful classifier than previous methods that use engineered features. This is evidenced by the stronger performance of the RNN (0.74/0.71 sensitivity/specificity), as compared to a Random Forest classifier (0.51/0.70 sensitivity/specificity), when both are trained on an un-augmented dataset. However, using engineered features allows for simple data augmentation methods to be used. These same methods are not as effective at augmentation for the feature representation learned from the raw data, likely due to an inability to match the temporal dynamics of the data.

Remote Eye Gaze Tracking Research: A Comparative Evaluation on Past and Recent Progress

“…Recently, the usage of REGTs has gotten even wider with emerging usage in privacy-aware interactions. We have identified emerging eye solutions and classified them under the existing categories, such as device interactions (IoT smart home control [256], semi-autonomous driving [257,258], artistic drawing in robots [259][260][261][262]), human behavior analysis (confusion prediction [263][264][265], intention extraction [266,267], driver's attention [30,32,[268][269][270], detecting personality traits [271]), medical support (medical image interpretation [272][273][274][275], patients support [276]), augmented reality (social games [277,278], virtual space control [279]), and privacy issues (privacy-aware eye tracking [280][281][282][283], gaze-touch authentication [284][285][286][287][288]). The arrow pointers indicate a solution application in a field and across deployment platforms.…”