Classification of Eye Tracking Data Using a Convolutional Neural Network

Yin, Yuehan; Juan, Chunghao; Chakraborty, Joyram; McGuire, Michael P.

doi:10.1109/icmla.2018.00085

Cited by 18 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of these studies do not focus on differentiating mental states from the data but rather improving the gaze estimation itself, unsupervised feature extractions, or predictions about the demographics of the participants. The use cases for the applications are many-fold, such as websites (Yin et al, 2018) or Augmented and Virtual Reality (Lemley et al, 2018).…”

Section: Related Work On Deep Learning For Eye Trackingmentioning

confidence: 99%

Imaging Time Series of Eye Tracking Data to Classify Attentional States

et al. 2021

View full text Add to dashboard Cite

It has been shown that conclusions about the human mental state can be drawn from eye gaze behavior by several previous studies. For this reason, eye tracking recordings are suitable as input data for attentional state classifiers. In current state-of-the-art studies, the extracted eye tracking feature set usually consists of descriptive statistics about specific eye movement characteristics (i.e., fixations, saccades, blinks, vergence, and pupil dilation). We suggest an Imaging Time Series approach for eye tracking data followed by classification using a convolutional neural net to improve the classification accuracy. We compared multiple algorithms that used the one-dimensional statistical summary feature set as input with two different implementations of the newly suggested method for three different data sets that target different aspects of attention. The results show that our two-dimensional image features with the convolutional neural net outperform the classical classifiers for most analyses, especially regarding generalization over participants and tasks. We conclude that current attentional state classifiers that are based on eye tracking can be optimized by adjusting the feature set while requiring less feature engineering and our future work will focus on a more detailed and suited investigation of this approach for other scenarios and data sets.

show abstract

Section: Related Work On Deep Learning For Eye Trackingmentioning

confidence: 99%

Imaging Time Series of Eye Tracking Data to Classify Attentional States

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In [57], a modified LeNet5 CNN model combined with feature engineering model was used to determine whether a user was interacting a particular interface (Google News or NewsMap) to answer questions about current events. The resulting grayscale images were fed to train the CNN model and perform two classification tasks to identify web user interfaces and nationalities of users.…”

Section: Related Workmentioning

confidence: 99%

“…The eye tracking data were sliced using a 10-s time window where every 10 seconds of gaze points were grouped for the three different information presentation methods. We chose a time window of 10 s to keep this study consistent with our previous study [57]. Each 10-s gaze point image was represented by a 2D array with a size of 1440 × 900 -which corresponds directly to the screen resolution of the computer used for the study.…”

Section: Data Preprocessingmentioning

confidence: 99%

“…As was the case in our previous study [57], the JavaScript Object Notation (JSON) format was used to transform the eye tracking data because it can be easily constructed using pairs of name/value and an ordered list of values [52]. Mon-goDB was chosen to store the data because it directly maps to the JSON format.…”

Section: Database and Data Formatsmentioning

confidence: 99%

“…In our previous study [57], we used an adapted version of the LeNet-5 model which consisted of 4 convolutional layers each with the ReLU activation function, 4 max pooling layers, one fully connected layer with the ReLU activation function, and one output layer containing the softmax activation function. The kernel size of each convolutional layer was 9-by-9, and the number of strides was 1.…”

Section: Cnn Model Based On Lenet-5mentioning

confidence: 99%

See 2 more Smart Citations

Classification of Eye Tracking Data in Visual Information Processing Tasks Using Convolutional Neural Networks and Feature Engineering

et al. 2021

Self Cite

View full text Add to dashboard Cite

Eye tracking technology has been adopted in numerous studies in the field of human-computer interaction (HCI) to understand visual and display-based information processing as well as the underlying cognitive processes employed by users when navigating a computer interface. Analyzing eye tracking data can also help identify interaction patterns with regard to salient regions of an information display. Deep learning technology is increasingly being used in the analysis of eye tracking data by allowing for the classification of large amounts of eye tracking results. In this paper, eye tracking data and convolutional neural networks (CNNs) were used to perform a classification task to predict three types of information presentation methods. As a first step, a number of data preprocessing and feature engineering approaches were applied to eye tracking data collected through a controlled visual information processing experiment. The resulting data were used as input for the comparison of four CNN models with different architectures. In this experiment, two CNN models were effective in classifying the information presentations with overall accuracy greater than 80%.

show abstract