Machine learning-based classification of viewing behavior using a wide range of statistical oculomotor features

Kootstra, T. M.; Teuwen, Jonas; Goudsmit, Jeroen P.; Nijboer, Tanja C.W.; Dodd, Michael D.; Stigchel, Stefan Van der

doi:10.1167/jov.20.9.1

Cited by 5 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Better classification performances relative to previous investigations are even more striking given that we have run analyses multiple times, leading to presumable conservative results. As Bednarik et al (2012), Kootstra et al (2020), andJang et al (2014) demonstrate, the inclusion of further gaze characteristics can substantially improve classification in related settings, which should be addressed in an investigation that allows free gaze movements, but controls for effects such as foreshortening errors affecting pupil size.…”

Section: Discussionmentioning

confidence: 99%

“…Addressing the question whether pupil dilation may decode decisions on a single trial level, a few investigations using support vector machine classifiers (SVMs) have been put forward over the past years (Bednarik et al, 2012;Jangraw, Wang, Lance, Chang, & Sajda, 2014;Medathati et al, 2020). Although several articles have found random forest classifiers to be particularly useful for predictions also based on pupil dilation (Kootstra et al, 2020;Pasquali et al, 2020), no prior work has focused on predicting intentions only using pupil size. Jangraw et al (2014) used electroencephalography and ocular parameters to infer objects that were of subjective interest to users in a virtual reality environment and found both to be predictive.…”

Section: Theoretical Backgroundmentioning

confidence: 99%

“…To classify a feature vector as an intent-or non-intent-related pupil size signal we opted for the state-of-the-art random forest classifier for binary classification that has been used in prior work with eye tracking data (Kootstra et al, 2020;Pasquali et al, 2020). The implementation was based on the RandomForestClassifier of the sklearn.ensemble package of the Scikit-learn Python library (Pedregosa et al, 2011).…”

Section: Classifiermentioning

confidence: 99%

See 2 more Smart Citations

Decoding binary decisions under differential target probabilities from pupil dilation: A random forest approach

et al. 2021

Self Cite

View full text Add to dashboard Cite

Although our pupils slightly dilate when we look at an intended target, they do not when we look at irrelevant distractors. This finding suggests that it may be possible to decode the intention of an observer, understood as the outcome of implicit covert binary decisions, from the pupillary dynamics over time. However, few previous works have investigated the feasibility of this approach and the few that did, did not control for possible confounds such as motor-execution, changes in brightness, or target and distractor probability. We report on our efforts to decode intentions from pupil dilation obtained under strict experimental control on a single trial basis using a machine learning approach. The basis for our analyses are data of 69 participants who looked at letters that needed to be selected with stimulus probabilities that varied systematically in a blockwise manner (n = 19,417 trials). We confirm earlier findings that pupil dilation is indicative of intentions and show that these can be decoded with a classification performance of up to 76% area under the curve for receiver operating characteristic curves if targets are rarer than distractors. To better understand which characteristics of the pupillary signal are most informative, we finally compare relative feature importances. The first derivative of pupil size changes was found to be most relevant, allowing us to decode intention within only about 800 ms of trial onset. Taken together, our results provide credible insights into the potential of decoding intentions from pupil dilation and may soon form the basis for new applications in visual search, gaze-based interaction, or human-robot interaction.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Theoretical Backgroundmentioning

confidence: 99%

Section: Classifiermentioning

confidence: 99%

See 1 more Smart Citation

Decoding binary decisions under differential target probabilities from pupil dilation: A random forest approach

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The most commonly used machine learning algorithms included support vector machine (SVM), K-nearest neighbor (KNN), and random forest. The predictions and classification tasks are done based on the features from datasets (Kootstra et al, 2020). Machine learning can be applied to many fields such as computer vision, where it is unworkable to create traditional algorithms to perform the tasks required.…”

Section: Machine Learningmentioning

confidence: 99%

Eye-Tracking Feature Extraction for Biometric Machine Learning

2022

View full text Add to dashboard Cite

ContextEye tracking is a technology to measure and determine the eye movements and eye positions of an individual. The eye data can be collected and recorded using an eye tracker. Eye-tracking data offer unprecedented insights into human actions and environments, digitizing how people communicate with computers, and providing novel opportunities to conduct passive biometric-based classification such as emotion prediction. The objective of this article is to review what specific machine learning features can be obtained from eye-tracking data for the classification task.MethodsWe performed a systematic literature review (SLR) covering the eye-tracking studies in classification published from 2016 to the present. In the search process, we used four independent electronic databases which were the IEEE Xplore, the ACM Digital Library, and the ScienceDirect repositories as well as the Google Scholar. The selection process was performed by using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) search strategy. We followed the processes indicated in the PRISMA to choose the appropriate relevant articles.ResultsOut of the initial 420 articles that were returned from our initial search query, 37 articles were finally identified and used in the qualitative synthesis, which were deemed to be directly relevant to our research question based on our methodology.ConclusionThe features that could be extracted from eye-tracking data included pupil size, saccade, fixations, velocity, blink, pupil position, electrooculogram (EOG), and gaze point. Fixation was the most commonly used feature among the studies found.

show abstract

“…Secondly, ever since the seminal works of Buswell [ 15 ] and Yarbus [ 16 ], we have been aware that eye movements are foremost driven by task type (top-down) and visual saliency (bottom-up). Later on, it has been shown more reliably that task type–such as search or free-viewing–influences gaze behaviour [ 17 – 19 ], and that oculomotor metrics besides pupil dilation [ 20 ] and peak saccade velocity (e.g., saccade amplitude, fixation duration) can provide sufficient information for machine learning algorithms to predict task type at above chance level [ 21 , 22 ]. In this manuscript, we describe that, besides top-down and bottom-up mechanisms, arousal–estimated by the link to heart rate–also contributes to eye movements.…”

Section: Introductionmentioning

confidence: 99%

Seeing the Forrest through the trees: Oculomotor metrics are linked to heart rate

et al. 2022

Self Cite

View full text Add to dashboard Cite

Fluctuations in a person’s arousal accompany mental states such as drowsiness, mental effort, or motivation, and have a profound effect on task performance. Here, we investigated the link between two central instances affected by arousal levels, heart rate and eye movements. In contrast to heart rate, eye movements can be inferred remotely and unobtrusively, and there is evidence that oculomotor metrics (i.e., fixations and saccades) are indicators for aspects of arousal going hand in hand with changes in mental effort, motivation, or task type. Gaze data and heart rate of 14 participants during film viewing were used in Random Forest models, the results of which show that blink rate and duration, and the movement aspect of oculomotor metrics (i.e., velocities and amplitudes) link to heart rate–more so than the amount or duration of fixations and saccades. We discuss that eye movements are not only linked to heart rate, but they may both be similarly influenced by the common underlying arousal system. These findings provide new pathways for the remote measurement of arousal, and its link to psychophysiological features.

show abstract

Machine learning-based classification of viewing behavior using a wide range of statistical oculomotor features

Cited by 5 publications

References 28 publications

Decoding binary decisions under differential target probabilities from pupil dilation: A random forest approach

Decoding binary decisions under differential target probabilities from pupil dilation: A random forest approach

Eye-Tracking Feature Extraction for Biometric Machine Learning

Seeing the Forrest through the trees: Oculomotor metrics are linked to heart rate

Contact Info

Product

Resources

About