An inverse Yarbus process: Predicting observers’ task from eye movement patterns

Haji-Abolhassani, Amin; Clark, James J.

doi:10.1016/j.visres.2014.08.014

Cited by 78 publications

(53 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A linear classifier was tested on the collected data, and the results showed an inability to classify correctly. Although further research has questioned this result by noting both a lack of variable classifying techniques and a lack of descriptive tasks—in addition to showing a positive classification for Yarbus’ tasks (Borji & Itti, 2014; Haji-Albolhassani & Clarke, 2014)—the key result from Greene, Liu, and Wolfe (2011) as it relates to the present work comes from their final experiment. Given that humans are the best pattern classifiers, they attempted to determine whether humans could succeed where their computer classifier failed.…”

mentioning

confidence: 62%

“…Previous computer classification has shown that a search task typically can be classified above chance (Borji & Itti, 2014; Borji, Lennartz, & Pomplun, 2015; Haji-Abolhassani & Clark, 2014; Henderson et al, 2013). Similarly, we found a strong correct classification rate for Search, particularly in the Fixation-NoScene condition.…”

Section: Methodsmentioning

confidence: 99%

“…This has proven possible under certain conditions (Borji & Itti, 2014; Borji, Lennartz, & Pomplun; 2015; Haji-Abolhassani & Clark, 2014; Henderson et al, 2013). For example, Borji and Itti (2014) measured eye movements as subjects performed Yarbus’ seven tasks on 15 natural scenes.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Human classifier: Observers can deduce task solely from eye movements

Bahle

Mills²,

Dodd³

2017

Atten Percept Psychophys

View full text Add to dashboard Cite

Computer classifiers have been successful at classifying various tasks using eye movement statistics. However, the question of human classification of task from eye movements has rarely been studied. Across two experiments, we examined whether humans could classify task based solely on the eye movements of other individuals. In Experiment 1, human classifiers were shown one of three sets of eye movements: Fixations, which were displayed as blue circles, with larger circles meaning longer fixation durations; Scanpaths, which were displayed as yellow arrows; and Videos, in which a neon green dot moved around the screen. There was an additional Scene manipulation in which eye movement properties were displayed either on the original scene where the task (Search, Memory, or Rating) was performed or on a black background in which no scene information was available. Experiment 2 used similar methods but only displayed Fixations and Videos with the same Scene manipulation. The results of both experiments showed successful classification of Search. Interestingly, Search was best classified in the absence of the original scene, particularly in the Fixation condition. Memory also was classified above chance with the strongest classification occurring with Videos in the presence of the scene. Additional analyses on the pattern of correct responses in these two conditions demonstrated which eye movement properties successful classifiers were using. These findings demonstrate conditions under which humans can extract information from eye movement characteristics in addition to providing insight into the relative success/failure of previous computer classifiers.

show abstract

mentioning

confidence: 62%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Human classifier: Observers can deduce task solely from eye movements

Bahle

Mills²,

Dodd³

2017

Atten Percept Psychophys

View full text Add to dashboard Cite

show abstract

“…There are further studies on this topic, but their data is not publicly available (DeAngelus & Pelz, 2009;Haji Abolhassani & Clark, 2014). Therefore, a direct comparison of our results to their performance is not possible.…”

Section: Yarbus: Image Viewing With Different Tasksmentioning

confidence: 98%

SubsMatch 2.0: Scanpath comparison and classification based on subsequence frequencies

et al. 2016

View full text Add to dashboard Cite

Our eye movements are driven by a continuous trade-off between the need for detailed examination of objects of interest and the necessity to keep an overview of our surrounding. In consequence, behavioral patterns that are characteristic for our actions and their planning are typically manifested in the way we move our eyes to interact with our environment. Identifying such patterns from individual eye movement measurements is however highly challenging. In this work, we tackle the challenge of quantifying the influence of experimental factors on eye movement sequences. We introduce an algorithm for extracting sequence-sensitive features from eye movements and for the classification of eye movements based on the frequencies of small subsequences. Our approach is evaluated against the state-of-the art on a novel and a very rich collection of eye movements data derived from four experimental settings, from static viewing tasks to highly dynamic outdoor settings. Our results show that the proposed method is able to classify eye movement sequences over a variety of experimental designs. The choice of parameters is discussed in detail with special focus on highlighting different aspects of general scanpath shape. Algorithms and evaluation data are available at: www.ti.uni-tuebingen.de/scanpathcomparison.html.

show abstract

“…Eye tracking is widely used in both animals and humans to study the mechanisms underlying perception, cognition, and action, and it is useful for investigating neurological and neurodegenerative diseases in human patients [1][2][3][4][5] . This is in part due to practical reasons: recording eye movements is relatively easy 6 , while, at the same time, eye movements can be highly informative about brain state 7,8 .…”

Section: Introductionmentioning

confidence: 99%

Human-level saccade detection performance using deep neural networks

Bellet

Nienborg

et al. 2018

Preprint

View full text Add to dashboard Cite

Saccades are ballistic eye movements that rapidly shift gaze from one location of visual space to another. Detecting saccades in eye movement recordings is important not only for studying the neural mechanisms underlying sensory, motor, and cognitive processes, but also as a clinical and diagnostic tool. However, automatically detecting saccades can be difficult, particularly when such saccades are generated in coordination with other tracking eye movements, like smooth pursuits, or when the saccade amplitude is close to eye tracker noise levels, like with microsaccades. In such cases, labeling by human experts is required, but this is a tedious task prone to variability and error. We developed a convolutional neural network (CNN) to automatically detect saccades at human-level performance accuracy. Our algorithm surpasses state of the art according to common performance metrics, and will facilitate studies of neurophysiological processes underlying saccade generation and visual processing.

show abstract

An inverse Yarbus process: Predicting observers’ task from eye movement patterns

Cited by 78 publications

References 85 publications

Human classifier: Observers can deduce task solely from eye movements

Human classifier: Observers can deduce task solely from eye movements

SubsMatch 2.0: Scanpath comparison and classification based on subsequence frequencies

Human-level saccade detection performance using deep neural networks

Contact Info

Product

Resources

About