Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning

Abstract

Being able to predict human gaze behavior has obvious importance for behavioral vision and for computer vision applications. Most models have mainly focused on predicting free-viewing behavior using saliency maps, but these predictions do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. The viewer's internal belief states were modeled as dynamic contextual belief maps of object locations. These maps were learned by IRL and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goaldirected fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive target-dependent patterns of object prioritization, which we interpret as a learned object context.

Weighting the factors affecting attention guidance during free viewing and visual search: The unexpected role of object recognition uncertainty

“…02) to obtain the target map, again assuming that there are some features at the bounding box locations that are guiding attention in proportion to their target similarity. Note that, whereas more sophisticated methods have been developed for predicting search fixations ( Yang et al, 2020 ; Zelinsky et al, 2020 a), we thought it best to err on the side of interpretability when selecting a method for obtaining a target map, which is often a problem for more sophisticated deep-learning methods. Our implementation of a target map is a simple bias much like a center bias, only the bias is introduced at the detected target locations.…”

“…After filtering out these object categories, which we did by using the corresponding MaskRCNN channels to detect these categories in the images, we were left with 145 images for analysis. Surprisingly few datasets have been developed for visual search behavior, but by far the largest is COCO-Search18 ( Chen, Yang, Ahn, Samaras, Hoai, & Zelinsky, 2021 ; Yang et al, 2020 ). It consists of roughly 300,000 fixations from 10 people searching for each of 18 target-object categories in 6202 images of natural scenes.…”

“…For even greater specificity, we refer to priority maps in the context of a search task as “target maps” ( Zelinsky, 2008 ), a name that makes clear that the prioritization is based on a comparison of a visual input to features of a target goal. We adopted and consistently used this terminology in several recent studies ( Yang et al, 2020 ; Zelinsky et al, 2020 b), most clearly defined in Zelinsky and Bisley (2015 ), and we believe that these distinctions are particularly useful given our present goal of better understanding how different attention biases are weighted in the context of free-viewing and search tasks. In our view, when an assumption of top–down input is made, even in cases of simple text and face detection ( Liao, Shi, Bai, Wang, & Liu, 2017 ; Boyko, Basystiuk, & Shakhovska, 2018 ; Long, Ruan, Zhang, He, Wu, & Yao, 2018 ), a mixture of priority signals occurs that makes it challenging to compare models, with model performance often correlating with how much top–down input can be added to the prediction.…”

“…This extension of search theory to object classification was significant in the bridge that it built to computer vision, where powerful methods have been developed for extracting and detecting objects in images by learning robust object category representations. Using this bridge, the recent models that predict search fixations are all deep networks ( Wei, Adeli, Nguyen, Zelinsky, & Samaras, 2016 ; Adeli & Zelinsky, 2018 ; Zhang, Feng, Ma, Lim, Zhao, & Kreiman, 2018 ), with the current state of the art being a model that predicts search fixations using a prioritization policy learned through imitation of previously observed search behavior ( Yang et al, 2020 ). The dataset of search fixations that we use in the present study, COCO-Search18, was developed to provide the observations of search behavior needed by this model for training ( Chen et al, 2020 ).…”