A Bayesian model of visual search in natural images and the role of memory

Abstract

Modelling human eye movements during visual search has been a research topic of interest for many years. Here, we investigate if a Bayesian ideal observer model can be configured to effectively explain human eye movements when searching for a known object target in natural images. We collected eye movements from participants who searched for a known target in 18 different natural textured images and compared their performance and strategy to the ideal observer model. Our model chooses search fixations that maximize information gain, given a probabilistic model of target detectability, at various eccentricities which is in turn approximated from features of a convolutional neural network. We collect and use ground truth data from a second experiment with human observers (observers detecting a target at various eccentricities on the 18 natural textured backgrounds) to fine-tune the parameters of our proposed pipeline for detectability estimation and verify our results. We further study the role of short-term memory in the ideal observer model and find that a model with limited memory best fits the human search patterns. Our results suggest that the visual system's strategy for choosing next fixation location in a visual search task is consistent with a Bayesian ideal observer model.

Extended Expectation Maximization for Under-Fitted Models

“…The main reason for the deficient localization in SOD stems from the limited information provided in the input image or video frame, compounded by the subsequent spatial degradation experienced as they pass through multiple layers in deep networks. Since small objects frequently appear in various application domains, such as pedestrian detection [3], medical image analysis [4], face recognition [5], traffic sign detection [6], traffic light detection [7], ship detection [8], Synthetic Aperture Radar (SAR)-based object detection [9], it is worth examining the performance of modern deep learning SOD techniques. In this paper, we compare transformer-based detectors with Convolutional Neural Networks (CNNs) based detectors in Aref Miri Rekavandi and Mohammed Bennamoun are with the Department of Computer Science and Software Engineering, The University of Western Australia (Emails: aref.mirirekavandi@uwa.edu.au, mohammed.bennamoun@uwa.edu.au).…”