Simulating Fixations When Looking at Visual Arts

Abstract

When people look at pictures, they fixate on specific areas. The sequences of such fixations are so characteristic for certain pictures that metrics can be derived that allow successful grouping of similar pieces of visual art. However, determining enough fixation sequences by eye tracking is not practically feasible for large groups of people and pictures. In order to get around this limitation, we present a novel algorithm that simulates eye movements by calculating scan paths for images and time frames in real time. The basis of our algorithm is an attention model that combines and optimizes rectangle features with Adaboost. The model is adapted to the characteristics of the retina, and its input is dependent on a few earlier fixations. This method results in significant improvements compared to previous approaches. Our simulation process delivers the same data structures as an eye tracker, thus can be analyzed by standard eye-tracking software. A comparison with recorded data from eye tracking experiments shows that our algorithm for simulating fixations has a very good prediction quality for the stimulus areas on which many subjects focus. We also compare the results with those from earlier works. Finally, we demonstrate how the presented algorithm can be used to calculate the similarity of pictures in terms of human perception.

A Multi-Scale Colour and Keypoint Density-Based Approach for Visual Saliency Detection

“…Some methods such as in [58] and [59] approach the object detection using image descriptors and the relative orientation of the camera. Pflüger et al [60] managed to build a method based on the optimisation of rectangle features with Adaboost to simulate eye-movements when one looks at visual artworks. Krejtz et al [61] focused their efforts on quantifying the eye movement transitions between different areas of interest using Shannon's entropy and Markov chains.…”

A language to analyze, describe, and explore collections of visual art

“…Bell et al [ 27 ] presented a procedure, which is mainly based on automatic object recognition in images; however, for interactive use alongside large number of images, the computation times of the typical object recognition methods used in that work were too long. Pflüger et al [ 28 ] presented a method for searching related images in large sets of images. The computing time of this method is sufficiently fast for the given application, but the method is relatively experimental and does not consider information given by metadata/annotations.…”

“…A brief overview is given in “Comparison functions” section. The two exceptions are the determination of characteristic areas in images using simulated fixations [ 28 ] and the detection and use of line elements as image features in addition to pixel information [ 7 ].…”

“…Such commonly used methods are designed for object recognition and require too much computation time the purposes of LadeCA. Therefore, algorithms tailored especially to LadeCA were developed: algorithms similar to convolutional neural networks [ 28 , 39 ] to determine characteristic areas in images and algorithms for matching these characteristic areas between two images. These methods are usually conducted pixel-wise, but it is also possible to detect lines in images and then use them for image comparisons [ 7 , 40 ].…”

“…), we can see refs. [ 7 , 28 , 41 – 44 ]. Experiments that compare global image feature-based classifications with local image feature (and corresponding spatial distribution)-based classifications are reported in refs.…”