Visual Attention for Rendered 3D Shapes

Abstract

Understanding the attentional behavior of the human visual system when visualizing a rendered 3D shape is of great importance for many computer graphics applications. Eye tracking remains the only solution to explore this complex cognitive mechanism. Unfortunately, despite the large number of studies dedicated to images and videos, only a few eye tracking experiments have been conducted using 3D shapes. Thus, potential factors that may influence the human gaze in the specific setting of 3D rendering, are still to be understood. In this work, we conduct two eye‐tracking experiments involving 3D shapes, with both static and time‐varying camera positions. We propose a method for mapping eye fixations (i.e., where humans gaze) onto the 3D shapes with the aim to produce a benchmark of 3D meshes with fixation density maps, which is publicly available. First, the collected data is used to study the influence of shape, camera position, material and illumination on visual attention. We find that material and lighting have a significant influence on attention, as well as the camera path in the case of dynamic scenes. Then, we compare the performance of four representative state‐of‐the‐art mesh saliency models in predicting ground‐truth fixations using two different metrics. We show that, even combined with a center‐bias model, the performance of 3D saliency algorithms remains poor at predicting human fixations. To explain their weaknesses, we provide a qualitative analysis of the main factors that attract human attention. We finally provide a comparison of human‐eye fixations and Schelling points and show that their correlation is weak.

Mesh Saliency via Weakly Supervised Classification-for-Saliency CNN

“…According to their recommendations, we selected the Pearson's Linear Correlation Coefficient (CC), computed as CC(H, T ) = cov(H, T ) σ H σ T (12) where H and T are the saliency maps produced by a competing method and the ground truth respectively. We also selected the area under the ROC curve (AUC) suggested in [6]. The ground truth maps are thresholded to be converted into binary maps (in our experiments we threshold to obtain M vertices considered as salient vertices and M is equal to the number of human-selected Schelling points on the 3D object).…”

“…Another reason for choosing the Schelling dataset is that it is much larger than existing eye fixation datasets. For example, the dataset proposed in [6] contains 32 meshes and the one proposed in [18] merely contains 5 meshes. As one of the aims is to demonstrate the generalisation capability of our approach, experiments on a larger dataset are more desired.…”

“…We noticed that all of the existing datasets of mesh saliency are very small (e.g. 400 meshes in [5], 79 meshes in [3], 150 meshes in [4] and 32 meshes in [6]). On the other hand, a neural network has to be sufficiently deep to generalise well since learning features at various levels of abstraction requires a sufficiently large number of layers.…”

“…However, as shown in quite a few user studies (e.g. [3], [4], [5], [6]), gathering such data is a difficult task. We noticed that all of the existing datasets of mesh saliency are very small (e.g.…”

On the usage of visual saliency models for computer generated objects

“…As a matter of fact, the viewing information is not included as a parameter neither in the annotated datasets (used to train models) such as MIT300 [8], CAT 2000 [9] that were established for the MIT saliency Benchmark (saliency.mit.edu) nor in the computational models themselves. -Since the few existing 3D models consider geometry information only without texture or shading [10], applying them in an immersive environment is very restricted (because of the lack of texture for example). On the other hand, several promising 2D models [11] that showed high performances could be applied in the immersive context by considering 2D projection views of 3D data, rendered by a specific rule.…”