Preferential Processing of Social Features and Their Interplay with Physical Saliency in Complex Naturalistic Scenes

Abstract

According to so-called saliency-based attention models, attention during free viewing of visual scenes is particularly allocated to physically salient image regions. In the present study, we assumed that social features in complex naturalistic scenes would be processed preferentially irrespective of their physical saliency. Therefore, we expected worse prediction of gazing behavior by saliency-based attention models when social information is present in the visual field. To test this hypothesis, participants freely viewed color photographs of complex naturalistic social (e.g., including heads, bodies) and non-social (e.g., including landscapes, objects) scenes while their eye movements were recorded. In agreement with our hypothesis, we found that social features (especially heads) were heavily prioritized during visual exploration. Correspondingly, the presence of social information weakened the influence of low-level saliency on gazing behavior. Importantly, this pattern was most pronounced for the earliest fixations indicating automatic attentional processes. These findings were further corroborated by a linear mixed model approach showing that social features (especially heads) add substantially to the prediction of fixations beyond physical saliency. Taken together, the current study indicates gazing behavior for naturalistic scenes to be better predicted by the interplay of social and physically salient features than by low-level saliency alone. These findings strongly challenge the generalizability of saliency-based attention models and demonstrate the importance of considering social influences when investigating the driving factors of human visual attention.

Overt attentional correlates of scene memorability and their relationships to scene semantics

“…Regarding the presence of face/human (Fig. 5), previous research showed that people prioritize their attention to faces, bodies, and other people (i.e., social features) in naturalistic scenes where they could obtain important social information (Bindemann et al, 2010;Cerf et al, 2009;End & Gamer, 2017;Flechsenhar & Gamer, 2017;Scrivner et al, 2019), suggesting scenes with informative and salient scene features, such as faces and people, may produce higher fixation map consistency than scenes without those. For example, Wilming and colleagues (2011) showed that fixation map consistency was higher in urban scenes than in nature scenes and explained that urban scenes have more people and concrete man-made objects, which are more likely to attract fixations.…”

Been there, seen that, done that: Modification of visual exploration across repeated exposures

“…Importantly, this study was designed to explore the impact of familiarity on two types of gaze behavior: content-free (i.e., deployment of fixations and saccades independently of the content of the viewed image) and content-based (deployment of fixations towards specific regions of the image based on their semantic and visual properties). Recently, a growing body of research has underscored the importance of high-level image content during visual exploration by showing an interaction between the semantic properties of the image and gaze position (End & Gamer, 2017;Henderson & Hayes, 2017). While these studies provide convincing arguments for the influence of image meaning during visual explora-tion, they do not address how gaze predilection towards semantic properties is modulated by familiarity.…”

The reward of seeing: Different types of visual reward and their ability to modify oculomotor learning

“…Social stimuli are unique in their significance for survival, which is why they might trigger a more automatic processing than nonsocial stimuli (Greene, Mooshagian, Kaplan, Zaidel, & Iacoboni, 2009). Orientation to social stimuli in a scene is well documented (Bindemann et al, 2007;Downing et al, 2004;Downing et al, 2001;End & Gamer, 2017;Hershler & Hochstein, 2005;Ro et al, 2007;Rothkopf et al, 2007) and occurs from a very young age on (de Schonen et al, 1989;Johnson et al, 1991). Therefore, the social content of our images may constitute a form of primary reinforcement to the visual system.…”

“…When viewing a scene special attention is given to features of human bodies or faces, often triggering an orientation or approach response (Bindemann, Burton, Langton, Schweinberger, & Doherty, 2007;Downing, Bray, Rogers, & Childs, 2004;Downing, Jiang, Shuman, & Kanwisher, 2001;Hershler & Hochstein, 2005;Ro, Friggel, & Lavie, 2007;Rothkopf, Ballard, & Hayhoe, 2007). Humans orient to other humans even in the presence of other more salient features (End & Gamer, 2017) and from a very young age (de Schonen, Mathivet, & Deruelle, 1989;Johnson, Dziurawiec, & Morton, 1991). Therefore, the orientation toward images of humans might be a primary mechanism.…”