Individual differences in visual salience vary along semantic dimensions

Abstract

150 words; Main text: 2615 words (excl. Methods, fig. and table captions); Methods: 17 1526 words; 5 figures:1 table; 1 supplementary figure 18 19 20 21 22 23 features and task demands govern fixation behaviour, while differences between observers 24 are 'noise'. Here, we investigated the fixations of > 100 human adults freely viewing a large 25 set of complex scenes. We found systematic individual differences in fixation frequencies 26 along six semantic stimulus dimensions. These differences were large (> twofold) and 27 highly stable across images and time. Surprisingly, they also held for first fixations directed 28 towards each image, commonly interpreted as 'bottom-up' visual salience. Their 29 perceptual relevance was documented by a correlation between individual face salience and 30 recognition skills. The dimensions of individual salience and their covariance pattern 31 replicated across samples from three different countries, suggesting they reflect 32 fundamental biological mechanisms of attention. Our findings show stable individual 33 salience differences along semantic dimensions, with meaningful perceptual implications. 34 Salience reflects features of the observer as well as the image. 35 36 image discontinuities of low level attributes, such as luminance, colour and orientation 13 . These 45 low-level models are inspired by 'early' visual neurons and their output correlates with neural 46 responses in subcortical 14 and cortical 15 areas thought to represent neural 'salience maps'. 47However, while these models work relatively well for impoverished stimuli, human gaze 48 behaviour towards richer scenes can be predicted at least as well by the locations of objects 16 and 49 perceived meaning 9 . When sematic object properties are taken into account, their weight for gaze 50 prediction far exceeds that of low-level attributes 8,17 . A common thread of low-and high-level 51 salience models is that they interpret salience as a property of the image and treat inter-individual 52 differences as unpredictable 7,18 , often using them as a 'noise ceiling' for model evaluations 18 . 53 However, even the earliest studies of fixation behaviour noted considerable individual 54 differences 19,20 and basic occulomotor traits, like mean saccadic amplitude and velocity, reliably 55 vary between observers 21-27 . Moreover, recent twin-studies revealed that social attention and 56 gaze traces across complex scenes are highly heritable 28,29 . This suggests individual differences 57 in fixation behaviour are not random, but systematic. However, it is largely unclear, how 58 individuals differ in their fixation behaviour and what may explain these differences. Can 59 individual fixation behaviour be captured along a limited set of dimensions? 60Here, we tested the hypothesis that individual gaze reflects individual salience differences 61 along a limited number of semantic dimensions. We investigated the fixation behaviour of > 100 62 human adults freely viewing 700 complex scenes, containing thousands of semantica...

OSIEshort: A small stimulus set can reliably estimate individual differences in semantic salience

“…Dots inside the distributions refer to the median correlation for each histogram and semantic dimension. Desaturated green scatterplots (A-D) and right-hand leaves (F) for the dimensions Motion and Taste are shown for completeness and were not preregistered (de Haas et al, 2019).…”

“…This was repeated across 1000 random splits of images. We calculated two-sided p values for the median correlations across these splits, which were Bonferroni adjusted for the number of tested dimensions: Faces, Text, Touched, Taste, and Motion for cumulative dwell times and Faces, Text, and Touched for the proportion of first fixations, as preregistered (de Haas et al, 2019). Finally, we calculated the minimum/maximum ratios of individual fixations between observers and for each dimension.…”

“…Such semantic salience biases may be useful in the study of neurobiological mechanisms of attentional gaze control (de Haas et al, 2019) and form a crucial baseline for evaluating the diagnostic potential of gaze behavior for neurodevelomental and clinical conditions. In fact, research investigating eye movements in autism spectrum disorder (ASD) has demonstrated reduced social visual engagement in infants and adults with ASD compared to healthy controls (Constantino et al, 2017;Jones & Klin, 2013;Wang, Jiang, Duchesne, Laugeson, Kennedy, Adolphs, & Zhao, 2015).…”

“…Further, we tested whether the proportion of first fixations varies reliably for objects of the categories Text, Faces, and Touched. We chose and preregistered these object dimensions, because they yielded large and consistent individual gaze differences (r > 0.6) in the original study by de Haas et al (2019). Note, we combined neutral and emotional Faces, given the high covariance in fixation tendencies toward both.…”

Not all fixations are created equal: The benefits of using ex-Gaussian modeling of fixation durations

“…Early studies of complex scene viewing (Buswell, 1935;Yarbus, 1967) examined how gaze position is influenced by instructions to observers and their level of expertise. More recent studies have expanded on these findings by examining how low-level features of visual input (e.g., Foulsham & Underwood, 2008;Itti, 2005;;Itti & Koch, 2000;Itti, Koch, & Niebur, 1998;Parkhurst, Law, & Niebur, 2002) and high-level observer factors (e.g., De Haas, Iakovidis, Schwarzkopf, & Gegenfurtner, 2019;Guy et al, 2019) modify where gaze is directed. In addition to this stream of works, which has focused on the spatial characteristics of fixations, other studies have aimed to decipher the underlying processes that determine the temporal aspect of fixations; that is, how long gaze remains relatively stable before it is shifted to a new location (i.e., fixation duration).…”