Using multidimensional scaling to quantify similarity in visual search and beyond

Abstract

Visual search is one of the most widely studied topics in vision science, both as an independent topic of interest, and as a tool for studying attention and visual cognition. A wide literature exists that seeks to understand how people find things under varying conditions of difficulty and complexity, and in situations ranging from the mundane (e.g., looking for one’s keys) to those with significant societal importance (e.g., baggage or medical screening). A primary determinant of the ease and probability of success during search are the similarity relationships that exist in the search environment, such as the similarity between the background and the target, or the likeness of the non-targets to one another. A sense of similarity is often intuitive, but it is seldom quantified directly. This presents a problem in that similarity relationships are imprecisely specified, limiting the capacity of the researcher to examine adequately their influence. In this article, we present a novel approach to overcoming this problem that combines multidimensional scaling (MDS) analyses with behavioral and eye-tracking measurements. We propose a method whereby MDS can be repurposed to successfully quantify the similarity of experimental stimuli, thereby opening up theoretical questions in visual search and attention that cannot currently be addressed. These quantifications, in conjunction with behavioral and oculomotor measures, allow for critical observations about how similarity affects performance, information selection, and information processing. We provide a demonstration and tutorial of the approach, identify documented examples of its use, discuss how complementary computer vision methods could also be adopted, and close with a discussion of potential avenues for future application of this technique.

Dual-target cost in visual search for multiple unfamiliar faces.

“…We used the spatial arrangement method (SpAM, Hout, Goldinger & Ferguson, 2013;Hout, Godwin, Fitzsimmons, Robbins, Menneer, & Goldinger, 2016) and multidimensional scaling to produce a plot of similarity between faces in the set to create distractors with steps of similarity to the target faces. Distractors were not morphs related to targets but faces of different identities.…”

“…In some circumstances, such as when searching for colour targets, overall speed decreases and accuracy falls when searching for two targets compared with single-target search, but both targets can be found with accuracy levels above chance. Indeed, much of the previous research on searching for multiple targets suggests that separate mental representations of all targets can be maintained successfully (Barrett & Zobay, 2014;Beck, Hollingworth, & Luck, 2012;Grubert & Eimer, 2015, 2016Irons, Folk, & Remington, 2012;Wolfe, 2012).…”

Dwelling on simple stimuli in visual search

“…While the importance of using more ecologically relevant common stimulus categories (e.g., faces or scenes) in studies of visual search have been substantiated recently (e.g., Alexander & Zelinsky, 2012;Einhäuser & Nuthmann, 2016), research has revealed differences in the processing of artificial and realistic stimuli (e.g., Jenkins, Grubert, & Eimer, 2018;Godwin, Walenchkok, Houpt, Hout, & Goldinger, 2015;Neider & Zelinsky, 2006;Zelinsky & Schmidt, 2009). Second, while we successfully induced differences in search efficiency with different target categories in the previous studies, with natural stimuli it is not possible to actively manipulate theoretically important dimensionsin particular, target-distractors similarity (but see Hout et al, 2016, for procedures to measure target-distractor similarity).…”

Psychological responses to natural patterns in architecture

“…First, naïve participants assessed the similarity of diverse architectural images in an image arrangement task referred to as the spatial arrangement method (SpAM; Hout, Goldinger, & Ferguson, 2013). We then applied multidimensional scaling analysis (MDS) to these similarity data to identify the underlying aesthetic dimensions that drove participants' grouping decisions (Berman et al, 2014;Hout et al, 2015;Hout, Papesh, et al, 2013;Shepard, 1980), predicting that latent perceptions of naturalness would strongly predict image grouping behavior. We tested this prediction by regressing dimension weights from the MDS analysis on subjective naturalness ratings collected in the first experiment.…”