Caveats for the spatial arrangement method: Comment on Hout, Goldinger, and Ferguson (2013).

Abstract

The gold standard among proximity data collection methods for multidimensional scaling is the (dis)similarity rating of pairwise presented stimuli. A drawback of the pairwise method is its lengthy duration, which may cause participants to change their strategy over time, become fatigued, or disengage altogether. Hout, Goldinger, and Ferguson (2013) recently made a case for the Spatial Arrangement Method (SpAM) as an alternative to the pairwise method, arguing that it is faster and more engaging. SpAM invites participants to directly arrange stimuli on a computer screen such that the interstimuli distances are proportional to psychological proximity. Based on a reanalysis of the Hout et al. (2013), data we identify three caveats for SpAM. An investigation of the distributional characteristics of the SpAM proximity data reveals that the spatial nature of SpAM imposes structure on the data, invoking a bias against featural representations. Individual-differences scaling of the SpAM proximity data reveals that the two-dimensional nature of SpAM allows individuals to only communicate two dimensions of variation among stimuli properly, invoking a bias against high-dimensional scaling representations. Monte Carlo simulations indicate that in order to obtain reliable estimates of the group average, SpAM requires more individuals to be tested. We conclude with an overview of considerations that can inform the choice between SpAM and the pairwise method and offer suggestions on how to overcome their respective limitations.

A Comparison of the Spatial Arrangement Method and the Total-Set Pairwise Rating Method for Obtaining Similarity Data in the Conceptual Domain

“…According to Hout et al (2013), the resolution that typical Likert scales offer is too limited for participants to convey their similarity perceptions. 1 Despite these concerns, average similarity data obtained with PRaM tend to be reliable (Bijmolt & Wedel, 1995;Giordano et al, 2011;Verheyen et al, 2016).…”

“…SpAM is not without disadvantages, however. Verheyen et al (2016) have formulated a number of caveats for the method. When the number of PRaM and SpAM participants is equated, the average SpAM similarity data tend to be less reliable.…”

SOLID-Similar object and lure image database

“…The spatial-arrangement method has been established as a fast and effective method for collecting similarity information Hout et al, 2013). However, it has previously been criticized for limiting the number of dimensions on which individuals can represent similarity (Verheyen, Voorspoels, Vanpaemel, & Storms, 2016). First, it is doubtful whether any of the current methods (e.g., spatial arrangement or exhaustive pairwise comparison) completely avoid this limitation .…”

Plainware and Polychrome: Quantifying Perceptual Differences in Ceramic Classification Between Diverse Groups to Further a Strong Objectivity

“…Even though the method has been critiqued for having caveats in comparison to asking for a pairwise evaluation of distances (Verheyen et al 2016), it has been shown to produce good results in a fraction of the time required for pairwise evaluation or similar methods, such as triad tests (Hout and Goldinger 2016;Hout, Goldinger and Ferguson 2013). As the field evaluators had to drive over 4,000 kilometres in a few weeks to connect with all of the study participants, time was an essential component.…”