The Use of Paired Comparisons for Evaluating Complex Route Matching Performance in a Spatial Awareness Task

Abstract

A method is proposed for evaluating participants' performance in a global spatial awareness task involving identification of complex winding routes. Rather than using coarse measures of spatial error, the method of paired comparisons employs impartial judges to compare sets of aggregated experimental data generated by the participants with respect closeness in shape to the target route. The method was applied to a set of data in an experimental investigation of the effect of height on a participant's ability to identify the route he had just flown over (in a 20 second video). Seven participants performed a total of 48 trials in a 4 (heights) X 2 (trial blocks) within-subjects experiment. Results from the paired comparison analysis suggested that height had a statistically significant effect on correct route identification. Of equal importance in the context of the present paper, the method of paired comparison analysis proved to be effective in quantifying performance data that did not otherwise lend themselves to conventional methods of quantification.

The Effects of Object Shape, Fidelity, Color, and Luminance on Depth Perception in Handheld Mobile Augmented Reality

“…The paired comparison method was first introduced by Thurstone in 1927 [56] and has since been expanded on several times [48,49]. It has been used in many experiments that analyze visual stimuli with some examples being [3,27,28,43,58]. It has also been used to analyze depth perception in particular [3,28].…”