Classification images reveal decision variables and strategies in forced choice tasks

Abstract

Despite decades of research, there is still uncertainty about how people make simple decisions about perceptual stimuli. Most theories assume that perceptual decisions are based on decision variables, which are internal variables that encode task-relevant information. However, decision variables are usually considered to be theoretical constructs that cannot be measured directly, and this often makes it difficult to test theories of perceptual decision making. Here we show how to measure decision variables on individual trials, and we use these measurements to test theories of perceptual decision making more directly than has previously been possible. We measure classification images, which are estimates of templates that observers use to extract information from stimuli. We then calculate the dot product of these classification images with the stimuli to estimate observers' decision variables. Finally, we reconstruct each observer's "decision space," a map that shows the probability of the observer's responses for all values of the decision variables. We use this method to examine decision strategies in two-alternative forced choice (2AFC) tasks, for which there are several competing models. In one experiment, the resulting decision spaces support the difference model, a classic theory of 2AFC decisions. In a second experiment, we find unexpected decision spaces that are not predicted by standard models of 2AFC decisions, and that suggest intrinsic uncertainty or soft thresholding. These experiments give new evidence regarding observers' strategies in 2AFC tasks, and they show how measuring decision variables can answer long-standing questions about perceptual decision making.any current questions about human cognition are related to how people make decisions, including decisions based on perceptual information. For example, how do we decide whether a search target is present in a cluttered display? How do we decide when to respond in a task where both speed and accuracy are important? How do we judge which of two signals is present in a discrimination task?Most theories of perceptual decision making rely on the notion of a decision variable, a quantity that the observer calculates from the stimulus to summarize task-relevant information, e.g., the probability that a faint signal is present in a detection task (1). Some theories of decision making are very simple, e.g., the observer gives one response if the decision variable is greater than a fixed criterion, and another response if the decision variable is less than the criterion. Other theories use more complex decision rules. Testing theories of decision making would be much easier if we had access to observers' decision variables, but these are usually thought of as theoretical constructs that cannot be measured psychophysically. Here we show that in some tasks, it is possible to estimate decision variables on individual trials, and this provides a very direct way of testing theories of perceptual decision making. We use this method to examine the long-sta...

Object segmentation controls image reconstruction from natural scenes

“…If we adopt a minimal SDT model [ 35 ] ( Fig 8A ) whereby orientation energy within the probe is processed by an orientation-selective filter [ 8 , 70 ], a source of intrinsic variability is added to the filter response [ 37 ], and an output binary decision is produced [ 71 ], there are 3 fundamentally distinct ways in which the sensitivity of this mechanism may be enhanced: 1) by reducing its internal noise (red in Fig 8A ); 2) by sharpening filter tuning around the congruent signal (blue thick line in A ); 3) by sharpening filter tuning around the incongruent signal (blue thin line). We sought to determine which of these alternatives apply.…”

Accuracy of Rats in Discriminating Visual Objects Is Explained by the Complexity of Their Perceptual Strategy

“…This marks the third difference from previous rat studies—we went beyond an image-level analysis of the classification images, and we explicitly tested their ability to account for rat visual perception. As such, our approach adds to a growing body of studies combining classification images with other computational methods to predict perceptual decisions [ 30 , 41 , 42 , 43 ] and infer visual processing mechanisms from behavioral and neurophysiological data [ 37 , 43 , 44 , 45 , 46 ]. Future behavioral studies could take inspiration from these approaches to investigate aspects of rodent vision that were beyond the scope of our experiments—e.g., the representation of object information in different spatial frequency bands could be explored, by relying on multi-resolution generative or filtering manipulations [ 34 , 44 , 47 ].…”

The empirical characteristics of human pattern vision defy theoretically-driven expectations

“…scalar [24]). There is good experimental evidence to support the difference rule, at least for low-level perceptual tasks [31, 32]. The second assumption is stronger, but it should not be interpreted literally: it neither implies that intrinsic noise originates from only one source, nor that the physiological action of said source is additive.…”

“…All models generate psychophysical decisions via the difference rule [32] detailed earlier, and apply front-end filter f to the stimulus. f is sampled at the same resolution used for the stimulus, i.e.…”