Decision processes in discrimination: Fundamental misrepresentations of signal detection theory.

Abstract

A new approach to studying decision making in discrimination tasks is described that does not depend on the technical assumptions of signal detection theory (e.g., normality of the encoding distributions). In 3 different experiments, results of these new distribution-free tests converge on a single, surprising conclusion: response biases had substantial effects on the encoding distributions but no effect on the decision rule, which was uniformly unbiased in equal and unequal base rate conditions and in symmetric and asymmetric payoff conditions. This seemingly paradoxical result is fundamentally inconsistent with the entire family of signal detection theory models, raising some important questions about the significance of many published results in the human performance literature.

Is signal detection theory fundamentally flawed? A response to Balakrishnan (1998a, 1998b, 1999)

“…Inspection of Balakrishnan's (1998aBalakrishnan's ( , 1998bBalakrishnan's ( , 1999) U R (k) curves confirms that they are generally flat-topped, much like the SDT predictions in Figure 5. Nevertheless, from such evidence, Balakrishnan (1999) has concluded that the decision rule in his experiments was uniformly unbiased despite payoff and base rate manipulations.…”

“…Nevertheless, from such evidence, Balakrishnan (1999) has concluded that the decision rule in his experiments was uniformly unbiased despite payoff and base rate manipulations. He concludes that "this seemingly paradoxical result is fundamentally inconsistent with the entire family of signal detecton theory models, raising some important questions about the significance of many published results" (Balakrishnan, 1999(Balakrishnan, , p. 1189. The logic of this argument is that of accepting the null hypothesis.…”

“…Thus, a biased criterion (x c shifted to the right) is indicated by a negative slope to the left of x c or (if x c is shifted to the left) by a positive slope to the right of x c . This observation is the foundation for Balakrishnan's (1998aBalakrishnan's ( , 1998bBalakrishnan's ( , 1999 first test of bias. In experiments in which a payoff or frequency manipulation was used that, according to SDT, should shift the criterion to a biased position, the author claims that displacements of x c from the U R (k) peak did not occur.…”

“…Balakrishnan (1998aBalakrishnan ( , 1998bBalakrishnan ( , 1999 has developed three new related measures or tests that he uses to examine whether response bias occurs in discrimination, using the rating procedure. He claims that these show that SDT is fundamentally flawed: Its central assumption, that manipulating payoffs or probabilities can shift the decision criterion, is invalid.…”

“…The account it provides of the processing of sensory information and the operation of the sensory decision-making process is now built into the fabric of much of experimental psychology. In three recent papers, Balakrishnan (1998aBalakrishnan ( , 1998bBalakrishnan ( , 1999 has claimed, however, that an alternative index is "better justified" than d9 and that SDT is fundamentally flawed and must be rejected. His argument is based on the use of new measures that he has developed and applied experimentally.…”