Estimating the difference limen in 2AFC tasks: Pitfalls and improved estimators

Abstract

Discrimination performance is often assessed by measuring the difference limen (DL; or just noticeable difference) in a two-alternative forced choice (2AFC) task. Here, we show that the DL estimated from 2AFC percentage-correct data is likely to systematically under-or overestimate true discrimination performance if order effects are present. We show how pitfalls with the 2AFC task may be avoided and suggest a novel approach for analyzing 2AFC data.

Reminder and 2AFC tasks provide similar estimates of the difference limen: A reanalysis of data from Lapid, Ulrich, and Rammsayer (2008) and a discussion of Ulrich and Vorberg (2009)

“…Indeed, the 2AFC discrimination task has a peculiarity relevant only to the experimenter, who must be aware that "probability correct" as a label for the ordinate of the psychometric function of Equation 1 indicates "probability that the comparison is reported to have a larger magnitude" and not "probability that the observer reported the stimulus whose magnitude was actually larger." The reason is that a psychometric function that abides by the latter definition is not monotonic, as are those derived by Ulrich and Vorberg (2009). On this caveat, all theoretical analyses (e.g., Foley & Schwarz, 1998;Werkhoven & Snippe, 1996) concur in showing that the psychometric function that holds in 2AFC discrimination tasks with become higher than they should be.…”

“…They noted that the response r is typically "comparison longer," and they also correctly acknowledged that a psychometric function "must satisfy three conditions: (1) F(s) 0 as s , (2) F(s) 1 as s , and (3) F(s) is monotonically increasing with s." However, a subsequent article by Ulrich and Miller (2004) was entirely based on the mistake of considering that the psychometric function for discrimination must have a lower asymptote at .5, a mistake perhaps arising from the true fact that the chance level in a 2AFC task is .5. In reality, the latter fact implies only that the psychometric function for discrimination is nonmonotonic and has a discontinuity at the PSE when regarded as expressing the probability of a correct response in the strict physical sense (i.e., the probability of reporting the stimulus whose physical magnitude is actually larger; see Ulrich & Vorberg, 2009). The mathematical inconvenience that nonmonotonicity brings around is generally avoided by considering the psychometric function to express instead the probability that the comparison stimulus is reported to have a larger magnitude, thus satisfying the three assumptions listed by Miller and Ulrich. To illustrate the differences between Equations 1 and 2, consider the data in Figure 5 of Lapid et al (2008), which we present here in Figure 1A.…”

“…This model is shown to account for the pattern of results across Lapid et al's experiments, and it also prescribes how to conduct discrimination experiments so as to eliminate the contaminating effects of response bias. We finally discuss the utility of a novel approach described by Ulrich and Vorberg (2009) for fitting psychometric functions to 2AFC discrimination data that are potentially affected by response bias.…”

“…An article by Ulrich and Vorberg (2009) discussing what form the psychometric function fitted to 2AFC discrimination data should have was published after the first draft of this article had passed the first review round. The material presented by Ulrich and Vorberg has obvious bearings on the issues addressed in this article, and it is worth commenting on them here.…”

“…We also show that their data are contaminated by response bias, and that the small remaining difference between DLs estimated with 2AFC and reminder tasks can be reasonably attributed to the differential effects that response bias has in either task as they were defined in Lapid et al's experiments. Finally, we discuss a novel approach presented by Ulrich and Vorberg (2009) for fitting psychometric functions to 2AFC discrimination data. explained in the next section, this definition of the DL and the scaled form of the psychometric function that was fitted to 2AFC data are both inadequate.…”