Estimating psychometric functions in forced-choice situations: Significant biases found in threshold and slope estimations when small samples are used

Abstract

When a theoretical psychometric function is fitted to experimental data (as in the obtaining of a psychophysical threshold), maximum-likelihood or probit methods are generally used. In the present paper, the behavior of these curve-fitting methods is studied for the special case of forcedchoice experiments, in which the probability of a subject's making a correct response by chance is not zero. A mathematical investigation of the variance of the threshold and slope estimators shows that, in this case, the accuracy of the methods is much worse, and their sensitivity to the way data are sampled is greater, than in the case in which chance level is zero. Further, Monte Carlo simulations show that, in practical situations in which only a finite number of observations are made, the mean threshold and slope estimates are significantly biased. The amount of bias depends on the curve-fitting method and on the range of intensity values, but it is always greater in forced-choice situations than when chance level is zero.In many applications, it is necessary to determine the relation between the frequency of an event and the intensity of an independent variable. The psychometric function describing such a relation can be obtained with empirical methods such as the method of constant stimuli, staircase methods, the method of limits, and adaptive methods. If more than simply the threshold value of the psychometric function is required-as, for example, when the slope of the function is of interest-it is necessary to fit a theoretical curve (often a cumulative normal or logistic function) to the data points. The choice of a particular function will depend on the experimenter's hypotheses, but the techniques used to lit the curve remain the same irrespective of the function chosen. The most general curve-fitting method is the method of maximum likelihood, in which the parameters of the psychometric function chosen are such that it is the one most likely to have generated the observed data.Theoretical work on fitting psychometric functions (Berkson, 1955;Wetherill, 1963) has shown that the maximum-likelihood method is reliable and gives unbiased estimates of the psychometric function's parameters. But the work has been limited to situations in which chance level is zero-that is, in which the observable proportions of correct responses vary from 0 to 1.The first part of the present paper recalls these classic conclusions and extends them to the case in which chance level is not zero. This is the case for two-alternative (or Correspondence may be addressed to

Adaptive psychophysical procedures

“…If two parameters are estimated, i.e. threshold and slope of the psychometric function, O'Regan and Humbert (1989) found that small samples (100 data points simulated with n = 10 number of presentations at N = 10 stimulus values, method of constant stimuli) produce both, low precision and biased estimates. These results were obtained for either maximum-likelihood and probit analysis and they are in accordance with the study of McKee et al (1985).…”

Sampling Plans for Fitting the Psychometric Function

“…O'Regan and Humbert (1989; see also Brand & Kollmeier, 2002) carried out a theoretical analysis elucidating the locations where three levels should be placed to minimize the asymptotic variance of maximum likelihood estimates of threshold and slope in the simplest case. Hill (2001a, 2001b) used simulations to evaluate seven MOCS plans that differed as to the spacing of six stimulus levels and their location relative to that of Ψ, showing that some of these plans are more prone to bias or imprecise estimation.…”

“…The tight association between estimating Ψ and using MOCS perhaps developed due to lack of alternative methods, but the link currently continues to be reinforced by the fact that even the most recent explicit attempts to determine how best to estimate Ψ either have only compared variants of MOCS or have only proposed or evaluated statistical criteria or algorithms to deal with MOCS data (Foster & Bischof, 1991;Lam, Mills, & Dubno, 1996;Maloney, 1990;Miller & Ulrich, 2001;O'Regan & Humbert, 1989;Treutwein & Strasburger, 1999;Wichmann & Hill, 2001a, 2001b.…”

Robust and efficient Bayesian adaptive psychometric function estimation

“…When wishing to estimateã andb jointly, Wetherill (1963) proposed minimizing the product of their variances leading to sweetpoints satisfying WðxÞ 2 f0:176; 0:824g; he found that there was no benefit in probing at WðxÞ ¼ 0:5 in addition to these two sweetpoints. O'Regan and Humbert (1989) generalized this work and determined the sweetpoint locations for estimatingã and/orb when c 6 ¼ 0 in Eq. (1).…”