Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties

Abstract

Visual detection and discrimination thresholds are often measured using adaptive staircases, and most studies use transformed (or weighted) up/down methods with fixed step sizes--in the spirit of Wetherill and Levitt (Br J Mathemat Statist Psychol 1965;18:1-10) or Kaernbach (Percept Psychophys 1991;49:227-229)--instead of changing step size at each trial in accordance with best-placement rules--in the spirit of Watson and Pelli (Percept Psychophys 1983;47:87-91). It is generally assumed that a fixed-step-size (FSS) staircase converges on the stimulus level at which a correct response occurs with the probabilities derived by Wetherill and Levitt or Kaernbach, but this has never been proved rigorously. This work used simulation techniques to determine the asymptotic and small-sample convergence of FSS staircases as a function of such parameters as the up/down rule, the size of the steps up or down, the starting stimulus level, or the spread of the psychometric function. The results showed that the asymptotic convergence of FSS staircases depends much more on the sizes of the steps than it does on the up/down rule. Yet, if the size delta+ of a step up differs from the size delta- of a step down in a way that the ratio delta-/delta+ is constant at a specific value that changes with up/down rule, then convergence percent-correct is unaffected by the absolute sizes of the steps. For use with the popular one-, two-, three- and four-down/one-up rules, these ratios must respectively be set at 0.2845, 0.5488, 0.7393 and 0.8415, rendering staircases that converge on the 77.85%-, 80.35%-, 83.15%- and 85.84%-correct points. Wetherill and Levitt's transformed up/down rules--which require delta-/delta+ = 1--and the general version of Kaernbach's weighted up/down rule--which allows any delta-/delta+ ratio--fail to reach their presumed targets. The small-sample study showed that, even with the optimal settings, short FSS staircases (up to 20 reversals in length) are subject to some bias, and their precision is less than reasonable, but their characteristics improve when the size delta+ of a step up is larger than half the spread of the psychometric function. Practical recommendations are given for the design of efficient and trustworthy FSS staircases.

The Role of Parametric Assumptions in Adaptive Bayesian Estimation.

“…Secondly, due to the effects of decisional and response bias on the slope and location of the psychometric function in comparative tasks, PSEs or DLs estimated from percent points are contaminated by these influences and do not portray time perception. Finally, and even in the absence of the previous two problems, the most widespread adaptive methods have been shown to provide percent-point estimates that are biased in magnitudes which cannot be assessed without knowledge of the shape of the psychometric function (García-Pérez, 1998, 2000, 2001García-Pérez, 2004, 2007;Faes et al, 2007;Alcalá-Quintana, 2007b, 2009b;Hsu and Chin, 2014). The foregoing discussion does not mean that adaptive methods should be entirely abandoned.…”

The relative benefit of word context is a constant proportion of letter identification time

“…After the second reversal, step size was halved in each direction. This one-up, one-down staircase with unequal step size results in about 78% correct responses (García-Pérez, 1998;Kaernbach, 1991). The staircase was modified so that it never fell below the frame rate of the monitor (13 msec) or above 250 msec.…”

Small-sample characterization of stochastic approximation staircases in forced-choice adaptive threshold estimation

“…Moreover, in the experimental design of adaptive procedures, forced-choice experiments are often preferred to yes-no paradigms, since they constitute a criterion-free approach to threshold estimation (Kershaw, 1985;Macmillan & Creelman, 1991). For these reasons, two-alternative forced-choice (2AFC) staircases are of widespread use in practical psychophysical experiments (García-Pérez, 1998;Leek, 2001). In up-down staircases, the stimulus intensity for the next trial is increased or decreased upon observing a subset of subject responses, according to a rule that determines the probability of correct response targeted by the procedure (Levitt, 1971;Wetherill & Levitt, 1965).…”

“…Indeed, the simulation studies of García-Pérez (1998, 2002 have demonstrated that neither the traditionally used transformed and weighted up-down staircases (Kaernbach, 1991;Wetherill & Levitt, 1965) nor some recent modifications of these methods (Kaernbach, 2001a;Zwislocki & Relkin, 2001) yield threshold estimates that correspond to the presumed probability of correct response. More importantly, the simulations in García-Pérez (1998, 2002 showed that unless particular conditions are met, the actual probability that is targeted varies greatly with the relative size of the placed intensity steps with respect to the spread of the underlying psychometric function. These asymptotic and small-sample findings should make 2AFC up-down staircases generally unfit for threshold estimation, since the spread of the psychometric function is an unknown parameter that often varies across subjects and stimuli.…”

“…Thus, the same staircase may converge to different target probabilities as a consequence of different experimental conditions. Moreover, the particular parameter settings allowing for the implementation of traditional and modified up-down staircases severely limit the range of probability levels that can be targeted with sufficient reliability (García-Pérez, 1998, 2002.…”