A quantitative confidence signal detection model: 1. Fitting psychometric functions

Abstract

Yi Y, Merfeld DM. A quantitative confidence signal detection model: 1. Fitting psychometric functions. J Neurophysiol 115: 1932J Neurophysiol 115: -1945J Neurophysiol 115: , 2016. First published January 13, 2016; doi:10.1152/jn.00318.2015.-Perceptual thresholds are commonly assayed in the laboratory and clinic. When precision and accuracy are required, thresholds are quantified by fitting a psychometric function to forced-choice data. The primary shortcoming of this approach is that it typically requires 100 trials or more to yield accurate (i.e., small bias) and precise (i.e., small variance) psychometric parameter estimates. We show that confidence probability judgments combined with a model of confidence can yield psychometric parameter estimates that are markedly more precise and/or markedly more efficient than conventional methods. Specifically, both human data and simulations show that including confidence probability judgments for just 20 trials can yield psychometric parameter estimates that match the precision of those obtained from 100 trials using conventional analyses. Such an efficiency advantage would be especially beneficial for tasks (e.g., taste, smell, and vestibular assays) that require more than a few seconds for each trial, but this potential benefit could accrue for many other tasks.thresholds; decision-making; confidence rating; confidence calibration MEASURING THRESHOLDS IS PROBABLY the most common psychophysical procedure in use today; applications range from experimental psychology to neuroscience to economics to engineering. Fitting psychometric functions using categorical data analyses (Agresti 1996) that describe the relationship between a stimulus characteristic (e.g., amplitude) and a subject's forced-choice categorical responses provides a standard approach used to estimate thresholds (Green and Swets 1966; Macmillan and Creelman 2005).A recent comprehensive analysis (Garcia-Perez and AlcalaQuintana 2005) concluded that only maximum likelihood methods should be used when accuracy and precision of psychometric function fit parameters is important and, furthermore, showed that more than 100 forced-choice trials are generally required to yield acceptable fit parameter estimates. Because such perceptual threshold tests are common and because many trials are needed to yield accurate and precise psychometric fits, studies spanning 50 yr (Garcia

The influence of size in weight illusions is unique relative to other object features

“…Another clue that something is particularly unique about size is the strength of the SWI relative to other weight illusions. For example, in considering the SWI alongside the MWI, which is the next most studied weight illusion, evidence suggests that the SWI is considerably stronger (Buckingham & Goodale, 2013;Buckingham, Michelakakis, & Cole, 2016). In the Buckingham and Goodale's (2013) study mentioned earlier, participants were tested on both illusions, in that they lifted two sets of large and small stimuli, one apparently made from polystyrene and one from aluminum.…”

Unbounded evidence accumulation characterizes subjective visual vertical forced-choice perceptual choice and confidence

“…More specifically, using forced-choice tasks in which observation duration is constrained but response time is not, we investigated 1) whether perceptual choice and confidence mechanisms include evidence accumulation and 2) whether such evidence accumulation is terminated by decision bounds. In short, two accumulator models, bounded DDM with "absorbing" bounds (Ratcliff and McKoon 2008) and unbounded DDM without decision bounds, were augmented to include confidence probability judgments (Yi and Merfeld 2016), and then these models were compared with empiric SVV binary choice (i.e., is the visual scene tilted left or right?) and confidence (i.e., what is the probability that the choice is correct?)…”

“…There is a general consensus that confidence reflects the state of the decision variables, although it is still debated whether decision and confidence involve a single-or double-stage process (Murphy et al 2015;Navajas et al 2016;Pleskac and Busemeyer 2010;Rahnev et al 2015;van den Berg et al 2016;Yu et al 2015). Yi and Merfeld recently proposed a psychophysical model explaining the correlations between confidence and perceptual precision (Yi and Merfeld 2016). Here we combine this earlier confidence model with a DDM.…”

“…This function represents perceptual noise, and thresholds are defined in terms of the spread parameter of such distribution. Unlike in Yi and Merfeld (2016), which developed the model in the stimulus domain, here we define confidence in terms of the decision variable (i.e., define confidence in terms of y instead of v).…”

“…where K(t) is a confidence factor at time t, K Ͻ 1 indicating overconfidence and K Ͼ 1 indicating underconfidence (Yi and Merfeld 2016). Here, as an empiric model required to match data, K(t) is modeled as an exponentially decaying function such that…”