Acquisition of decision making criteria: reward rate ultimately beats accuracy

Abstract

Speed-accuracy tradeoffs strongly influence the rate of reward that can be earned in many decision-making tasks. Previous reports suggest that human participants often adopt suboptimal speed-accuracy tradeoffs in single session, two-alternative forced-choice tasks. We investigated whether humans acquired optimal speed-accuracy tradeoffs when extensively trained with multiple signal qualities. When performance was characterized in terms of decision time and accuracy, our participants eventually performed nearly optimally in the case of higher signal qualities. Rather than adopting decision criteria that were individually optimal for each signal quality, participants adopted a single threshold that was nearly optimal for most signal qualities. However, setting a single threshold for different coherence conditions resulted in only negligible decrements in the maximum possible reward rate. Finally, we tested two hypotheses regarding the possible sources of suboptimal performance: a) favoring accuracy over reward rate and b) misestimating the reward rate due to timing uncertainty. Our findings provide support for both hypotheses, but also for the hypothesis that participants can learn to approach optimality. We find specifically that an accuracy bias dominates early performance, but diminishes greatly with practice. The residual discrepancy between optimal and observed performance can be explained by an adaptive response to uncertainty in time estimation.

Age-related differences in diffusion model boundary optimality with both trial-limited and time-limited tasks

“…These results suggest that becoming more cautious when evidence becomes more ambiguous is a powerful heuristic, one that is intuitive and probably reinforced by everyday experience. Indeed, Balci et al (2011) found that even young participants with 13-15 sessions of practice on a decision task were too cautious to optimize reward rate in the difficult condition, despite performing close to reward-rate optimal in easier conditions.…”

“…Second, even the young participants showed limitations in their ability to optimize reward rate across the difficulty conditions with fixed-time blocks, and it is reasonable to think that these limitations would be more severe for the older population. Older participants might be less likely to explore different boundary settings or less able to monitor the accuracy and duration of their responses (Balci et al, 2011;Starns & Ratcliff, 2010). The deficit might also have a physiological basis.…”

“…Reward rate has the advantage of specifying a "parameter-free" target for an appropriate boundary value. Even if participants do not always attempt to optimize reward rate in normal circumstances, tasks like our own make reward rate optimization an explicit goal and offer an objective standard for success in meeting this goal (Balci et al, 2011;Bogacz et al, 2006;Bogacz et al, 2010;Gold & Shadlen, 2002;Simen et al, 2009). As such, reward rate is an important tool in advancing our understanding of how participants control speed-accuracy criteria.…”

“…In the present experiment, three difficulty conditions varied between blocks with the difficulty level signaled before each block began. Different boundary settings are required to optimize reward rate at different levels of difficulty (Balci et al, 2011;Bogacz et al, 2006). With extremely easy tasks, the correct answer can be produced even at a fast pace; thus, narrow boundaries are optimal.…”

“…In the present experiment, we added a condition in which blocks lasted a set amount of time regardless of the number of trials completed (we will call these fixed-time blocks; see Balci et al, 2011;Bogacz, Hu, Holmes, & Cohen, 2010;Simen et al, 2009). Fixed-time participants were told to get as many correct answers as possible within the time limit (that is, to maximize reward rate).…”