Sensitivity of reaction time to the magnitude of rewards reveals the cost-structure of time

Abstract

The Drift-Diffusion Model (DDM) is the prevalent computational model of the speed-accuracy trade-off in decision making. The DDM provides an explanation of behavior by optimally balancing reaction times and error rates. However, when applied to value-based decision making, the DDM makes the stark prediction that reaction times depend only on the relative utility difference between the options and not on absolute utility magnitudes. This prediction runs counter to evidence that reaction times decrease with higher utility magnitude. Here, we ask if and how it could be optimal for reaction times to show this observed pattern. We study an algorithmic framework that balances the cost of delaying rewards against the utility of obtained rewards. We find that the functional form of the cost of delay plays a key role, with the empirically observed pattern becoming optimal under multiplicative discounting. We add to the empirical literature by testing whether utility magnitude affects reaction times using a novel methodology that does not rely on functional form assumptions for the subjects’ utilities. Our results advance the understanding of how and why reaction times are sensitive to the magnitude of rewards.

Magnitude-sensitive reaction times reveal non-linear time costs in multi-alternative decision-making

“…As previously shown ( Marshall, 2019 ; Steverson et al, 2019 ), assuming geometric temporal discounting, the optimal policy for binary decisions is magnitude-sensitive. In ternary decisions, geometric discounting has the same effect;regardless of utility function linearity, the optimal policy is magnitude-sensitive (Fig.…”

“…As remarked in the introduction above, for binary decisions magnitude-sensitive reaction times can be explained by optimal decision policies for either multiplicative ( e.g . geometric) time discounting ( Marshall, 2019 ; Steverson et al, 2019 ) or nonlinear subjective utility with linear time costs ( Tajima et al, 2016 ). In the multi-alternative case, on the other hand, the picture is more nuanced; moving from linear costing of time to geometric discounting of future rewards changes complicated time-dependent non-linear decision thresholds (( Tajima et al, 2019 ) Fig.…”

“…Interestingly, the theoretically optimal policy for the binary decision case ( Tajima et al, 2016 ) is inconsistent with empirical observations of magnitude-sensitive reaction-times ( Teodorescu et al ( 2016 ); Pirrone et al ( 2018a ); Steverson et al ( 2019 ); Zajkowski et al ( 2019 ); Turner et al ( 2019 ), but see Bhui ( 2019 )), unless assumptions are made that subjective utilities for decision-makers are nonlinear, or decisions are embedded in a fixed-length time period with known or learnable distributions of trial option values, so that a variable opportunity cost arises from decision time ( Tajima et al, 2016 ). Furthermore, even single-trial dynamics lead to magnitude sensitive reaction times ( Pirrone et al, 2018b ).…”

The wisdom of stalemates: consensus and clustering as filtering mechanisms for improving collective accuracy

“…quality of nesting sites or food patches) as the main driving factor of decision making [40]. Of particular interest, empirical evidence has shown that humans and other primates decide more quickly when shown two equally high-quality options compared with two equally low-quality options [41,42], with theoretical models replicating the 'value-sensitive hypothesis' [22,42]. Our results suggest that animals in groups could additionally adjust the structure of their social network in order to titrate between speed and accuracy of decision making.…”

Is attentional discounting in value-based decision making magnitude sensitive?

“…Traditionally, only the difference between alternatives was taken under consideration in two-alternatives decision making and magnitude was deemed as 'irrelevant' (Ashby, Jekel, Dickert & Glöckner, 2016;Bogacz et al, 2006). Recently, it has been proposed and demonstrated that the overall magnitude of the alternatives affects decision making too, both in perceptual and value-based decision making, for organisms at different levels of biological complexity, from unicellular organisms to monkeys and humans (Bose, Pirrone, Reina & Marshall, 2020;Dussutour, Ma & Sumpter, 2019;Hunt et al, 2012;Kvam & Pleskac, 2016;Pais et al, 2013;Pirrone, Azab, Hayden, Stafford & Marshall, 2018;Pirrone, Stafford & Marshall, 2014;Pirrone, Wen & Li, 2018;Ratcliff et al, 2018;Steverson, Chung, Zimmermann, Louie & Glimcher, 2019;Teodorescu, Moran & Usher, 2016).…”