Strategically managing learning during perceptual decision making

Abstract

Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats’ strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process.

Learning to Choose: Behavioral Dynamics Underlying the Initial Acquisition of Decision Making

“…Some of these studies trained animals to detect single stimuli and to report the identity of the stimulus by responding to the left or right, i.e. object-place learning (Zoccolan et al, 2009; Kurylo et al, 2020; Masis et al, 2023). Others trained rodents to make lateralized movements towards the chosen stimulus and trained discrimination between stimuli from the start of initial training (Clark et al, 2011; Reinagel, 2013; Broschard et al, 2019; Broschard et al, 2021; Kurylo et al, 2020; Masis et al, 2023).…”

“…As such, neuroscientific studies of decision-making have traditionally recorded neural activity from subjects with abundant experience in making choices (Carandini and Churchland, 2013). For instance, studies employing two-alternative forced-choice (2AFC) paradigms train animals to discriminate between stimulus pairs and record brain activity after animals reach high levels of behavioral performance (Zoccolan et al, 2009; Busse et al, 2011; Brunton et al, 2013; Reinagel, 2013; Erlich et al, 2015; Hanks et al, 2015; Burgess et al, 2017; Kurylo et al, 2020; Masis et al, 2023). These studies provide valuable insights into neural and computational mechanisms underlying decision making, but have not addressed how the training process may influence decision making strategies or neural activity.…”

Decision bias and sampling asymmetry in reward-guided learning

“…In conclusion, our research adds a critical voice to the ongoing challenge [50][51][52][53] against the standard tenet of neuroeconomics that valuation is menu-invariant [54][55][56] . We highlight the importance of incorporating strategic learning dynamics to reconceptualize how biological brains strive to optimize decisions 57 . This becomes particularly significant when value is not anchored to sensory properties dictated by hardwired neural coding principles but is rather a dynamic construct continuously updated by the influx of newly sampled information.…”

Neural correlates of visual object recognition in rats

“…A total of N=56 rats were trained on the basic three-port task, out of which 48 passed criterion. A subset of these rats are part of a previously published dataset 17 . 36 rats were separately trained on the basic two-choice task to test the reliability of the behavior box and paradigm.…”

“…Despite significant progress in engineering machine vision systems that solve recognition tasks at human-comparable levels [6][7][8] , mechanistic insight into how the brain achieves robust and "invariant" object recognition remains elusive due to the limited number of animal models where this capacity can be dissected with genetic and spatial precision. Rats have been shown to perform tasks that require them to visually recognize objects across a range of contexts [9][10][11][12][13][14][15][16][17] , suggesting they may serve as a paradigm of complex visual behavior in which neural circuits are both genetically and optically accessible. However, the lack of methods for targeted access to neural populations in rats has limited the ability to link visual circuits to perceptual behavior.…”