Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling

Abstract

When humans and animals learn by trial-and-error to select the most advantageous action, the progressive increase in action selection accuracy due to learning is typically accompanied by a decrease in the time needed to execute this action. Both choice and response time (RT) data can thus provide information about decision and learning processes. However, traditional reinforcement learning (RL) models focus exclusively on the increase in choice accuracy and ignore RTs. Consequently, they neither decompose the interactions between choices and RTs, nor investigate how these interactions are influenced by contextual factors. However, at least in the field of perceptual decision-making, such interactions have proven to be important to dissociate between the underlying processes. Here, we analyzed such interactions in behavioral data from four experiments, which feature manipulations of two factors: outcome valence (gains vs. losses) and feedback information (partial vs. complete feedback). A Bayesian metaanalysis revealed that these contextual factors differently affect RTs and accuracy. To disentangle the processes underlying the observed behavioral patterns, we jointly fitted choices and RTs across all experiments with a single, Bayesian, hierarchical diffusion decision model (DDM). In punishmentavoidance contexts, compared to reward-seeking contexts, participants consistently slowed down without any loss of accuracy. The DDM explained these effects by shifts in the non-decision time and threshold parameters. The reduced motor facilitation may represent the basis of Pavlovian-toinstrumental transfer biases, while the increased cautiousness might be induced by the expectation of losses and be consistent with the loss attention framework.