The reward-complexity trade-off in schizophrenia

Abstract

Action selection requires a policy that maps states of the world to a distribution over actions. The amount of memory needed to specify the policy (the policy complexity) increases with the state-dependence of the policy. If there is a capacity limit for policy complexity, then there will also be a trade-off between reward and complexity, since some reward will need to be sacrificed in order to satisfy the capacity constraint. This paper empirically characterizes the trade-off between reward and complexity for both schizophrenia patients and healthy controls. Schizophrenia patients adopt lower complexity policies on average, and these policies are more strongly biased away from the optimal reward-complexity trade-off curve compared to healthy controls. However, healthy controls are also biased away from the optimal trade-off curve, and both groups appear to lie on the same empirical trade-off curve. We explain these findings using a cost-sensitive actor-critic model. Our empirical and theoretical results shed new light on cognitive effort abnormalities in schizophrenia.

Undermatching is a consequence of policy compression

“…Moreover, they operate near the optimal frontier, which suggests that they are close to optimally balancing reward and complexity. We additionally find that policy complexity does not appreciably change across task variants (Figure 4B), suggesting a constant resource constraint, which we have observed in prior applications of policy complexity (Gershman and Lai, 2021).…”

Reinforcement learning modeling reveals a reward-history-dependent strategy underlying reversal learning in squirrel monkeys

“…Theories about the relevant cognitive processes are used to form and test explicit hypotheses, in the form of models, about behavioral strategies (Wilson and Collins, 2019; Daw et al, 2011; Heathcote et al, 2015). From the perspective of computational psychiatry, these models in turn allow us to understand and quantify aberrant information processing in disease (Huys et al, 2016; Aylward et al, 2019; Gershman and Lai, 2020; Mason et al, 2017; Redish, 2004; Radulescu and Niv, 2019), as well as effects of therapy (Michely et al, 2020; Frank et al, 2007; Paulus et al, 2016).…”

Suboptimal human inference inverts the bias-variance trade-off for decisions with asymmetric evidence

“…The second is that this metric of information complexity does not necessarily correspond to computational complexity; i.e., the computational and memory-based resources needed to implement a particular strategy, which also likely contributes to trade-offs that can limit accuracy [58–61]. Moreover, we do not know the extent to which our subjects using low-complexity strategies had poorly tuned heuristics, were more distracted, employed fewer cognitive resources, and/or allocated less attention, as has been reported previously [62, 63]. Nevertheless, our findings add to a growing literature emphasizing that “optimality” in human decision-making is not a simple concept that can be related only to objective performance benchmarks (e.g., accuracy and biases) but also limitations on the capacity to achieve those benchmarks given the information available and the strategy used to process that information [64, 65].…”