Contextual overtraining accelerates habit formation in new stimuli

Abstract

Context plays an important role in the formation and expression of habits but is overlooked in the classical view on habit formation. An important obstacle to empirically studying contextual effects has been the scarcity of reliable habit formation protocols. Here, we introduce a habit formation protocol (N=142) and demonstrate devaluation-insensitivity – the gold standard for assessing habit – in extensively overtrained, but not minimally trained (criterion-trained) subjects. Crucially, in a third group we show habit formation for new, minimally trained stimuli following overtraining in the task context (contextual overtraining). We further show that following overtraining, devaluation-insensitive habits predict performance on a two-stage task, a widely used indirect measure of habitual versus goal-directed processing. Finally, we find that a working memory load slows response times in conditions that require the suppression of trained responses. Our findings shed new light on the role of context in habit formation, showing that extensive training in a stable task context not only causes devaluation-insensitivity of the overtrained stimuli, but accelerates new habit formation in that context.

Learning environment-specific learning rates

“…Another recent and relevant study appears in [ 28 ]. These authors demonstrated that participants who were overtrained on a task (but not participants who were just criterion-trained), afterwards developed habitual responding more quickly for different stimuli, suggesting that what the participants had learned was not just stimulus-action associations, but setting the learning rates in an optimal manner.…”

Age differences in routine formation: the role of automatization, motivation, and executive functions

“…Such a longer period of practice with medication intake may have eased adherence to a new medication and could also have added to the rapid automatization of this behavior. Indeed, a recent study suggests that acquiring new actions in a context in which overtraining of other, similar actions has already taken place can lead to faster conversion of the new behavior into a habit ( Lesage and Verguts, 2021 ). Additionally, the experience with regular medication intake in older adults may have contributed to the lack of a relation between pill intake and switch costs as measured in the lab in this group.…”

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

“…Not only these dichotomies but also the particular delta‐learning rules of the two‐dimensional GRL model distinguish it from previous modifications of model‐free RL. Often arrived at without the due diligence of model comparison, some modifications have simply yoked value representations—for example, Q t (s t ,a 1 ) ≡ −Q t (s t ,a 2 ) —or otherwise incorporated only one type of generalization (Aquino et al, 2020; Balcarras & Womelsdorf, 2016; Ballard et al, 2019; Baram et al, 2021; Charpentier et al, 2020; Collette et al, 2017; Daw & Shohamy, 2008; Gläscher et al, 2009; Hampton et al, 2007; Hauser et al, 2014, 2015; Lesage & Verguts, 2021; Liu et al, 2021; Matsumoto et al, 2007; Mattar & Daw, 2018; Reiter et al, 2017; Vinckier et al, 2016; Wimmer et al, 2012; Zaki et al, 2016). Moreover, such models are often formulated without parameterization (e.g., g A = −1) or with a second, counterfactual RPE inverting the only outcome (i.e., r′ = −r or r′ = 0 for r > 0) in parallel—and, by extension, multiple RPEs as required—as opposed to the current algorithmic scheme of GRL with weighted duplications of the original RPE signal to be relayed to parallel representations of estimated values.…”

“…The present study operationalizes the concepts of associative versus discriminative generalization in relation to implicitly inferential counterfactual learning (cf. Aquino et al, 2020; Balcarras & Womelsdorf, 2016; Ballard et al, 2019; Baram et al, 2021; Charpentier et al, 2020; Collette et al, 2017; Daw & Shohamy, 2008; Gläscher et al, 2009; Hampton et al, 2007; Hauser et al, 2014, 2015; Lesage & Verguts, 2021; Liu et al, 2021; Matsumoto et al, 2007; Mattar & Daw, 2018; Reiter et al, 2017; Vinckier et al, 2016; Wimmer et al, 2012; Zaki et al, 2016) that also differs with respect to states versus actions. The simpler associative generalization treats different representations as if they were equivalent or at least similar, which can but does not necessarily imply inference.…”