Signed reward prediction errors drive declarative learning

Abstract

Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning–a quintessentially human form of learning–remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; “better-than-expected” signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

Failure to modulate reward prediction errors in declarative learning with theta (6 Hz) frequency transcranial alternating current stimulation

“…For this purpose, participants acquired 60 Dutch-Swahili word pairs, associated with RPEs of different sizes and values, while the MFC was stimulated. We replicated our earlier finding of SRPEs driving declarative learning [7]. Word pair recognition increased for large and positive RPEs.…”

“…Larger and more positive RPEs during study improved subsequent declarative memory during testing. The effect of RPEs in this experimental paradigm was further substantiated in a follow-up EEG study, where oscillatory signatures at reward feedback were detected in the theta (4)(5)(6)(7)(8), high-beta (20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) and high-alpha (10)(11)(12)(13)(14)(15) frequency ranges, suggesting the experience of RPEs by the participants [8]. Further validation came from an fMRI study using a similar paradigm in which famous faces were associated with Swahili village names [9].…”

“…One robust experimental paradigm to test this RPE effect on declarative memory, was proposed in [7]. Here, a variable-choice experimental paradigm was used where participants learned Dutch-Swahili word pairs.…”

Dopaminergic signalling modulates reward-driven music memory consolidation

“…In the context of prediction, data posits that dopaminergic neurons in the VS (and Nucleus Accumbens) are the key factor driving the attachment of hedonic value to music (Salimpoor et al, 2015;Gold et al, 2019; but see Cheung et al, 2019). Reward prediction errors (RPEs) are also crucial for reinforcement learning processes (Sutton and Barto, 2018), and growing evidence suggests that they play a pivotal role in episodic memory (Davidow et al, 2016;De Loof et al, 2018;Calderon et al, 2019). By experimentally manipulating RPEs during the encoding of faces, Calderon et al (2019) recently showed that trial-specific RPE responses in the VS during learning predict the strength of the subsequent episodic memory.…”

Deeply Felt Affect: The Emergence of Valence in Deep Active Inference

“…It is useful to contrast the view of motivated behaviour on offer here with existing normative models of behaviour and associated neural theories. In studies on reinforcement learning(De Loof et al, 2018, Sutton & Barto, 2018, signed "reward prediction error" (RPE) has been introduced as a measure of the difference between expected and obtained reward, which is used to update beliefs about the values of actions. Positive versus negative RPEs are often also (at least implicitly) assumed to correspond to unexpected pleasant and unpleasant experiences, respectively.…”