Computational models of episodic-like memory in food-caching birds

Abstract

Birds of the crow family adapt food-caching strategies to anticipated needs at the time of cache recovery and rely on memory of the what, where and when of previous caching events to recover their hidden food. It is unclear if this behavior can be explained by simple associative learning or if it relies on higher cognitive processes like mental time-travel. We present a computational model and propose a neural implementation of food-caching behavior. The model has hunger variables for motivational control, reward-modulated update of retrieval and caching policies and an associative neural network for remembering caching events with a memory consolidation mechanism for flexible decoding of the age of a memory. Our methodology of formalizing experimental protocols is transferable to other domains and facilitates model evaluation and experiment design. Here, we show that memory-augmented, associative reinforcement learning without mental time-travel is sufficient to explain the results of 28 behavioral experiments with food-caching birds.

A non-Hebbian code for episodic memory

“…5). In contrast, most existing models treat one-shot episodic memory as a generic buffer for temporarily storing arbitrary information (6,7,34,36,39,41,49,67,68) (typically through Hebbian plasticity), with policies existing in a separate function space. In episodic RL, for instance, episodes are stored as memorized action sequences that can be sampled to efficiently approximate state-action values to shape policy (34,36).…”

“…It has long been hypothesized that Hebbian plasticitythe strengthening of synaptic connections between repeatedly co-active neurons-is the neural substrate for memory formation (1)(2)(3)(4)(5)(6)(7)(8). The canonical mechanism thought to implement Hebbian plasticity in vivo is spike-timing dependent plasticity (STDP), in which a synapse between two neurons potentiates if the postsynaptic neuron spikes (emits an action potential) immediately after the presynaptic neuron spikes (5,(9)(10)(11).…”

Remembering the “When”: Hebbian Memory Models for the Time of Past Events

“…Behavioral experiments with California scrub-jays showed convincingly that these birds have a flexible “what-where-when” memory system ( Clayton and Dickinson, 1998, 1999 ; Clayton et al, 2001, Clayton et al 2003, Clayton et al 2005 ; Brea et al, 2023 ). Although the existing experimental results cannot discriminate between the different models considered here, the observation that they can learn different behavioral rules that depend on the content and age of memories within few trials, speaks in favor of an Age-Organization model, that allows fast and flexible learning.…”

Sequence representation as an early step in the evolution of language

“…Performing behaviour sequences does not require recognizing and remembering sequential information [45], as it can be learned through primary and conditioned reinforcement [46][47][48]. Furthermore, computational models that do not rely on sequence representation account well for the acquisition of various behavior sequences in non-human animals, including tool use [49], planning [50], social learning [51] and caching [52].…”