The functional role of episodic memory in spatial learning

Zeng, Xiangshuai; Wiskott, Laurenz; Cheng, Sen

doi:10.1101/2021.11.24.469830

Cited by 4 publications

(5 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As part of the Gym Interface, a DQN agent implemented using Keras/Tensorflow receives the processed image observation, the instant reward and then takes an action, deciding whether the artificial agent should move forward, turn left, etc. The DQN agents can utilize memory replay to speed up learning and to study the role of short-term memory and episode replay in spatial navigation tasks (Zeng et al, 2022 ). It is possible to turn the memory on or off, limit its size, or change the statistics of how memories are replayed to model the effect of manipulating episodic memory (Diekmann and Cheng, 2022 ).…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, we introduced CoBeL-RL, a RL framework oriented toward computational neuroscience, which provides a large range of environments, established RL models and analysis tools, and can be used to simulate a variety of behavioral tasks. Already, a set of computational studies focusing on explaining animal behavior (Walther et al, 2021 ; Zeng et al, 2022 ) as well as neural activity (Diekmann and Cheng, 2022 ; Vijayabaskaran and Cheng, 2022 ) have employed predecessor versions of CoBeL-RL. The framework has been expanded and refined since these earlier studies.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Diekmann

Vijayabaskaran²,

Zeng³

et al. 2023

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Diekmann

Vijayabaskaran²,

Zeng³

et al. 2023

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, there is a framework for simulating closed-loop behavior based on reinforcement learning that is oriented towards neuroscience (CoBeL-RL) (Walther et al, 2021). While this allows for studying the computational solutions that emerge under different constraints, such as place-cell like representations Vijayabaskaran and Cheng (2022), how the statistics of hippocampal replay emerges in the network Diekmann and Cheng (2022), or the function of memory replay Zeng, Wiskott, and Cheng (2022), these studies were based on machine learning implementations of reinforcement learning and neural networks. Even though they help us understand the abstract computations performed by the brain during spatial navigation, the internal processes of this model significantly differ from those in the brain.…”

Section: Discussionmentioning

confidence: 99%

Navigation and the Efficiency of Spatial Coding: Insights from Closed-Loop Simulations

Ghazinouri

Nejad

Cheng

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Spatial learning is critical for survival and its underlying neuronal mechanisms have been studied extensively. These studies have revealed a wealth of information about the neural representations of space, such as place cells and boundary cells. While many studies have focused on how these representations emerge in the brain, their functional role in driving spatial learning and navigation has received much less attention. We extended an existing computational modeling tool-chain to study the functional role of spatial representations using closed-loop simulations of spatial learning. At the heart of the model agent was a spiking neural network that formed a ring attractor. This network received inputs from place and boundary cells and the location of the activity bump in this network was the output. This output determined the movement directions of the agent. We found that the navigation performance depended on the parameters of the place cell input, such as their number, the place field sizes, and peak firing rate, as well as, unsurprisingly, the size of the goal zone. The dependence on the place cell parameters could be accounted for by just a single variable, the overlap index, but this dependence was nonmonotonic. By contrast, performance scaled monotonically with the Fisher information of the place cell population. Our results therefore demonstrate that efficiently encoding spatial information is critical for navigation performance.

show abstract

“…In this paper, we introduced CoBeL-RL, a RL framework oriented towards computational neuroscience, which provides a large range of environments, established RL models and analysis tools, and can be used to simulate a variety of behavioral tasks. Already, a set of computational studies focusing on explaining animal behavior (Walther et al, 2021; Zeng et al, 2021) as well as neural activity (Diekmann and Cheng, 2022; Vijayabaskaran and Cheng, 2022) have employed predecessor versions of CoBeL-RL. The framework has been expanded and refined since these earlier studies.…”

Section: Discussionmentioning

confidence: 99%

“…As part of the Gym Interface, a DQN agent implemented using Keras/Tensorflow receives the processed image observation, the instant reward and then takes an action, deciding whether the artificial agent should move forward, turn left, etc. The DQN agents can utilize memory replay to speed up learning and to study the role of short-term memory and episode replay in spatial navigation tasks (Zeng et al, 2021). It is possible to turn the memory on or off, limit its size, or change the statistics of how memories are replayed to model the effect of manipulating episodic memory (Diekmann and Cheng, 2022).…”

Section: Cobel-rlmentioning

confidence: 99%

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Diekmann

Vijayabaskaran

Zeng

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g. T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g. a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g. Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.

show abstract

The functional role of episodic memory in spatial learning

Cited by 4 publications

References 63 publications

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Navigation and the Efficiency of Spatial Coding: Insights from Closed-Loop Simulations

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

Contact Info

Product

Resources

About