From Cognitive to Habit Behavior During Navigation, Through Cortical-Basal Ganglia Loops

How to Reduce Computation Time While Sparing Performance During Robot Navigation? A Neuro-Inspired Architecture for Autonomous Shifting Between Model-Based and Model-Free Learning

“…The idea of taking inspiration from how the brain coordinates multiple learning systems to enable more flexibility in robots is getting more and more attention in the robotics community [1][2][3][4][5][6]. One of the expected advantages of such a strategy would be for robots to autonomously learn which system is the most appropriate for each encountered task or situation.…”

NLM-HS: Navigation Learning Model Based on a Hippocampal–Striatal Circuit for Explaining Navigation Mechanisms in Animal Brains

“…One of the characteristics of the NLM-HS is to apply the cognitive map generated by the hippocampal model to the striatal navigation learning model, which promotes the convergence rate of habit generation. Banquet et al pointed out that both goal-directed cognitive learning systems and habitual learning systems receive similar hippocampal transition field inputs [ 34 ], which shows the same usage of hippocampal cognitive maps. The advantage of using the hippocampal cognitive map as the input for habit generation in the striatum is that the state space that needs to be learned is greatly reduced, thereby accelerating the convergence speed of habit generation in the striatum.…”

Time as the fourth dimension in the hippocampus

“…Note that the HS projects to nucleus accumbens shell (in ventral striatum) which then projects to VTA. This connectivity could account for similarities in the activities of HS and ventral striatum during sequence coding (Banquet et al, 2016). This increase in reverse replay following changes in reward contingencies suggests a possible solution to the temporal credit assignment dilemma (Foster and Wilson, 2006), that is, how does the brain reinforce activated pathways leading up to the reward?…”

“…The presence of prospective and retrospective activity in tasks not requiring the HS suggests that other associated areas such as mPFC and striatum could be implicated in the functional elaboration of this contextual activity. Depending upon task and learning conditions, in particular stability vs variability in the task demands, the most appropriate control structure would be engaged (Banquet et al, 2016). In this view, the introduction of a delay during the alternation task could disrupt the 'HS-independent' automatic performance of continuous alternation resulting from overtraining, and lead to behavioral control by the HS and/or mPFC, with their working memory processing.…”

“…Third, REM sleep suppression in humans affects only the acquisition of procedural memories (see Stickgold et al, 2001, for a review). After overtraining, these cortical sequences may become critical for the HS-independent performance of procedural tasks and habitual behavior in general (Banquet et al, 2016;Karni et al, 1994). Finally, hippocampal lesions only slow down learning the eight arm maze (Jarrard, 1995), but do not usually prevent procedural learning.…”