A working memory model improves cognitive control in agents and robots

Persiani, Michele; Franchi, Alessio Mauro; Gini, Giuseppina

doi:10.1016/j.cogsys.2018.04.014

Cited by 12 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the aim of this approach was primarily to increase the learning performance of the agents, compared to an uniform sampling baseline the entropybased sampling also seemed to have induced an intrinsically motivated exploration that became an important part of their tactics to increase the overall performance. Persiani et al [28] introduces an approach that also uses the replay memory structure in order to improve cognition. The algorithm is able to actively learn to select the most appropriate chunks of the agent's experience to be stored in the replay memory buffer based on maximizing the expected future reward.…”

Section: B Artificial Attention As a Behavior Inducing Mechanismmentioning

confidence: 99%

Selective Perception as a Mechanism to Adapt Agents to the Environment: An Evolutionary Approach

Ramicic

Bonarini

2020

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Rapid advancement of machine learning makes it possible to consider large amounts of data to learn from. Learning agents may get data ranging on real intervals directly from the environment they interact with, in a process usually time-expensive. To improve learning and manage these data, approximated models and memory mechanisms are adopted. In most of the implementations of reinforcement learning facing this type of data, approximation is obtained by neural networks and the process of drawing information from data is mediated by a short-term memory that stores the previous experiences for additional re-learning, to speed-up the learning process, mimicking what is done by people. In this work, we are proposing a novel computational approach able to selectively filter the information, or cognitive load, for the agent's short-term memory, thus emulating the attention mechanism characteristic of human perception. We devised an evolutionary model of agent's perception to adapt the attention filter present in the proposed architecture to the actual environment faced by the agent, by selecting the experiences that most likely influence in a positive way its learning characteristics. This approach can evolve a filter able to provide an optimal cognitive load of the experiences entering in the agent's short-term memory of a limited capacity. The evolved sampling dynamics can also lead to the emergence of intrinsically motivated curiosity.

show abstract

Section: B Artificial Attention As a Behavior Inducing Mechanismmentioning

confidence: 99%

Selective Perception as a Mechanism to Adapt Agents to the Environment: An Evolutionary Approach

Ramicic

Bonarini

2020

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…There has also been significant work in robotics seeking to use insights from psychological research on WM to better inform the design of select components of larger robot cognitive architectures that do not necessarily aspire towards cognitive plausibility. For example, a diverse body of researchers has collaborated on the development and use of the WM Toolkit (Phillips & Noelle, 2005;Gordon & Hall, 2006;Kawamura et al, 2008;Persiani et al, 2018); a software toolkit that maintains pointers to a fixed number of chunks containing arbitrary information. At each timestep, this toolkit proposes a new set of chunks, and then uses neural networks to select a subset of these chunks to retain.…”

Section: Models Of Working Memory In Integrated Robot Architecturesmentioning

confidence: 99%

Toward Forgetting-Sensitive Referring Expression Generationfor Integrated Robot Architectures

Williams¹,

Johnson²,

Culpepper³

et al. 2020

Preprint

View full text Add to dashboard Cite

To engage in human-like dialogue, robots require the ability to describe the objects, locations, and people in their environment, a capability known as "Referring Expression Generation." As speakers repeatedly refer to similar objects, they tend to re-use properties from previous descriptions, in part to help the listener, and in part due to cognitive availability of those properties in working memory (WM). Because different theories of working memory "forgetting" necessarily lead to differences in cognitive availability, we hypothesize that they will similarly result in generation of different referring expressions. To design effective intelligent agents, it is thus necessary to determine how different models of forgetting may be differentially effective at producing natural human-like referring expressions. In this work, we computationalize two candidate models of working memory forgetting within a robot cognitive architecture, and demonstrate how they lead to cognitive availability-based differences in generated referring expressions.

show abstract

“…Persiani et al [19] proposes a cognitive improvement through the use of replay memory structure like AMR. The algorithm makes it possible to learn which chunks of agent's experiences are most appropriate for replay based on their ability to maximize the future expected reward.…”

Section: Related Workmentioning

confidence: 99%

Augmented Replay Memory in Reinforcement Learning With Continuous Control

Ramicic,

Bonarini

2019

Preprint

View full text Add to dashboard Cite

Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent's state space enabling it to scale up to a more complex problems but also increases the risk of forgetting by learning on redundant or conflicting data. To improve the approximation of a large amount of data, a random mini-batch of the past experiences that are stored in the replay memory buffer is often replayed at each learning step. The proposed work takes inspiration from a biological mechanism which act as a protective layer of human brain higher cognitive functions: active memory consolidation mitigates the effect of forgetting of previous memories by dynamically processing the new ones. The similar dynamics are implemented by a proposed augmented memory replay AMR capable of optimizing the replay of the experiences from the agent's memory structure by altering or augmenting their relevance. Experimental results show that an evolved AMR augmentation function capable of increasing the significance of the specific memories is able to further increase the stability and convergence speed of the learning algorithms dealing with the complexity of continuous action domains.

show abstract

A working memory model improves cognitive control in agents and robots

Cited by 12 publications

References 18 publications

Selective Perception as a Mechanism to Adapt Agents to the Environment: An Evolutionary Approach

Selective Perception as a Mechanism to Adapt Agents to the Environment: An Evolutionary Approach

Toward Forgetting-Sensitive Referring Expression Generationfor Integrated Robot Architectures

Augmented Replay Memory in Reinforcement Learning With Continuous Control

Contact Info

Product

Resources

About