Artificial Development by Reinforcement Learning Can Benefit From Multiple Motivations

Palm, Günther; Schwenker, Friedhelm

doi:10.3389/frobt.2019.00006

Cited by 7 publications

(3 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea to shape the reward function with the expert's demonstrations has been explored in previous studies, such as inverse reinforcement learning (Abbeel and Ng, 2004 ) and multi-motivation learning (Palm and Schwenker, 2019 ). However, implementation and evaluation of this idea in multi-agent environments are still lacking.…”

Section: Background and Frameworkmentioning

confidence: 99%

Structure-Preserving Imitation Learning With Delayed Reward: An Evaluation Within the RoboCup Soccer 2D Simulation Environment

Nguyen

Prokopenko

2020

Front. Robot. AI

View full text Add to dashboard Cite

We describe and evaluate a neural network-based architecture aimed to imitate and improve the performance of a fully autonomous soccer team in RoboCup Soccer 2D Simulation environment. The approach utilizes deep Q-network architecture for action determination and a deep neural network for parameter learning. The proposed solution is shown to be feasible for replacing a selected behavioral module in a well-established RoboCup base team, Gliders2d, in which behavioral modules have been evolved with human experts in the loop. Furthermore, we introduce an additional performance-correlated signal (a delayed reward signal), enabling a search for local maxima during a training phase. The extension is compared against a known benchmark. Finally, we investigate the extent to which preserving the structure of expert-designed behaviors affects the performance of a neural network-based solution.

show abstract

Section: Background and Frameworkmentioning

confidence: 99%

Structure-Preserving Imitation Learning With Delayed Reward: An Evaluation Within the RoboCup Soccer 2D Simulation Environment

Nguyen

Prokopenko

2020

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…Great strides have been made recently toward solving hard problems with deep learning, including reinforcement learning 1 , 2 . While these are groundbreaking and show superior performance over humans in some domains, humans nevertheless exceed computers in the ability to find creative and efficient solutions to novel problems, especially with changing internal motivation values 3 . Artificial general intelligence (AGI), especially the ability to learn autonomously to solve arbitrary problems, remains elusive 4 .…”

Section: Introductionmentioning

confidence: 99%

Foundations of human spatial problem solving

Zarr

Brown

2023

Sci Rep

View full text Add to dashboard Cite

Despite great strides in both machine learning and neuroscience, we do not know how the human brain solves problems in the general sense. We approach this question by drawing on the framework of engineering control theory. We demonstrate a computational neural model with only localist learning laws that is able to find solutions to arbitrary problems. The model and humans perform a multi-step task with arbitrary and changing starting and desired ending states. Using a combination of computational neural modeling, human fMRI, and representational similarity analysis, we show here that the roles of a number of brain regions can be reinterpreted as interacting mechanisms of a control theoretic system. The results suggest a new set of functional perspectives on the orbitofrontal cortex, hippocampus, basal ganglia, anterior temporal lobe, lateral prefrontal cortex, and visual cortex, as well as a new path toward artificial general intelligence.

show abstract

“…However, fluctuations in physiological states can profoundly affect behavior. Recent suggestions include using time-varying multiobjective reward functions in biological context ( Koulakov, 2018 ; Palm and Schwenker, 2019 ). Modeling such factors is thus an important goal in computational neuroscience and is in the early stages of mathematical description ( Berridge, 2012 ; Berridge and Robinson, 2016 ).…”

Section: Introductionmentioning

confidence: 99%

Neural Networks With Motivation

Shuvaev

Tran

Stephenson-Jones

et al. 2021

Front. Syst. Neurosci.

View full text Add to dashboard Cite

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent’s dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

show abstract

Artificial Development by Reinforcement Learning Can Benefit From Multiple Motivations

Cited by 7 publications

References 79 publications

Structure-Preserving Imitation Learning With Delayed Reward: An Evaluation Within the RoboCup Soccer 2D Simulation Environment

Structure-Preserving Imitation Learning With Delayed Reward: An Evaluation Within the RoboCup Soccer 2D Simulation Environment

Foundations of human spatial problem solving

Neural Networks With Motivation

Contact Info

Product

Resources

About