Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging

Whitehead, Steven D.; Karlsson, Jonas; Tenenberg, Josh

doi:10.1007/978-1-4615-3184-5_3

Cited by 71 publications

(29 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each module has its own Q matrix representing its partial knowledge of the world state s i . An agent-level module selection or action selection policy chooses an action from the module preferences, such as the Greatest Mass (GM) strategy [49,75]:…”

Section: Modular Reinforcement Learningmentioning

confidence: 99%

Cooperative Multi-Agent Reinforcement Learning for Multi-Component Robotic Systems: guidelines for future research

Graña

Fernández-Gauna

López-Guede

2011

Paladyn, Journal of Behavioral Robotics

View full text Add to dashboard Cite

Reinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent to optimally achieve a goal with minimal feedback information about the desired behavior, which is not precisely specified. Scalar rewards are returned to the agent as response to its actions endorsing or opposing them. RL algorithms have been successfully applied to robot control design. The extension of the RL paradigm to cope with the design of control systems for Multi-Component Robotic Systems (MCRS) poses new challenges, mainly related to coping with scaling up of complexity due to the exponential state space growth, coordination issues, and the propagation of rewards among agents. In this paper, we identify the main issues which o er opportunities to develop innovative solutions towards fully-scalable cooperative multi-agent systems.

show abstract

Section: Modular Reinforcement Learningmentioning

confidence: 99%

Cooperative Multi-Agent Reinforcement Learning for Multi-Component Robotic Systems: guidelines for future research

Graña

Fernández-Gauna

López-Guede

2011

Paladyn, Journal of Behavioral Robotics

View full text Add to dashboard Cite

show abstract

“…Many modular architectures have been proposed so far (for example [7], [8], [9]). Each module is responsible for learning to achieve a single Fig.2 shows a sketch of such a modular learning system.…”

Section: B Modular Learning Systemmentioning

confidence: 99%

Mutual Development of Behavior Acquisition and Recognition based on Value System

Takahashi¹,

Tamura²,

Asada³

2009

J. SOFT

View full text Add to dashboard Cite

Abstract-Both self-learning architecture (embedded structure) and explicit/implicit teaching from other agents (environmental design issue) are necessary not only for one behavior learning but more seriously for life-time behavior learning. This paper presents a method for a robot to understand unfamiliar behavior shown by others through the collaboration between behavior acquisition and recognition of observed behavior, where the state value has an important role not simply for behavior acquisition (reinforcement learning) but also for behavior recognition (observation). That is, the state value updates can be accelerated by observation without real trial and error while the learned values enrich the recognition system since it is based on estimation of the state value of the observed behavior. The validity of the proposed method is shown by applying it to a dynamic environment where two robots play soccer.

show abstract

“…To remedy the problem of combinatorial explosion in multiagent reinforcement learning, Whitehead [39] proposed an architecture called modular Q-learning, which decomposes the whole problem space into smaller subproblem spaces and distributes them among multiple modules. This way, the goals of multiple-goal problems are decomposed into subgoals, which are then distributed among multiple modules.…”

Section: B Multiagent Learning Fuzzy Modular Approach With Internal mentioning

confidence: 99%

Fuzzy OLAP Association Rules Mining-Based Modular Reinforcement Learning Approach for Multiagent Systems

Kaya

Alhajj

2005

IEEE Trans. Syst., Man, Cybern. B

View full text Add to dashboard Cite

Abstract-Multiagent systems and data mining have recently attracted considerable attention in the field of computing. Reinforcement learning is the most commonly used learning process for multiagent systems. However, it still has some drawbacks, including modeling other learning agents present in the domain as part of the state of the environment, and some states are experienced much less than others, or some state-action pairs are never visited during the learning phase. Further, before completing the learning process, an agent cannot exhibit a certain behavior in some states that may be experienced sufficiently. In this study, we propose a novel multiagent learning approach to handle these problems. Our approach is based on utilizing the mining process for modular cooperative learning systems. It incorporates fuzziness and online analytical processing (OLAP) based mining to effectively process the information reported by agents. First, we describe a fuzzy data cube OLAP architecture which facilitates effective storage and processing of the state information reported by agents. This way, the action of the other agent, not even in the visual environment 1 of the agent under consideration, can simply be predicted by extracting online association rules, a well-known data mining technique, from the constructed data cube. Second, we present a new action selection model, which is also based on association rules mining. Finally, we generalize not sufficiently experienced states, by mining multilevel association rules from the proposed fuzzy data cube. Experimental results obtained on two different versions of a well-known pursuit domain show the robustness and effectiveness of the proposed fuzzy OLAP mining based modular learning approach. Finally, we tested the scalability of the approach presented in this paper and compared it with our previous work on modular-fuzzy Q-learning and ordinary Q-learning.

show abstract

Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging

Cited by 71 publications

References 0 publications

Cooperative Multi-Agent Reinforcement Learning for Multi-Component Robotic Systems: guidelines for future research

Cooperative Multi-Agent Reinforcement Learning for Multi-Component Robotic Systems: guidelines for future research

Mutual Development of Behavior Acquisition and Recognition based on Value System

Fuzzy OLAP Association Rules Mining-Based Modular Reinforcement Learning Approach for Multiagent Systems

Contact Info

Product

Resources

About