An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning

Ishiwaka, Yuko; Sato, Takamasa; Kakazu, Yukinori

doi:10.1016/s0921-8890(03)00040-x

Cited by 70 publications

(45 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a popular variant, several "predator" robots have to capture a "prey" robot by converging on it [83], [96].…”

Section: B Robotic Teamsmentioning

confidence: 99%

“…In these cases, approximate solutions must be sought, e.g., by extending to multiple agents the work on approximate singleagent RL [111]- [122]. A fair number of approximate MARL algorithms have been proposed: for discrete, large state-action spaces, e.g., [123], for continuous states and discrete actions, e.g., [96], [98], and [124], and finally for continuous states and actions, e.g., [95], and [125]. Unfortunately, most of these algorithms only work in a narrow set of problems and are heuristic in nature.…”

Section: A Practical Marlmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Survey of Multiagent Reinforcement Learning

Buşoniu

Babuška

Schutter

2008

IEEE Trans. Syst., Man, Cybern. C

1,710

947

View full text Add to dashboard Cite

Abstract-Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the multiagent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents' learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim-either explicitly or implicitly-at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied. Finally, an outlook for the field is provided.

show abstract

“…In a popular variant, several "predator" robots have to capture a "prey" robot by converging on it [83], [96].…”

Section: B Robotic Teamsmentioning

confidence: 99%

Section: A Practical Marlmentioning

confidence: 99%

A Comprehensive Survey of Multiagent Reinforcement Learning

Buşoniu

Babuška

Schutter

2008

IEEE Trans. Syst., Man, Cybern. C

1,710

947

View full text Add to dashboard Cite

show abstract

“…Another investigation of the pursuit domain is that of Ishiwaka et al [71], which considers heterogeneous agents in a continuous state-action world with partial, noisy observations. In this work, the authors investigate how the predators can learn online using Q-learning.…”

Section: Pursuit Domainmentioning

confidence: 99%

Making Friends on the Fly: Advances in Ad Hoc Teamwork

Barrett¹

2015

Studies in Computational Intelligence

View full text Add to dashboard Cite

“…Thus, there is no need to observe the other agent's actual rewards received from the environment, and to know the parameters that the other agent uses for Q-learning. Finally, Ishiwaka et al [19] presented a method for two kinds of prediction needed for each hunter agent acting in pursuit domain. One of these predictions is the location of the other hunter agents and prey agent, and the other is the movement direction of the prey agent at the next time step.…”

Section: B Multiagent Learning Fuzzy Modular Approach With Internal mentioning

confidence: 99%

Fuzzy OLAP Association Rules Mining-Based Modular Reinforcement Learning Approach for Multiagent Systems

Kaya

Alhajj

2005

IEEE Trans. Syst., Man, Cybern. B

View full text Add to dashboard Cite

Abstract-Multiagent systems and data mining have recently attracted considerable attention in the field of computing. Reinforcement learning is the most commonly used learning process for multiagent systems. However, it still has some drawbacks, including modeling other learning agents present in the domain as part of the state of the environment, and some states are experienced much less than others, or some state-action pairs are never visited during the learning phase. Further, before completing the learning process, an agent cannot exhibit a certain behavior in some states that may be experienced sufficiently. In this study, we propose a novel multiagent learning approach to handle these problems. Our approach is based on utilizing the mining process for modular cooperative learning systems. It incorporates fuzziness and online analytical processing (OLAP) based mining to effectively process the information reported by agents. First, we describe a fuzzy data cube OLAP architecture which facilitates effective storage and processing of the state information reported by agents. This way, the action of the other agent, not even in the visual environment 1 of the agent under consideration, can simply be predicted by extracting online association rules, a well-known data mining technique, from the constructed data cube. Second, we present a new action selection model, which is also based on association rules mining. Finally, we generalize not sufficiently experienced states, by mining multilevel association rules from the proposed fuzzy data cube. Experimental results obtained on two different versions of a well-known pursuit domain show the robustness and effectiveness of the proposed fuzzy OLAP mining based modular learning approach. Finally, we tested the scalability of the approach presented in this paper and compared it with our previous work on modular-fuzzy Q-learning and ordinary Q-learning.

show abstract

An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning

Cited by 70 publications

References 3 publications

A Comprehensive Survey of Multiagent Reinforcement Learning

A Comprehensive Survey of Multiagent Reinforcement Learning

Making Friends on the Fly: Advances in Ad Hoc Teamwork

Fuzzy OLAP Association Rules Mining-Based Modular Reinforcement Learning Approach for Multiagent Systems

Contact Info

Product

Resources

About