Specialization in multi-agent systems through learning

Abstract

Specialization is a common feature in animal societies that leads to an improvement in the fitness of the team members and to an increase in the resources obtained by the team. In this paper we propose a simple reinforcement learning approach to specialization in an artificial multi-agent system. The system is composed of homogeneous and non-communicating agents. Because there is no communication, the number of agents in the team can easily scale up. Agents have the same initial functionalities, but they learn to specialize and so cooperate to achieve a complex gathering task efficiently. Simulation experiments show how the multi-agent system specializes appropriately so as to reach optimal (or near-to-optimal) performance in unknown and changing environments.

Swarm robotic odor localization: Off-line optimization and validation with real robots

“…Optimization algorithms allowing controller differentiation or using local reinforcement signals face a daunting credit assignment problem, because it is difficult to determine which actions of which agent are responsible for the outcomes observed in the system. Learning under these conditions has been addressed with the help of explicit communication, 34 a priori information about proper task completion, 35 and careful alignment of individual and group performance metrics, 36,37 although none of these approaches can be easily applied to the problem of plume traversal. Extensive peer-to-peer communication is undesirable, because the overhead of providing and maintaining each agent's unique identification, as well as a possibly exponentially increasing number of messages, makes it difficult to scale to large group sizes.…”

Emergent Specialization in Biologically Inspired Collective Behavior Systems

“…Such approaches have become popular in collective behavior task domains where one does not know, a priori, the degree of specialization required to optimally solve the given task (Stanley, Bryant, & Miikkulainen, 2005b), (Waibel, Floreano, Magnenat, & Keller, 2006), (Gautrais, Theraulaz, Deneubourg, & Anderson, 2002), (Potter et al, 2001), (Luke & Spector, 1996), (Theraulaz, Bonabeau, & Deneubourg, 1998b), , (Murciano, Millan, & Zamora, 1997).…”

Parallel learning in heterogeneous multi-robot swarms

“…Multirobot learning using several methods in a wide variety of scenarios has been explored [2], [23]. Specialization in multiagent systems using reinforcement learning was studied in [18]. Techniques for increasing individual learning speed via multi-robot learning were studied in [8] and [14].…”