Efficient Evaluation Functions for Multi-rover Systems

Abstract

Evolutionary computation can be a powerful tool in cresting a control policy for a single agent receiving local continuous input. This paper extends single-agent evolutionary computation to multi-agent systems, where a collection of agents strives to maximize a global fitness evaluation function that rates the performance of the entire system. This problem is solved in a distributed manner, where each agent evolves its own population of neural networks that are used as the control policies for the agent. Each agent evolves its population using its own agent-specific fitness evaluation function. We propose to create these agent-specific evaluation functions using the theory of collectives to avoid the coordination problem where each agent evolves a population that maximizes its own fitness function, yet the system has a whole achieves low values of the global fitness function. Instead we will ensure that each fitness evaluation function is both "aligned" with the global evaluation function and is LLlearnable,'' i.e., the agents can readily see how their behavior affects their evaluation function. We then show how these agent-specific evaluation functions outperform global evaluation methods by up to 600% in a domain where a set of rovers attempt to maximize the amount of information observed while navigating through a simulated environment.

Emergent specialization in the extended multi-rover problem

“…The evaluation criterion for the rover group was the number of communications received by the lander. In order to avoid the problem where each rover evolves neural networks that maximize their own fitness function, yet the system as a whole achieves low values of the global fitness function, we used a difference evaluation function [1] to evaluate group fitness.…”

“…The conventional neuro-evolution method was based on the research of [1], [21], where each of the n rovers in the group maintained and evolved its own population of complete neural network controllers. This approach differs from the CONE method (section II-B), which evolved n populations of neurons, from which n neural controllers were constructed.…”

“…We propose an extended version of the multi-rover task [1], [21], in which the complexity of the rovers and environment is increased. As in the original rover problem, each rover attempts to maximize its own private evaluation function (subsection IV-E.2) in order to maximize a global evaluation function (subsection IV-E.1).…”

“…Adapted from the research of [1], this task simulation was characterized by a set of rovers operating on a discrete two dimensional plane (200 x 200 quadrants) which served as a survey area for a simulated search and find mission. At the start of a simulation 100 rovers and 1 lander were initialized in a random quadrant (x, y, where: 0 ≤ x, y < 200).…”

“…NE has been highlighted as being most appropriately applied to multiagent tasks that are neither effectively addressed via pure evolutionary or neural computation methods [8]. NE methods have been successfully applied to the rover task domain [1], [21]. However, extending this domain to include the notion of using NE to facilitate emergent specialization, in order to increase task performance, has not yet been investigated.…”