No abstract

No abstract

Multi agent strategies in mixed cooperativecompetitive environments can be hard to craft by hand because each agent needs to coordinate with its teammates while competing with its opponents. Learning based algorithms are appealing but many scenarios require heterogeneous agent behavior for the team's success and this increases the complexity of the learning algorithm. In this work, we develop a competitive multi agent environment called FortAttack in which two teams compete against each other. We corroborate that modeling agents with Graph Neural Networks and training them with Reinforcement Learning leads to the evolution of increasingly complex strategies for each team. We observe a natural emergence of heterogeneous behavior amongst homogeneous agents when such behavior can lead to the team's success. Such heterogeneous behavior from homogeneous agents is appealing because any agent can replace the role of another agent at test time. Finally, we propose ensemble training, in which we utilize the evolved opponent strategies to train a single policy for friendly agents.

Control of robotic swarms through control over a leader(s) has become the dominant approach to supervisory control over these largely autonomous systems. Resilience in the face of attrition is one of the primary advantages attributed to swarms yet the presence of leader(s) makes them vulnerable to decapitation. Algorithms which allow a swarm to hide its leader are a promising solution. We present a novel approach in which neural networks, NNs, trained in a graph neural network, GNN, replace conventional controllers making them more amenable to training. Swarms and an adversary intent of finding the leader were trained and tested in 4 phases: 1-swarm to follow leader, 2-adversary to recognize leader, 3-swarm to hide leader from adversary, and 4-swarm and adversary compete to hide and recognize the leader. While the NN adversary was more successful in identifying leaders without deception, humans did better in conditions in which the swarm was trained to hide its leader from the NN adversary. The study illustrates difficulties likely to emerge in arms races between machine learners and the potential role humans may play in moderating them.

Control of robotic swarms through control over a leader(s) has become the dominant approach to supervisory control over these largely autonomous systems. Resilience in the face of attrition is one of the primary advantages attributed to swarms yet the presence of leader(s) makes them vulnerable to decapitation. Algorithms which allow a swarm to hide its leader are a promising solution. In prior work we found that using a graph neural network, GNN, a swarm could be trained to flock following a leader. An Adversary NN trained to identify that leader (naïve condition) performed substantially better than human observers. When the swarm was trained to hide its leader (deception conditions), however, the advantage reversed with humans outperforming the Adversary. This human advantage persisted even when the swarm and Adversary were jointly trained, allowing the Adversary to adapt to the swarm’s evolving strategies for hiding its leader. The present study investigates the robustness of human leader identification by testing identifications made in the presence of medium and high levels of visual clutter. Clutter degraded human performance to some extent but human accuracy in leader identification remained well above that of the Adversary in deception conditions. Human performance even approached that for an unhidden leader under joint training. This study confirms the robustness of the human superiority effect and argues for the inclusion of humans in AI systems which may confront learned deception.