Adaptive Regret Minimization for Learning Complex Team-Based Tactics

Abstract

This paper presents an approach and analysis for performing decentralized cooperative control of a team of decoys to achieve the Honeypot Ambush tactic. In this tactic, the threats are successfully lured into a designated region where they can be easily defeated. The decoys learn to cooperate by incorporating a game-theory-based online-learning method, known as regret minimization, to maximize the team's global reward. The decoy agents are assumed to have physical limitations and to be subject to certain stringent range constraints required for deceiving the networked threats. By employing an efficient coordination mechanism, the agents learn to be less greedy and allow weaker agents to catch up on their rewards to improve team performance. Such a coordination solution corresponds to achieving convergence to coarse correlated equilibrium. The numerical results verify the effectiveness of the proposed solution to achieve a global satisfaction outcome and to adapt to a wide spectrum of scenarios.