Limitations of Theory of Mind Defenses against Deception in Multi-Agent Systems

Abstract

The presence of deceptive agents within a multi-agent system can exert a detrimental influence on the performance and stability ofother agents in shared environments. In multi-agent interactions, the utilization of Theory of Mind (ToM) has become a considerable mech-anism for defenses against deception where it can possibly provide defenses with robustness and resilience to adverse effects against diverseforms of deception. ToM is the cognitive process through which an agent ascribes states to other agents, encompassing beliefs, objec-tives, intentions, desires, or any other characteristic. This paper investigates into the effectiveness and inherent limitations of employ-ing ToM-based defenses. We leverage stochastic games and Bayesian games, specifically highlighting formulations of Multi-Agent Reinforce-ment Learning (MARL) to establish the boundaries of ToM as a defense.