Algorithmically identifying strategies in multi-agent game-theoretic environments

Zaroukian, Erin; Rodriguez, Sebastian S.; Barton, Sean L.; Schaffer, James; Perelman, Brandon S.; Waytowich, Nicholas R.; Hoffman, Blaine; Asher, Derrik E.

doi:10.1117/12.2518609

Cited by 9 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Pearson correlation [23] and KL divergence [24] are widely used to measure the relationship between two random variables, to match the probability of the desired strategy [25], or to identify strategy switching [26], but these metrics produce a symmetric correlation between variables and do not address the fact that correlation does not imply causation. In economics and ecosystems, statistical tests including Granger Causality [27] and Convergence Cross Mapping (CCM) [12] have been developed to detect the causality between time series of two variables.…”

Section: Related Workmentioning

confidence: 99%

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Ghadami

Bayrak

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Coordination in multi-agent systems is a vital component in teaming effectiveness. In dynamically changing situations, agent decisions depict emergent coordination strategies from following pre-defined rules to exploiting incentive-driven policies. While multi-agent reinforcement learning shapes team behaviors from experience, interpreting learned coordination strategies offers benefits in understanding complex agent dynamics and further improvement in developing adaptive strategies for evolving and unexpected situations. In this work, we develop an approach to quantitatively measure team coordination by collecting decision time series data, detecting causality between agents, and identifying statistically high coordinated sub-teams in decentralized multiagent task allocation operations. We focus on multi-agent systems with homogeneous agents and homogeneous tasks as the strategy formation is more ambiguous and challenging than heterogeneous teams with specialized capabilities. Emergent team coordination is then analyzed using rule-based and reinforcement learning-based strategies for task allocation in operations at different demand stages (stress) levels. We also investigate correlation vs. causation and agent over-or under-estimating demand levels.

show abstract

Section: Related Workmentioning

confidence: 99%

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Ghadami

Bayrak

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Asher et al 37 proposed the adoption of militarily relevant tactics to a simulated predator-prey pursuit task that allowed for teams of deep RL agents to engage in various tactics through capability modifications such as decoys, traps, and camouflage. Furthermore, this research group has introduced methods for measuring coordination [38][39][40] toward a framework for integrating AI agents into mixed soldier-agent teams. 41,42 With respect to simulators built using existing game engines, Fu et al 43 applied deep RL to the C2 of air defense operations.…”

Section: Background and Related Workmentioning

confidence: 99%

On games and simulators as a platform for development of artificial intelligence for command and control

Goecks

Waytowich

Asher

et al. 2022

Journal of Defense Modeling & Simulation

View full text Add to dashboard Cite

Games and simulators can be a valuable platform to execute complex multi-agent, multiplayer, imperfect information scenarios with significant parallels to military applications: multiple participants manage resources and make decisions that command assets to secure specific areas of a map or neutralize opposing forces. These characteristics have attracted the artificial intelligence (AI) community by supporting development of algorithms with complex benchmarks and the capability to rapidly iterate over new ideas. The success of AI algorithms in real-time strategy games such as StarCraft II has also attracted the attention of the military research community aiming to explore similar techniques in military counterpart scenarios. Aiming to bridge the connection between games and military applications, this work discusses past and current efforts on how games and simulators, together with the AI algorithms, have been adapted to simulate certain aspects of military missions and how they might impact the future battlefield. This paper also investigates how advances in virtual reality and visual augmentation systems open new possibilities in human interfaces with gaming platforms and their military parallels.

show abstract

“…Coordination is typically ill-defined in multi-agent tasks and is often used to indicate that a group of agents has performed successfully in some cooperative task domain. In prior work, various novel methods were developed and employed to measure the interdependence between agent actions while performing cooperative tasks, to confirm that these agents had in fact learned to coordinate [16][17][18][19][20][21][22][23] . The confirmation of coordination is a precursor to establishing that a MAS is capable of working with its partners instead of simply taking actions that result in some measure of optimality.…”

Section: Challengesmentioning

confidence: 99%

Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination

Asher,

Basak,

Fernandez

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement learning (RL) approaches can illuminate emergent behaviors that facilitate coordination across teams of agents as part of a multi-agent system (MAS), which can provide windows of opportunity in various military tasks. Technologically advancing adversaries pose substantial risks to a friendly nation's interests and resources. Superior resources alone are not enough to defeat adversaries in modern complex environments because adversaries create standoff in multiple domains against predictable military doctrine-based maneuvers. Therefore, as part of a defense strategy, friendly forces must use strategic maneuvers and disruption to gain superiority in complex multi-faceted domains such as multi-domain operations (MDO). One promising avenue for implementing strategic maneuver and disruption to gain superiority over adversaries is through coordination of MAS in future military operations. In this paper, we present overviews of prominent works in the RL domain with their strengths and weaknesses for overcoming the challenges associated with performing autonomous strategic maneuver and disruption in military contexts.

show abstract

Algorithmically identifying strategies in multi-agent game-theoretic environments

Cited by 9 publications

References 16 publications

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

On games and simulators as a platform for development of artificial intelligence for command and control

Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination

Contact Info

Product

Resources

About