Learning Strategic Group Formation for Coordinated Behavior in Adversarial Multi-Agent with Double DQN

Diallo, Elhadji Amadou Oury; Sugawara, Toshiharu

doi:10.1007/978-3-030-03098-8_30

Cited by 7 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address the issue of black box decisions made by artificial intelligence, explainable reinforcement learning [17], [18], [19] tends to evaluate the interestingness elements in agent decisions such as frequency, execution uncertainty, and favorable/adverse situations [20]. Several studies have investigated learned formation strategies [21] and path planning strategies using heat maps [22]. However, theses approaches have not explained the correlation or causality effect between agents, and they are not general enough to merely use state and decision information.…”

Section: Related Workmentioning

confidence: 99%

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Ghadami

Bayrak

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Coordination in multi-agent systems is a vital component in teaming effectiveness. In dynamically changing situations, agent decisions depict emergent coordination strategies from following pre-defined rules to exploiting incentive-driven policies. While multi-agent reinforcement learning shapes team behaviors from experience, interpreting learned coordination strategies offers benefits in understanding complex agent dynamics and further improvement in developing adaptive strategies for evolving and unexpected situations. In this work, we develop an approach to quantitatively measure team coordination by collecting decision time series data, detecting causality between agents, and identifying statistically high coordinated sub-teams in decentralized multiagent task allocation operations. We focus on multi-agent systems with homogeneous agents and homogeneous tasks as the strategy formation is more ambiguous and challenging than heterogeneous teams with specialized capabilities. Emergent team coordination is then analyzed using rule-based and reinforcement learning-based strategies for task allocation in operations at different demand stages (stress) levels. We also investigate correlation vs. causation and agent over-or under-estimating demand levels.

show abstract

Section: Related Workmentioning

confidence: 99%

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Ghadami

Bayrak

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Diallo et al [ 55 ] proposed that deep reinforcement learning algorithms can be used to cooperate between two agents to achieve a specific task. The fully observable ping-pong scenario tested different deep reinforcement learning algorithms by teaming up the two agents to play against the hard-coded player.…”

Section: Related Studiesmentioning

confidence: 99%

“…Learning. Diallo et al [55] proposed that deep reinforcement learning algorithms can be used to cooperate between two agents to achieve a specific task.…”

Section: Introduction Of Cooperation and Emotions Through Reinforcementmentioning

confidence: 99%

Improving the Survival Time of Multiagents in Social Dilemmas through Neurotransmitter-Based Deep Q-Learning Model of Emotions

Hassan

Shahid

Hayat

et al. 2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

In multiagent systems, social dilemmas often arise whenever there is a competition over the limited resources. The major challenge is to establish cooperation among intelligent virtual agents for solving the situations of social dilemmas. In humans, personality and emotions are the primary factors that lead them toward a cooperative environment. To make agents cooperate, they have to become more like humans, that is, believable. Therefore, we hypothesize that emotions according to the personality give birth to believability, and if believability is introduced into agents through emotions, it improves their survival rate in social dilemma situations. The existing researches have introduced different computational models to introduce emotions in virtual agents, but they lack emotions through neurotransmitters. We have proposed a neurotransmitters-based deep Q-learning computational model in multiagents that is a suitable choice for emotion modeling and, hence, believability. The proposed model regulates the agents’ emotions by controlling the virtual neurotransmitters (dopamine and oxytocin) according to the agent’s personality. The personality of the agent is introduced using OCEAN model. To evaluate the proposed system, we simulated a survival scenario with limited food resources in different experiments. These experiments vary the number of selfish agents (higher neuroticism personality trait) and the selfless agents (higher agreeableness personality trait). Experimental results show that by adding the selfless agents in the scenario, the agents develop cooperation, and their collective survival time increases. Thus, to resolve the social dilemma problems in virtual agents, we can make agents believable through the proposed neurotransmitter-based emotional model. This proposed work may help in developing nonplayer characters (NPCs) in games.

show abstract

“…Research on MADRL has received much attention in recent years [11,18,24] For example, Diallo et al [4] showed that a large number of agents can behave cooperatively and generate strategic team formations of more than 100 agents by sharing a centralized DQN with teammates in adversarial multi-agent games. Shao et al [20] proposed a curriculum transfer learning method for MADRL to solve problems with difficult scenarios in StarCraft, a real-time strategy game, and this method accelerates the training process, thereby improving the learning performance.…”

Section: Related Workmentioning

confidence: 99%

Analysis of coordinated behavior structures with multi-agent deep reinforcement learning

Miyashita

Sugawara

2020

Appl Intell

Self Cite

View full text Add to dashboard Cite

Cooperation and coordination are major issues in studies on multi-agent systems because the entire performance of such systems is greatly affected by these activities. The issues are challenging however, because appropriate coordinated behaviors depend on not only environmental characteristics but also other agents’ strategies. On the other hand, advances in multi-agent deep reinforcement learning (MADRL) have recently attracted attention, because MADRL can considerably improve the entire performance of multi-agent systems in certain domains. The characteristics of learned coordination structures and agent’s resulting behaviors, however, have not been clarified sufficiently. Therefore, we focus here on MADRL in which agents have their own deep Q-networks (DQNs), and we analyze their coordinated behaviors and structures for the pickup and floor laying problem, which is an abstraction of our target application. In particular, we analyze the behaviors around scarce resources and long narrow passages in which conflicts such as collisions are likely to occur. We then indicated that different types of inputs to the networks exhibit similar performance but generate various coordination structures with associated behaviors, such as division of labor and a shared social norm, with no direct communication.

show abstract

Learning Strategic Group Formation for Coordinated Behavior in Adversarial Multi-Agent with Double DQN

Cited by 7 publications

References 7 publications

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

Improving the Survival Time of Multiagents in Social Dilemmas through Neurotransmitter-Based Deep Q-Learning Model of Emotions

Analysis of coordinated behavior structures with multi-agent deep reinforcement learning

Contact Info

Product

Resources

About