Explainable Action Advising for Multi-Agent Reinforcement Learning

Guo, Yue; Campbell, Joseph; Stepputtis, Simon; Li, Ruiyu; Hughes, Dana; Fang, Fei; Sycara, Katia

doi:10.1109/icra48891.2023.10160557

Cited by 5 publications

(2 citation statements)

References 35 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Advising mechanism shares policy-level knowledge, where less experienced agents can take good actions without making decisions themselves. Unfortunately, many methods based on advising mechanism assume the teacher has a well-trained policy (Ilhan, Gow, and Perez Liebana 2021;Anand et al 2021;Guo et al 2023), or have a centralized information structure (Omidshafiei et al 2019;Kim et al 2020;Gupta et al 2021), or increase training overhead (Ilhan, Gow, and Perez Liebana 2021), or are limited to two agents (Omidshafiei et al 2019;Kim et al 2020). Two works that are similar to ours are AdHocTD (da Silva, Glatt, and Costa 2017) and PSAF (Zhu et al 2021), but both are based on tabular Q-learning and lack robustness to suboptimal advice.…”

Section: Knowledge Sharingmentioning

confidence: 99%

Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning

Ba,

Liu,

Chen

et al. 2024

AAAI

View full text Add to dashboard Cite

While decentralized training is attractive in multi-agent reinforcement learning (MARL) for its excellent scalability and robustness, its inherent coordination challenges in collaborative tasks result in numerous interactions for agents to learn good policies. To alleviate this problem, action advising methods make experienced agents share their knowledge about what to do, while less experienced agents strictly follow the received advice. However, this method of sharing and utilizing knowledge may hinder the team's exploration of better states, as agents can be unduly influenced by suboptimal or even adverse advice, especially in the early stages of learning. Inspired by the fact that humans can learn not only from the success but also from the failure of others, this paper proposes a novel knowledge sharing framework called Cautiously-Optimistic kNowledge Sharing (CONS). CONS enables each agent to share both positive and negative knowledge and cautiously assimilate knowledge from others, thereby enhancing the efficiency of early-stage exploration and the agents' robustness to adverse advice. Moreover, considering the continuous improvement of policies, agents value negative knowledge more in the early stages of learning and shift their focus to positive knowledge in the later stages. Our framework can be easily integrated into existing Q-learning based methods without introducing additional training costs. We evaluate CONS in several challenging multi-agent tasks and find it excels in environments where optimal behavioral patterns are difficult to discover, surpassing the baselines in terms of convergence rate and final performance.

show abstract

Section: Knowledge Sharingmentioning

confidence: 99%

Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning

Ba,

Liu,

Chen

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Performance estimates have been achieved so far using black-box inferences of representations following LLMs performance, requiring interpretation of specific beliefs a priori using some function-approximation mechanism (e.g. Oguntola et al, 2023). Likewise, allowing greater efficiency may reduce the expense of continually needing to scale models.…”

Section: Considerations For Artificial Agentsmentioning

confidence: 99%

A Standard Framework for Social Cognition: interoperable algorithms for inference and representation

Barnby,

Alon,

Bellucci

et al. 2023

Preprint

View full text Add to dashboard Cite

Interpersonal relationships are a central feature of what it is to be human. Theory of Mind (ToM), or mentalising, is the ability to represent the hidden thoughts and beliefs of the self and others to navigate these relationships. However, to date there is no common set of formal principles that can be used to understand interpersonal relationships across context, disorder, and diversity, and in humans and artificial agents. Here we present formal generative principles of social representation and inference as a general research framework that provides a language for expressing falsifiable theories that explain what ToM is, and what it is not. We discuss the state-of-the-art in how this framework accommodates dysfunctional ToM, the neural implementation of ToM, the implications for considering ToM within Artificial Intelligence, and some future directions for the field. This provides an initial computational structure for a wide range of social psychological processes.

show abstract

Extraction and Transfer of General Deep Feature in Reinforcement Learning

Chen,

Pan,

et al. 2024

2024 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Explainable Action Advising for Multi-Agent Reinforcement Learning

Cited by 5 publications

References 35 publications

Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning

Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning

A Standard Framework for Social Cognition: interoperable algorithms for inference and representation

Extraction and Transfer of General Deep Feature in Reinforcement Learning

Contact Info

Product

Resources

About