Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

Lee, Timothy E.; Zhao, Jialiang; Sawhney, Amrita S.; Girdhar, Siddharth; Kroemer, Oliver

doi:10.1109/icra48506.2021.9561439

Cited by 12 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work (Seitzer, Schölkopf, and Martius 2021) proposes integrating measures of causal influence into reinforcement learning algorithms to address the problems of exploration and learning in the robot manipulation environment. The work (Lee et al 2021) utilizes causal intervention to identify the most relevant state variables for completing a task, thereby reducing the dimensionality of the state space. The work (Foerster et al 2018) utilizes influence detection to create counterfactual data to enhance the training of RL agents.…”

Section: Causality In Reinforcement Learningmentioning

confidence: 99%

Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning

Du,

Ye,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

Learning to collaborate has witnessed significant progress in multi-agent reinforcement learning (MARL). However, promoting coordination among agents and enhancing exploration capabilities remain challenges. In multi-agent environments, interactions between agents are limited in specific situations. Effective collaboration between agents thus requires a nuanced understanding of when and how agents' actions influence others.To this end, in this paper, we propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning (SCIC), which incorporates a novel Intrinsic reward mechanism based on a new cooperation criterion measured by situation-dependent causal influence among agents.Our approach aims to detect inter-agent causal influences in specific situations based on the criterion using causal intervention and conditional mutual information. This effectively assists agents in exploring states that can positively impact other agents, thus promoting cooperation between agents.The resulting update links coordinated exploration and intrinsic reward distribution, which enhance overall collaboration and performance.Experimental results on various MARL benchmarks demonstrate the superiority of our method compared to state-of-the-art approaches.

show abstract

Section: Causality In Reinforcement Learningmentioning

confidence: 99%

Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning

Du,

Ye,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Some works explore causality to distinguish between taskrelevant and -irrelevant variables [10]. For example, CREST [11] uses causal interventions on environment variables to discover which of the variables affect an RL policy. They find that excluding irrelevant variables positively impacts generalizability and sim-to-real transfer.…”

Section: A Causality In Roboticsmentioning

confidence: 99%

Why Did I Fail? A Causal-Based Method to Find Explanations for Robot Failures

Diehl

Ramirez-Amaro

2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Robot failures in human-centered environments are inevitable. Therefore, the ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this letter are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on the obtained model. We address I) by learning a causal Bayesian network from simulation data. Concerning II), we propose a novel method that enables robots to generate contrastive explanations upon task failures. The explanation is based on setting the failure state in contrast with the closest state that would have allowed for successful execution. This state is found through breadth-first search and is based on success predictions from the learned causal model. We assessed our method in two different scenarios I) stacking cubes and II) dropping spheres into a container. The obtained causal models reach a sim2real accuracy of 70% and 72%, respectively. We finally show that our novel method scales over multiple tasks and allows real robots to give failure explanations like "the upper cube was stacked too high and too far to the right of the lower cube."

show abstract

“…Causal reasoning is beneficial to the pursuit of reliable humancomputer interaction by uncovering, modeling the heterogeneous spatial-temporal information in a reliable and explainable way. Especially for robot interaction [88,[218][219][220], where the relevant environmental features are not known in advance, prior knowledge can be utilized as good candidate causal structures. The strong relation between causal reasoning and its ability to intervene in the world suggests that causal reasoning can can greatly address this challenge for robotics, which benefits the application of robotics significantly.…”

Section: Future Directionsmentioning

confidence: 99%

Causal Reasoning Meets Visual Representation Learning: A Prospective Study

Liu¹,

Yushen²,

Yan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Spatial-temporal representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. The majority of the existing methods tend to fit the original data/variable distributions and ignore the essential causal relations behind the multi-modal knowledge, which lacks an unified guidance and analysis about why modern spatial-temporal representation learning methods are easily collapse into data bias and have limited generalization and cognitive abilities. Inspired by the strong inference ability of human-level agents, recent years have therefore witnessed great effort in developing causal reasoning paradigms to realize robust representation and model learning with good cognitive ability. In this paper, we conduct a comprehensive review of existing causal reasoning methods for spatialtemporal representation learning, covering fundamental theories, models, and datasets. The limitations of current methods and datasets are also discussed. Moreover, we propose some primary challenges, opportunities, and future research directions for benchmarking causal reasoning algorithms in spatialtemporal representation learning. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods, publicly available benchmarks, and consensus-building standards for reliable spatial-temporal representation learning and related real-world applications more efficiently.

show abstract

Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

Cited by 12 publications

References 21 publications

Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning

Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning

Why Did I Fail? A Causal-Based Method to Find Explanations for Robot Failures

Causal Reasoning Meets Visual Representation Learning: A Prospective Study

Contact Info

Product

Resources

About