Learning Causal Relationships of Object Properties and Affordances Through Human Demonstrations and Self-Supervised Intervention for Purposeful Action in Transfer Environments

Abstract

Learning object affordances enables robots to plan and perform purposeful actions. However, a fundamental challenge for the utilization of affordance knowledge lies in its generalization to unknown objects and environments. In this paper we present a new method for learning causal relationships between object properties and object affordances which can be transferred to other environments. Our approach, implemented on a PR2 robot, generates hypotheses of property-affordance models in a toy environment based on human demonstrations that are subsequently tested through interventional experiments.The system relies on information theory to choose experiments for maximal information gain, performs them self-supervised and uses the observed outcome to iteratively refine the set of candidate causal models. The learned causal knowledge is humaninterpretable in the form of graphical models, stored in the knowledge graph. We validate our method through a task requiring affordance knowledge transfer to three different unknown environments. Our results show that extending learning from human demonstrations by causal learning through interventions led to a 71.7% decrease in model uncertainty and improved affordance classification in the transfer environments on average by 47.49%.