On the use of probabilistic relational affordance models for sequential manipulation tasks in robotics

Abstract

In this paper we employ probabilistic relational affordance models in a robotic manipulation task. Such affordance models capture the interdependencies between properties of multiple objects, executed actions, and effects of those actions on objects. Recently it was shown how to learn such models from observed video demonstrations of actions manipulating several objects. This paper extends that work and employs those models for sequential tasks. Our approach consists of two parts. First, we employ affordance models sequentially in order to recognize the individual actions making up a demonstrated sequential skill or high level concept. Second, we utilize the models of concepts to plan a suitable course of action to replicate the observed consequences of a demonstration. For this we adopt the framework of relational Markov decision processes. Empirical results show the viability of the affordance models for sequential manipulation skills for object placement.

Affordances in Psychology, Neuroscience, and Robotics: A Survey

“…A few computational models of affordances deal with multiobjects scenarios, either in terms of tool use [214]- [223] or pairwise object interaction [224], [225], with the longterm objective of obtaining more complex problem solving abilities in autonomous robots. A robot agent specifically tailored towards learning tool use is reported by Wood et al [214].…”

“…Moreover, it is worth noting that in [215]- [220] the properties of the acted objects are not explicitly considered in the model; only the general affordances of tools are learned, regardless of the objects that the tools act upon. The works of Moldovan et al [224], [225] consider a multi-object scenario in which the relational affordances between objects pairs are exploited to plan a sequence of actions to achieve a desired goal, using probabilistic reasoning. The pairwise interactions are described in terms of the objects relative distance, orientation and contact; however, they do not investigate how these interactions are affected by different geometrical properties of the objects.…”

Geometric Affordance Perception: Leveraging Deep 3D Saliency With the Interaction Tensor

“…Later works have built on Montesano's model and have used statistical relational learning to encode relations between (afforded) actions and percepts . For instance, extending the model to encode the effects of single-object actions relative to other objects, which allows the robot to learn two-object relational affordances (Moldovan et al, 2012 ), higher-level manipulation actions such as makeSpace or moveAround (Moldovan et al, 2013 ), and two-arm manipulation (Moldovan and De Raedt, 2014a ). Furthermore, with a similar model, Moldovan and De Raedt ( 2014b ) learn co-occurrence probabilities for occluded object search using a list of object properties and afforded-actions.…”

A Survey of Knowledge Representation in Service Robotics

“…The robot's interaction with its environment serves to either learn new motor primitives or skills (trajectory-level) or to learn new properties associated with the type of grasp they make or the skills they use, the object's physical features, and the effects that occur from executing an action (symbolic-level). In works such as [64,65] [66,67,68] [69], a robot can use basic, pre-programmed motor skills (viz. grasping, tapping or touching) to learn about relationships between an object's features (such as shapes, sizes or textures) and features of its actions (such as velocities and point-of-contact).…”