Relational affordances for multiple-object manipulation

Abstract

The concept of affordances has been used in robotics to model action opportunities of a robot and as a basis for making decisions involving objects. Affordances capture the interdependencies between the objects and their properties, the executed actions on those objects, and the effects of those respective actions. However, existing affordance models cannot cope with multiple objects that may interact during action execution. Our approach is unique in that possesses the following four characteristics. First, our model employs recent advances in probabilistic programming to learn affordance models that take into account (spatial) relations between different objects, such as relative distances. Two-object interaction models are first learned from the robot interacting with the world in a behavioural exploration stage, and are then employed in worlds with an arbitrary number of objects. The model thus generalizes over both the number of and the particular objects used in the exploration stage, and it also effectively deals with uncertainty. Secondly, rather than using a (discrete) action repertoire, the actions are parametrised according to the motor capabilities of the robot, which allows to model and achieve goals at several levels of complexity. It also supports a two-arm parametrised action. Thirdly, the relational affordance model represents the state of the world using both discrete (action and object features) and continuous (effects) random variables. The effects follow a multivariate Gaussian distribution with the correlated discrete variables (actions and object properties). Fourthly,

Learning Distributional Programs for Relational Autocompletion

“…To the best of our knowledge, the present paper makes the first attempt to learn interpretable hybrid probabilistic logic programs from partially observed probabilistic data as well as background knowledge. DC programs have been successfully applied in robotics and perceptual anchoring using handcrafted programs or by learning parameters of simple programs with defined structure (Moldovan et al 2018;Persson et al 2019). The technique we present in the present paper has already been successfully applied for structure learning in the perceptual anchoring context (Zuidberg Dos Martires et al 2020) and extends these other results.…”

A Survey of Knowledge Representation in Service Robotics

“…The robot's interaction with its environment serves to either learn new motor primitives or skills (trajectory-level) or to learn new properties associated with the type of grasp they make or the skills they use, the object's physical features, and the effects that occur from executing an action (symbolic-level). In works such as [64,65] [66,67,68] [69], a robot can use basic, pre-programmed motor skills (viz. grasping, tapping or touching) to learn about relationships between an object's features (such as shapes, sizes or textures) and features of its actions (such as velocities and point-of-contact).…”

Semantic and geometric reasoning for robotic grasping: a probabilistic logic approach

“…However, none of these frameworks solves the generalization capability needed for task-dependent grasping following a semantic and affordance-based behavior. Relational affordance models for robots have been learned in a multi-object manipulation task context [38,39]. We propose a probabilistic logic framework to infer pre-grasp configurations using task-category affordances.…”

“…... We employ a manually-defined probabilistic affordance model. However, the parameters of the model can be learned to obtain better estimations of the object-task affordances [38,39]. They can be also re-estimated by our probabilistic logic module, given the prior distribution over object categories.…”