Representation of manipulation-relevant object properties and actions for surprise-driven exploration

Abstract

Abstract-We propose a framework for the sensor-based estimation of manipulation-relevant object properties and the abstraction of known actions in a learning setup from the observation of humans. The descriptors consists of an objectcentric representation of manipulation constraints and a scenespecific action graph. The graph spans between the typical places, where objects are placed. This framework allows to abstract the strongly varying actions of a human operator and to monitor unexpected new actions, that require a modification of the knowledge stored in the system. The usage of an abstract, object-centric structure enables not only the application of knowledge in the same situation, but also the transfer to similar environments. Furthermore, the information can be derived from different sensing modalities.The proposed system builds up the representation of manipulation-relevant properties and actions. The properties, which are directly related to the object, are stored in the Object Container. The Functionality Map links the actions with the typical action areas in the environment. We present experimental results on real human actions, showing the quality of the results, that can be obtained with our system. I. MOTIVATIONA robot should be able to learn unsupervised through the observation of human actions in its environment. Unfortunately, humans do not follow exact trajectories, while performing repetitive manipulation tasks. The system needs to be able to abstract the manipulation actions, in order to focus only on information, which is necessary to accomplish a manipulation or to cooperate with a human in a given environment. A mismatch between the expectation of the robot as an observer system and a current human action should occur only in situations, in which the change appears to be a result of a changed function or physical property of the object. We will call such a mismatch a surprise event in the following. A surprise event triggers the refinement or the modification of the stored information. Important examples are the following: Motion constraints are suddenly changed in the object transport phase (e.g., a cup carried always upright is now tilted arbitrarily); an object is suddenly placed on an unexpected place, e.g., a cup on the floor. These observations are usually an indication, that the physical properties of the object (e.g., the level of the liquid in the object) or their function (not a drinking cup, but a dirty dish) changed. This needs to be considered in the internal representation of the manipulation system.Our aim is to define a model, that allows to map different physical properties of the object to modifications in the handling properties. The model should efficiently abstract known actions applied to a given object, in order to be able to correctly predict the often strongly varying transport trajectories and goals. This second property of the system allows to reason about changes in the function of a specific object in a given environment. For example, a tool is not used for i...

Contact state based representation of object relations in the environment for dexterous manipulations

“…A representation of such properties has already been presented in [1]. We want to go a step further in this paper.…”

“…The aim contact state is, then, the contact of the object with its final location. Hence, the Functionality Map of [1] can be easily used. Moreover, the contact state perspective has advantages concerning the execution of transportation tasks.…”

“…Our work in this paper is inspired from the abstract representation of actions in [1]. In contrast to [1], we focus here on more dexterous manipulation, which requires a much more detailed knowledge about the manipulation and the desired aims.…”

“…In contrast to [1], we focus here on more dexterous manipulation, which requires a much more detailed knowledge about the manipulation and the desired aims.…”

Estimation of inverse kinematics of arbitrary serial chain manipulators and human-like robotic hands

“…The experiments are processed on two data sets extracted from human actions in manipulation experiments (see [17]). The human transported objects between characteristic places.…”