Abstract-We propose a framework for the sensor-based estimation of manipulation-relevant object properties and the abstraction of known actions in a learning setup from the observation of humans. The descriptors consists of an objectcentric representation of manipulation constraints and a scenespecific action graph. The graph spans between the typical places, where objects are placed. This framework allows to abstract the strongly varying actions of a human operator and to monitor unexpected new actions, that require a modification of the knowledge stored in the system. The usage of an abstract, object-centric structure enables not only the application of knowledge in the same situation, but also the transfer to similar environments. Furthermore, the information can be derived from different sensing modalities.The proposed system builds up the representation of manipulation-relevant properties and actions. The properties, which are directly related to the object, are stored in the Object Container. The Functionality Map links the actions with the typical action areas in the environment. We present experimental results on real human actions, showing the quality of the results, that can be obtained with our system. I. MOTIVATIONA robot should be able to learn unsupervised through the observation of human actions in its environment. Unfortunately, humans do not follow exact trajectories, while performing repetitive manipulation tasks. The system needs to be able to abstract the manipulation actions, in order to focus only on information, which is necessary to accomplish a manipulation or to cooperate with a human in a given environment. A mismatch between the expectation of the robot as an observer system and a current human action should occur only in situations, in which the change appears to be a result of a changed function or physical property of the object. We will call such a mismatch a surprise event in the following. A surprise event triggers the refinement or the modification of the stored information. Important examples are the following: Motion constraints are suddenly changed in the object transport phase (e.g., a cup carried always upright is now tilted arbitrarily); an object is suddenly placed on an unexpected place, e.g., a cup on the floor. These observations are usually an indication, that the physical properties of the object (e.g., the level of the liquid in the object) or their function (not a drinking cup, but a dirty dish) changed. This needs to be considered in the internal representation of the manipulation system.Our aim is to define a model, that allows to map different physical properties of the object to modifications in the handling properties. The model should efficiently abstract known actions applied to a given object, in order to be able to correctly predict the often strongly varying transport trajectories and goals. This second property of the system allows to reason about changes in the function of a specific object in a given environment. For example, a tool is not used for i...

Abstract-We propose a system for vision-based estimation of manipulation-relevant properties of objects in natural scenes based on observation of human actions. The system consists of an a-priori (Atlas) knowledge about known generic objects in the scene and classifies the scene into mission relevant objects and background geometry that is important only for collision avoidance. We present the object-centric structure of our system consisting of an Atlas representation and a Working Memory storing the current knowledge about the scene, the manipulated objects and actions applied to them in the local environment.We present experimental results how the system maintains the information in the database and we show the quality of the results that can be obtained with our system. I. MOTIVATIONCognitive systems need to be capable of identifying the mission relevance and of learning the model description of objects by themselves during a joint action with a human operator. Most generally, a model of context specifies the entities to observe, the properties to measure and the relations to detect according to [17]. Dey [6] proposed an operational model for context aware perception. In this model, a situation is defined as a configuration of entities and relations relative to a task. The task serves to determine which entities and relations are of interest and should be observed. We transfer these findings into our environment representation, which allows to decouple complex object recognition loops from the low level 3D reconstruction.Sensation and perception are key components of cognitive systems. Cognition can be defined as "generation of knowledge on the basis of perception, reasoning, learning and prior-models". Perception is the main source of information for reasoning and learning capabilities. Scene classification is an important task in cognitive systems. It helps in sensorbased 3D model generation to discriminate between objects interesting for missions (foreground) and background objects relevant merely for localization and obstacle avoidance.A cognitive system is one that is capable of interacting with humans and other systems in an environment and that is capable to respond to an unexpected event that we will refer to as a surprise in the following text. Our system uses the surprise to control the learning about the scene and to trigger its own actions as responses to the external stimuli in the environment. We aim to develop a knowledge representation

Abstract-In order to be able to replace a human operator, a robotic manipulation system needs to deal with a variety of possible actions. These actions may be more or less constrained in their motion profile and in the accuracy of the transport goals. The robotic system can make use of some of this variation to simplify the control to improve the efficiency of the generated motion. Nevertheless, the human's intention behind the manipulation may not change. We introduce the Elastic Power Path to optimize paths with respect to efficient control in the context of abstractly represented tasks.Our experiments show, that the proposed Elastic Power Path is an efficient method to achieve this aim. The magnitude and the number of turnarounds of the accelerations along the path are significantly reduced. I. MOTIVATIONA human, who repeats an action, does usually not use the same paths all the time, but a variety of paths. However, the human has a certain intention regarding the action. This intention is not changing during the repetitions. It can be encapsulated in the characteristic properties of an abstract task representation. When a robotic system is supposed to perform a task instead of the human, it needs to consider the characteristic properties of the task. At the same time, the system can make use of the freedom in planning, since it is not restricted to a certain path. A variety of paths fulfilling the characteristic properties are possible. Of course, it is desirable to select the most efficient one to enable an easy and efficient control. Furthermore, energy can be saved and the strain on the hardware can be reduced. Hence, each joint of the robot should move slowly and smoothly. Abrupt turnarounds of the acceleration should be avoided. But how should the path look like to support efficient dynamics?To sum up, we want to make use of the freedom in planning which comes along with the abstract representation of tasks. We want to optimize the path with respect to efficiency in control.We introduce the Elastic Power Path for this purpose. In analogy to an elastic band, the Elastic Power Path has a certain elasticity. When an elastic band is stretched, additional energy is required. In the context of efficiency, we want to minimize this additional energy. The less stretched the band, the closer desired optimum is. This depends, of course, on the elasticity of the band. Analogously to an elastic band, the Elastic Power Path can be stretched and relaxed. Its elasticity depends on the dynamics of the robot along the path. The more efficient the dynamics, the closer the Elastic Power Path is to its optimum. Our aim is an optimal path with respect to efficient dynamics. We measure the required power in the style of a hiker climbing up and down hills. The required power is increasing, if more hills have to be climbed or if higher hills have to be climbed. The speed and acceleration profiles play an important role regarding the power. In the context of path optimization for a manipulator, we focus on the speed and acceler...

Abstract-Robots are meanwhile able to perform several tasks. But what happens, if one or multiple of the robot's joints fail? Is the robot still able to perform the required tasks? Which capabilities of the robot get limited and which ones are lost?We propose an analysis of manipulator structures for the comparison of a robot's capabilities with respect to efficient control. The comparison is processed (1) within a robot in the case of joint failures and (2) between robots with or without joint failures. It is important, that the analysis can be processed independently of the structure of the manipulator. The results have to be comparable between different manipulator structures. Therefore, an abstract representation of the robot's dynamic capabilities is necessary. We introduce the Maneuverability Volume and the Spinning Pencil for this purpose. The Maneuverability Volume shows, how efficiently the endeffector can be moved to any other position. The Spinning Pencil reflects the robot's capability to change its end-effector orientation efficiently.Our experiments show not only the different capabilities of two manipulator structures, but also the change of the capabilities if one or multiple joints fail.

Abstract-The general solution for inverse kinematics is a problem known for a long time. A lot of problems, algorithms, etc., depend on inverse kinematics. While solutions for specific serial or parallel chain manipulators exist, a good estimation for an arbitrary serial robot within a short computation time is still missing. We provide not only a tool for the discrete estimation of inverse kinematics of arbitrary serial chain manipulators, but also concepts for inverse kinematics optimization along entire paths (adaptive tunneling) and an approximation of a desired grasp with a human-like robotic hand (virtual shut grasp). The latter concept includes an efficient reduction technique of a complex, arbitrary hand, which enables a fast and accurate estimation of the inverse kinematics of an entire hand. In contrast to existing work, our approach is general, since it is neither restricted to certain configurations of the serial manipulator nor to specific structures of the hand. Moreover, our approach does neither rely on proximate starting positions nor does it require specific properties of the objective function concerning the position of the minima. It works even under the presence of multiple local minima in the solution space. Experiments show the performance of our system. The results of the estimation of the inverse kinematics are very accurate and the maximally required joint speeds along the paths are low. I. MOTIVATIONA robot which is supposed to reach a certain position with its end-effector needs to be moved to an appropriate configuration to place the end-effector in the desired position. In many situations, it is advantageous to have a system which takes the desired end-effector position as input and gives one, several or all possible goal configurations as output. The necessary mapping from the robot's workspace to its joint space is called inverse kinematics. The general solution of inverse kinematics is a known problem for a long time. Sometimes, the inverse kinematics can be computed explicitly for manipulators with certain structures. A general estimation framework for the inverse kinematics of an arbitrary serial robot is still missing. It can already be enough to know, whether a solution exits. In other cases, the knowledge about one, several or all solutions is desirable.We focus on the discrete analysis of the configurationspace. Such an analysis is, e.g., necessary for the evaluation of an arbitrary manipulator structure. We are interested in the inverse kinematics of an arbitrary serial chain manipulator at a given point without any incremental methods based on intermediate positions or (partial) derivatives (e.g., through The box should be transported. This means, first, that the object has to be grasped by the robotic hand (brown palm, magenta fingers). A serial chain manipulator has to put the hand into an appropriate position, such that a successful grasp is possible. Several configurations of the manipulator could be possible to reach a point (e.g., blue and red configurations of the...