The Assistive Kitchen &amp;#x2014; A demonstration scenario for cognitive technical systems

Beetz, Michael; Stulp, Freek; Radig, Bernd; Bandouch, Jan; Blodow, Nico; Dolha, Mihai Emanuel; Fedrizzi, Andreas; Jain, Dominik; Klank, Ulrich; Kresse, Ingo; Maldonado, Alexis; Márton, Zoltán-Csaba; Mösenlechner, Lorenz; Ruiz, Federico; Rusu, Radu Bogdan; Tenorth, Moritz

doi:10.1109/roman.2008.4600634

Cited by 42 publications

(28 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The TUM Kitchen Data Set has been recorded in the TUM kitchen [4], a sensor-equipped intelligent kitchen environment used to perform research on assistive technologies for helping elderly or disabled people to improve their quality of life. The recorded (synchronized) data consists of calibrated videos, motion capture data and recordings from the sensor network, and is well-suited for evaluating approaches for motion tracking as well as for activity recognition.…”

Section: Datamentioning

confidence: 99%

See 1 more Smart Citation

The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition

Tenorth

Bandouch

Beetz

2009

2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops

Self Cite

168

162

View full text Add to dashboard Cite

We introduce the publicly available TUM Kitchen Data Set as a comprehensive collection of activity sequences recorded in a kitchen environment equipped with multiple complementary sensors. The recorded data consists of observations of naturally performed manipulation tasks as encountered in everyday activities of human life. Several instances of a table-setting task were performed by different subjects, involving the manipulation of objects and the environment. We provide the original video sequences, fullbody motion capture data recorded by a markerless motion tracker, RFID tag readings and magnetic sensor readings from objects and the environment, as well as corresponding action labels. In this paper, we both describe how the data was computed, in particular the motion tracker and the labeling, and give examples what it can be used for. We present first results of an automatic method for segmenting the observed motions into semantic classes, and describe how the data can be integrated in a knowledge-based framework for reasoning about the observations.

show abstract

Section: Datamentioning

confidence: 99%

“…RFID and reed sensor data was recorded using the sensor network in the kitchen [4]. Tagged objects include the placemat, the napkin, the plate and the cup.…”

Section: Sensor-equipped Kitchen Environmentmentioning

confidence: 99%

The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition

Tenorth

Bandouch

Beetz

2009

2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops

Self Cite

168

162

View full text Add to dashboard Cite

show abstract

“…The final model in XML format can be imported into a 3D simulation environment (Gazebo) as seen in Figure 14, where models of kitchens were already used for task planning and optimization [15].…”

Section: Discussion and Functional Map Usagementioning

confidence: 99%

“…We are currently investigating ways of adding more semantic information to our maps, by making use of sensor networks as we presented in [15], [16], and plan to further extend our current results by using techniques such as active mapping (i.e. automatically detect when an area needs to be rescanned or scanned at a higher resolution).…”

Section: Discussionmentioning

confidence: 99%

Functional object mapping of kitchen environments

Rusu

Márton

Blodow

et al. 2008

2008 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-In this paper we investigate the acquisition of 3D functional object maps for indoor household environments, in particular kitchens, out of 3D point cloud data. By modeling the static objects in the world into hierarchical classes in the map, such as cupboards, tables, drawers, and kitchen appliances, we create a library of objects which a household robotic assistant can use while performing its tasks.Our method takes a complete 3D point cloud model as input, and computes an object model for it. The objects have states (such as open and closed), and the resulted model is accurate enough to use it in physics-based simulations, where the doors can be opened based on their hinge position. The model is built through a series of geometrical reasoning steps, namely: planar segmentation, cuboid decomposition, fixture recognition and interpretation (e.g. handles and knobs), and object classification based on object state information.

show abstract

“…A complete humanoid robot is equipped with two legs, two arms with hands or grippers, an upper body and a head. A humanoid robot could serve as as a household assistant [7], that e.g. supports elderly citizens in their homes [92].…”

Section: Service Robotsmentioning

confidence: 99%

Knowledge Representations for Planning Manipulation Tasks

Zacharias

2012

Cognitive Systems Monographs

View full text Add to dashboard Cite

A complex manipulation task for a robot has to be planned on different levels of abstraction before it can be executed on a real system. At a high-level of abstraction, a task planner generates a sequence of subtasks from a given abstract command, its pre-and postconditions and a current world model. To restrict the already huge search space of the task planner, aspects like geometric information are not contained in the world model at this level. The task planner then assigns the subtasks to planners on a lower level of abstraction like path planners or grasp planners. These planners rely on geometric information to compute their solutions. However, without considering geometric information the task planner cannot determine good parameters for these low-level planners. Therefore, the derived plans may not be valid for a given situation and may require backtracking. Knowledge representations can bridge the gap between these planning levels and thereby enable scene reasoning. The kinematic capabilities of a robot are one aspect the task planner has to be aware of. Therefore this thesis introduces a novel general representation of the kinematic capabilities of a robot arm. The versatile workspace is defined to describe in which orientations the end effector attached to a robot arm can reach a position. The capability map is a calculable representation of the versatile workspace that allows to determine how well regions of the workspace are reachable. It accurately represents the versatile workspace of arbitrary arm kinematics and enables autonomous taskspecific reasoning. Furthermore, its visualization scheme is intuitively comprehensible for the human. The versatile applicability of the capability map is shown by examples from several distinct application domains. In human-robot interaction, the capability map is used to objectively evaluate a bi-manual interface for tele-operation. The results can also supplement the design process for humanoid robot design. In low-level geometric planning, the capability map is used to reduce the search space and to parameterize path planners and grasp planners. Thus, in a manipulation task for a humanoid robot more human-like motion is planned while simultaneously the computation time is reduced. In high-level task reasoning, the capability map is used to evaluate how well a robot is suited for a task. In an example a humanoid robot has to perform a task involving 3D trajectories. Regions are extracted from the capability map that permit the task execution and the suitability of the robot is inferred.

show abstract

The Assistive Kitchen — A demonstration scenario for cognitive technical systems

Cited by 42 publications

References 9 publications

The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition

The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition

Functional object mapping of kitchen environments

Knowledge Representations for Planning Manipulation Tasks

Contact Info

Product

Resources

About