Semantic Object Maps for robotic housework - representation, acquisition and use

Abstract

Abstract-In this article we investigate the representation and acquisition of Semantic Objects Maps (SOMs) that can serve as information resources for autonomous service robots performing everyday manipulation tasks in kitchen environments. These maps provide the robot with information about its operation environment that enable it to perform fetch and place tasks more efficiently and reliably. To this end, the semantic object maps can answer queries such as the following ones: "What do parts of the kitchen look like?", "How can a container be opened and closed?", "Where do objects of daily use belong?", "What is inside of cupboards/drawers?", etc.The semantic object maps presented in this article, which we call SOM + , extend the first generation of SOMs presented by Rusu et al. [1] in that the representation of SOM + is designed more thoroughly and that SOM + also include knowledge about the appearance and articulation of furniture objects. Also, the acquisition methods for SOM + substantially advance those developed in [1] in that SOM + are acquired autonomously and with low-cost (Kinect) instead of very accurate (laser-based) 3D sensors. In addition, perception methods are more general and are demonstrated to work in different kitchen environments. I. INTRODUCTIONRobots that do not know where objects are have to search for them. Robots that do not know how objects look have to guess whether they have fetched the right one. Robots that do not know the articulation models of drawers and cupboards have to open them very carefully in order to not damage them. Thus, robots should store and maintain knowledge about their environment that enables them to perform their tasks more reliably and efficiently. We call the collection of this knowledge the robot's maps and consider maps to be models of the robot's operation environment that serve as information resources for better task performance. Robots build environment maps for many purposes. Most robot maps so far have been proposed for navigation. Robot maps for navigation enable robots to estimate their position in the environment, to check the reachability of the destination and to compute navigation plans. Depending on their purpose maps have to store different kinds of information in different forms. Maps might represent the occupancy of environment of 2D or 3D grid cells, they might contain landmarks or represent the topological structure of the environment. The maps might model objects of daily use, indoor, outdoor, underwater, extraterrestrial surfaces, and aerial environments.

Hierarchical Semantic Labeling for Task-Relevant RGB-D Perception

“…Katz and Brock [22] developed interactive segmentation for observing object motion during manipulation. Pangercic et al [36] built semantic object maps for manipulation tasks for an autonomous service robot. Hinkle and Edwin [18] proposed a technique for functionally classifying objects using features obtained through physical simulations.…”

Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency

“…Several shape detection methods have been proposed in the computer vision community that can cope with uncertainties and clutter in the data sets [17,25]. Additionally, the current trend in 3D point cloud interpretation is to infer higher level knowledge [23].…”

Semi-task-dependent and uncertainty-driven world model maintenance

“…In Pangercic et al (2012) a first generation semantic object maps (SOMs) is extended. The so-called SOM + map builds upon earlier work (Rusu et al 2009;Tenorth et al 2010) and stores information about the pose, appearance and category of objects.…”