Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors

Abstract

To successfully manipulate in unknown environments, a robot must be able to perceive degrees of freedom of objects in its environment. Based on the resulting kinematic model and joint configurations, the robot is able to select and adapt actions, recognize their successful completion and detect failure. We present an RGB-D-based online algorithm for the interactive perception of articulated objects. The algorithm decomposes the perception problem into three interconnected levels of recursive estimation. The estimation problems at each level are much simpler than the original problem and their robustness is improved by level-specific priors that help reject noise in the measurements. These three estimators mutually inform each other to further improve the convergence properties of the three estimation solutions. We demonstrate that the resulting algorithm is robust, accurate, and versatile in realworld experiments. We also show how the perceptual skill can be used online to control the robot's behavior in real-world manipulation tasks.

An integrated approach to visual perception of articulated objects

“…We distinguish two main approaches to visual tracking: based on a known shape model of the object [2], and based on sufficient texture using point features [4] or dense optical flow [5,7]. While tracking with a known model is more accurate, texturebased approaches are also applicable to unknown objects.…”

“…In our previous work we have shown that tracking can also greatly benefit from a known kinematic structure [4].…”

“…1) Feature-based Tracking and Kinematic Structure Estimation: To bootstrap our pipeline, we acquire information about object motion and location by using our previously developed combined approach to motion tracking and kinematic structure estimation Online Interactive Perception (Online IP) [4]. Online IP is composed of three interconnected processes: a 3-D point feature tracker maintains a constant set of tracked point features.…”

“…On the other hand, to track the pose of an object, methods commonly rely on knowledge of the object's shape and its segmentation in the image [2,3]. Similarly, the estimation of kinematic structure of an unknown object is facilitated by knowing the poses of its rigid parts-but knowing the kinematic structure can also improve pose estimation [4]. Since each of these problems requires input that is provided by the others, we propose to combine them in a synergistic manner so that each sub-problem provides helpful information to the others.…”

Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems

“…For example, SLAM [48] couples localization and mapping in a recursive estimation loop to solve the joint problem more effectively. More recently, combining interaction and perception led to advances in robot perception [6,27,34].…”