Interactive Perception of Articulated Objects

Abstract

We present a skill for the perception of three-dimensional kinematic structures of rigid articulated bodies with revolute and prismatic joints. The ability to acquire such models autonomously is required for general manipulation in unstructured environments. Experiments on a mobile manipulation platform with realworld objects under varying lighting conditions demonstrate the robustness of the proposed method. This robustness is achieved by integrating perception and manipulation capabilities: the manipulator interacts with the environment to move an unknown object, thereby creating a perceptual signal that reveals the kinematic properties of the object. For good performance, the perceptual skill requires the presence of trackable visual features in the scene.

Probabilistic Segmentation and Targeted Exploration of Objects in Cluttered Environments

“…If 3D-data is available, 3D features such as surface normals or curvature might additionally be exploited [10,11]. However, visual or spatial boundaries need not always correspond to object boundaries [12,13], so not all ambiguities can be resolved [12,[14][15][16]]. An alternative is to look at video streams [17]; however, in real-robot setups there may be too much (self-)occlusion for this strategy to be viable.…”

“…For learning how actions change the state of objects [13,15,[21][22][23][24], interaction is even required, as the necessary information is not present in static (visual) data.…”

“…In a subsequent stage, movement and object membership can be estimated for each of these parts. Alternative approaches use algorithms such as iterative closest point (ICP) to determine whether the tracked point cloud is a single rigid object [12,31], or estimate the movement of trackable visual features [13,16,25,[31][32][33].…”

“…For example, Katz et al [13] used hand-tuned predictors based on compactness and appearance, in addition to co-movement of features. Bergström et al [16] combined rigid motion clues with color-and disparity clues, whereas Schiebener et al [25] validated hypotheses based on proximity and shared parametric surfaces using co-movement.…”

“…The methods discussed in the previous paragraph used visual features to create hypotheses that were tested by interaction [16,25,33] or to modify binary potentials between parts [13]. Instead, a probabilistic approach offers a principled way to integrate noisy clues from multiple sources.…”