Visual Identification of Articulated Object Parts

Abstract

As autonomous robots interact and navigate around real-world environments such as homes, it is useful to reliably identify and manipulate articulated objects, such as doors and cabinets. Many prior works in object articulation identification require manipulation of the object, either by the robot or a human. While recent works have addressed predicting articulation types from visual observations alone, they often assume prior knowledge of category-level kinematic motion models or sequence of observations where the articulated parts are moving according to their kinematic constraints. In this work, we propose training a neural network through largescale domain randomization to identify the articulation type of object parts from a single image observation. Training data is generated via photorealistic rendering in simulation. Our proposed model predicts motion residual flows of object parts, and these residuals are used to determine the articulation type and parameters. We train the network on six object categories with 149 objects and 100K rendered images, achieving an accuracy of 82.5%. Experiments show our method generalizes to novel object categories in simulation and can be applied to real-world images without fine-tuning.1 https://sim2realai.github.io/ Synthetic-Datasets-of-Objects-Part-I/

AKB-48: A Real-World Articulated Object Knowledge Base

“…For tackling with unseen objects, Li et al [16] follow the pose estimation setting and propose a normalized coordinate space to estimate 6D pose and joint state for articulated objects. In terms of jointcentered perception tasks, several works attempt to mine joint configurations of articulated objects [11,18,33]. To investigate manipulation points for articulated objects from visual input, Mo et al attempt to define six types of action primitives and predict interactions [23].…”

OPD: Single-view 3D Openable Part Detection

“…More recent work has started to explore training of single models for motion prediction across categories and structures [10,11,11,16]. Zeng et al [44] proposed an optical flow-based approach on RGB-D images given segmentation masks of the moving part and fixed part. They evaluate only on ground truth segmentation and do not investigate how part segmentation and detection influences the accuracy of motion prediction.…”

Registering Articulated Objects With Human-in-the-loop Corrections

“…As an example, Anguelov et al [7] decompose an articulated mesh into approximate rigid parts and use Expectation Maximization (EM) to estimate part assignments and transformations. Other recent methods focus on estimating articulation of novel objects though images [8], [9], [10] or physical interaction [11], [12]. For example, Jain et al [13] learn a distribution over articulation model parameters for novel objects with different degrees of freedom.…”