OPD: Single-View 3D Openable Part Detection

CA$^2$T-Net: Category-Agnostic 3D Articulation Transfer from Single Image

“…Part Segmentation In computer vision, most research on 2D or 3D part segmentation focuses on semantic [19,33] or functional [7,9,10,24] meanings of the parts. Jiang et al [9] extended 2D semantic segmentation networks (e.g. Mask R-CNN) to predict the location of openable parts from a single-view image.…”

“…Wei et al, [25] trained an implicit representation of the geometry, appearance, and motion of an object in a self-supervised manner from image observations, however their approach is limited to certain classes of articulated objects. Recently, Jiang et al, [9] introduced a method for estimating 3D motion parameters from single-view RGB image. Their approach takes only a single image and aims to predict motion parameters for all possible valid parts, rather than focusing on transferring motion from one object to another.…”

“…We consider three baseline approaches to single-view articulation transfer: selecting a random motion and part (RandMot), selecting the most frequent motion parameters (FreqMot), and OPD-O [9], the current state-of-the-art approach for 3D motion prediction. RandMot and FreqMot are inspired by similar baselines used by OPD [9], however modified to predict additional outputs such as object pose and motion magnitude. RandMot gives a lower bound on performance whereas FreqMot is a relatively strong heuristic-based baseline approach.…”

“…Specifically, given a single RGB image, I, showing how an object can be articulated, and a 3D mesh model M in its rest state, our goal is to infer the pose, motion parameters, and 3D part segmentation of M such that we can re-pose and trans-Method Task Input Pose 3D Part Seg. Category-Specific DeepPartInduction [32] Transfer articulation from PC → PC 2 PCs × No Shape2Motion [24] Predict motion for all moveable parts PC × No RPM-Net [30] Predict motion for all moveable parts PC × No ANCSH [15] Articulated pose estimation PC Yes NASAM [25] Reconstruct and animate objects RGBs + pose × Yes OPD [9] Predict motion for all openable parts RGB × No CA form it to match the articulation in the input image. Our method is both category-agnostic and structure-agnostic, as it makes no assumption on the topology of the input model and inferences can be made on objects from arbitrary categories.…”

“…To our knowledge, our work is the first to pose the problem of single-view articulation transfer onto a target 3D object. As indicated in Table 1, the most relevant works [9,15,24,30,32] differ from our problem setting in one way or another. We call our network CA 2 T-Net for category-agnostic articulation transfer, and demonstrate that our method has superior performance in single-view 3D articulation transfer compared to baseline methods and the current state-of-theart in terms of motion prediction.…”