An integrated approach to visual perception of articulated objects

Martí­n-Martí­n, Roberto; Höfer, Sebastian; Brock, Oliver

doi:10.1109/icra.2016.7487714

Cited by 44 publications

(25 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in contrast to our approach, it can only track one moving object and requires a manual initialization. [12] propose a combined approach for estimating pose, shape, and the kinematic structure of articulated objects based on motion segmentation. While it is also based on joint tracking and segmentation, the focus is on discovering the articulated structure, only foreground objects are reconstructed and its performance is not real time.…”

Section: Related Workmentioning

confidence: 99%

Co-fusion: Real-time segmentation, tracking and fusion of multiple objects

Rünz

Agapito

2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

203

218

View full text Add to dashboard Cite

Fig. 1: A sequence demonstrating our dynamic SLAM system. Three objects were sequentially placed on a table: first a small bin (blue label), a flask (yellow) and a teddy bear (green). The results show that all objects were successfully segmented, tracked and modeled.Abstract-In this paper we introduce Co-Fusion, a dense SLAM system that takes a live stream of RGB-D images as input and segments the scene into different objects (using either motion or semantic cues) while simultaneously tracking and reconstructing their 3D shape in real time. We use a multiple model fitting approach where each object can move independently from the background and still be effectively tracked and its shape fused over time using only the information from pixels associated with that object label. Previous attempts to deal with dynamic scenes have typically considered moving regions as outliers, and consequently do not model their shape or track their motion over time. In contrast, we enable the robot to maintain 3D models for each of the segmented objects and to improve them over time through fusion. As a result, our system can enable a robot to maintain a scene description at the object level which has the potential to allow interactions with its working environment; even in the case of dynamic scenes.

show abstract

Section: Related Workmentioning

confidence: 99%

Co-fusion: Real-time segmentation, tracking and fusion of multiple objects

Rünz

Agapito

2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

203

218

View full text Add to dashboard Cite

show abstract

“…For each cluster, the 3D pose is estimated and the approach [14] is applied to infer the skeleton structure. Recently, [17] presented a method that combines shape reconstruction with the estimation of the skeleton structure. While these approaches operate only with point clouds, our approach generates fully rigged models consisting of a watertight mesh, embedded skeleton, and skinning weights.…”

Section: Related Workmentioning

confidence: 99%

Reconstructing Articulated Rigged Models from RGB-D Videos

Tzionas

Gall

2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Although commercial and open-source software exist to reconstruct a static object from a sequence recorded with an RGB-D sensor, there is a lack of tools that build rigged models of articulated objects that deform realistically and can be used for tracking or animation. In this work, we fill this gap and propose a method that creates a fully rigged model of an articulated object from depth data of a single sensor. To this end, we combine deformable mesh tracking, motion segmentation based on spectral clustering and skeletonization based on mean curvature flow. The fully rigged model then consists of a watertight mesh, embedded skeleton, and skinning weights.

show abstract

“…Schmidt et al, 2014) and building models of articulated objects (e.g. Martín-Martín et al, 2016). Apart from benchmarking, the dataset can also be used to develop data-driven algorithms by exploiting the provided models to generating virtual visual data.…”

Section: Introductionmentioning

confidence: 99%

The RBO dataset of articulated objects and interactions

Martín-Martín

Eppner

Brock

2019

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

We present a dataset with models of 14 articulated objects commonly found in human environments and with RGB-D video sequences and wrenches recorded of human interactions with them. The 358 interaction sequences total 67 minutes of human manipulation under varying experimental conditions (type of interaction, lighting, perspective, and background). Each interaction with an object is annotated with the ground truth poses of its rigid parts and the kinematic state obtained by a motion capture system. For a subset of 78 sequences (25 minutes), we also measured the interaction wrenches. The object models contain textured three-dimensional triangle meshes of each link and their motion constraints. We provide Python scripts to download and visualize the data. The data is available at https://turbo.github.io/articulated-objects/ and hosted at https://zenodo.org/record/1036660/.

show abstract

An integrated approach to visual perception of articulated objects

Cited by 44 publications

References 20 publications

Co-fusion: Real-time segmentation, tracking and fusion of multiple objects

Co-fusion: Real-time segmentation, tracking and fusion of multiple objects

Reconstructing Articulated Rigged Models from RGB-D Videos

The RBO dataset of articulated objects and interactions

Contact Info

Product

Resources

About