3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding

Deng, Shengheng; Xu, Xun; Wu, Chaozheng; Chen, Ke; Jia, Kui

doi:10.48550/arxiv.2103.16397

Cited by 2 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Object affordances Several methods have been proposed for detecting functional object parts and their corresponding affordance labels. These works involve the detection of object affordance parts by considering their visual and geometric features (Deng et al, 2021;Wang & Tarr, 2020). One of the early works in this direction focused in the detection of graspable object areas by creating local visual descriptors of grasping points and estimating the probability of the presence of a graspable object based on the Bernoulli trial (Montesano & Lopes, 2009).…”

Section: Literature Reviewmentioning

confidence: 99%

“…One of the early works in this direction focused in the detection of graspable object areas by creating local visual descriptors of grasping points and estimating the probability of the presence of a graspable object based on the Bernoulli trial (Montesano & Lopes, 2009). New approaches employ Convolutional Neural Network (CNN) models to produce classes of functional object parts from RGB (Nguyen et al, 2017;Do et al (Montesano & Lopes, 2009) 2D & Keypoints (Zhao & Zhu, 2013) 2D & 3D (Myers et al, 2015) 2.5D (Nguyen et al, 2016) 2.5D (Nguyen et al, 2017) 2D (Kokic et al, 2017) Synthetic (Sawatzky et al, 2017) 2D & Keypoints (Do et al, 2018) 2D (Wang & Tarr, 2020) 2D (Deng et al, 2021) 3D (Xu et al, 2021) 2.5D & Keypoints (Turek et al, 2010) 2D (Qi et al, 2018) 3D (Kjellström et al, 2011) 2D (Yao et al, 2013) 3D (Qi et al, 2017) 2.5D (Gkioxari et al, 2018) 2D (Fang et al, 2018) 2D (Chuang et al, 2018) 2D (Tan et al, 2019) 2D (Wu et al, 2020) Synthetic (Hou et al, 2021) 2D (Sridhar et al, 2008) 2D (Aksoy et al, 2010) 2D (Aksoy et al, 2011) 2D (Pieropan et al, 2013) 2.5D (Pieropan et al, 2014) 2D (Moldovan & De Raedt, 2014) Synthetic (Liang et al, 2016) 2.5D (Liang et al, 2018) 2.5D 2018; Sawatzky et al, 2017) and synthetic data (Kokic et al, 2017). However, depth cues along with the RGB information have demonstrated a greater detection accuracy in this task (Nguyen et al, 2016;…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Object-agnostic Affordance Categorization via Unsupervised Learning of Graph Embeddings

Toumpa

Cohn²

2023

jair

View full text Add to dashboard Cite

Acquiring knowledge about object interactions and affordances can facilitate scene understanding and human-robot collaboration tasks. As humans tend to use objects in many different ways depending on the scene and the objects’ availability, learning object affordances in everyday-life scenarios is a challenging task, particularly in the presence of an open set of interactions and objects. We address the problem of affordance categorization for class-agnostic objects with an open set of interactions; we achieve this by learning similarities between object interactions in an unsupervised way and thus inducing clusters of object affordances. A novel depth-informed qualitative spatial representation is proposed for the construction of Activity Graphs (AGs), which abstract from the continuous representation of spatio-temporal interactions in RGB-D videos. These AGs are clustered to obtain groups of objects with similar affordances. Our experiments in a real-world scenario demonstrate that our method learns to create object affordance clusters with a high V-measure even in cluttered scenes. The proposed approach handles object occlusions by capturing effectively possible interactions and without imposing any object or scene constraints.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Object-agnostic Affordance Categorization via Unsupervised Learning of Graph Embeddings

Toumpa

Cohn²

2023

jair

View full text Add to dashboard Cite

show abstract

“…A benchmark for object affordances has been recently introduced [14] to evaluate point cloud deep learning networks. Our framework complements the aforementioned approaches by introducing manifolds that contain possible grasp points from which an algorithm can sample and execute a grasp.…”

Section: Related Workmentioning

confidence: 99%

GraspME -- Grasp Manifold Estimator

Hager¹,

Bauer²,

Toussaint³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we introduce a Grasp Manifold Estimator (GraspME) to detect grasp affordances for objects directly in 2D camera images. To perform manipulation tasks autonomously it is crucial for robots to have such graspability models of the surrounding objects. Grasp manifolds have the advantage of providing continuously infinitely many grasps, which is not the case when using other grasp representations such as predefined grasp points. For instance, this property can be leveraged in motion optimization to define goal sets as implicit surface constraints in the robot configuration space. In this work, we restrict ourselves to the case of estimating possible end-effector positions directly from 2D camera images. To this extend, we define grasp manifolds via a set of keypoints and locate them in images using a Mask R-CNN [1] backbone. Using learned features allows to generalize to different view angle, with potentially noisy images, and objects that were not part of the training set. We rely on simulation data only and perform experiments on simple and complex objects, including unseen ones. Our framework achieves an inference speed of 11.5 fps on a GPU, an average precision for keypoint estimation of 94.5% and a mean pixel distance of only 1.29. This shows that we can estimate the objects very well via bounding boxes and segmentation masks as well as approximate the correct grasp manifold's keypoint coordinates.

show abstract

3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding

Cited by 2 publications

References 32 publications

Object-agnostic Affordance Categorization via Unsupervised Learning of Graph Embeddings

Object-agnostic Affordance Categorization via Unsupervised Learning of Graph Embeddings

GraspME -- Grasp Manifold Estimator

Contact Info

Product

Resources

About