Affordance-map: Mapping human context in 3D scenes using cost-sensitive SVM and virtual human models

Piyathilaka, Lasitha; Kodagoda, Sarath

doi:10.1109/robio.2015.7419073

Cited by 9 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Using RGB-D images of indoor scenes [24] perform segmentation for human actions such as walkable, sittable, lyable. Similarly, affordances are studied in [23,8,10] to map locations suitable for sitting, or laying down; particularly in these cases using human skeleton hallucinated on the different indoor scenes. Crucially, these previous methods are heavy in terms of requiring multiple learning examples, impose a particular parameterization such as detection of planes or shapes and or are highly specific to an object e.g.…”

Section: Related Workmentioning

confidence: 99%

“…At test time, we are able to predict affordance location candidates by approximating the iT on a previously unseen input scene. The method allows us to use a model of say a humanoid skeleton and predict human affordances such as sitting; similarly to [8,10,23,24]. But importantly, it also allows us to build these tensors more generally for any other pair of objects.…”

Section: Our Approachmentioning

confidence: 99%

See 1 more Smart Citation

Where can i do this? Geometric Affordances from a Single Example with the Interaction Tensor

Ochoa

Mayol-Cuevas

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

This paper develops and evaluates a new tensor field representation to express the geometric affordance of one object over another. We expand the well known bisector surface representation to one that is weight-driven and that retains the provenance of surface points with directional vectors. We also incorporate the notion of affordance keypoints which allow for faster decisions at a point of query and with a compact and straightforward descriptor. Using a single interaction example, we are able to generalize to previously-unseen scenarios; both synthetic and also real scenes captured with RGBD sensors. We show how our interaction tensor allows for significantly better performance over alternative formulations. Evaluations also include crowdsourcing comparisons that confirm the validity of our affordance proposals, which agree on average 84% of the time with human judgments, and which is 20-40% better than the baseline methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Our Approachmentioning

confidence: 99%

Where can i do this? Geometric Affordances from a Single Example with the Interaction Tensor

Ochoa

Mayol-Cuevas

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…We humans perceive and interact with the surrounding 3D world to accomplish everyday tasks. To equip AI systems with the same capabilities, researchers have done many works investigating shape functionality [Guan et al, 2020, Hu et al, 2016, Lai et al, 2021, grasp affordance [Brahmbhatt et al, 2019, Corona et al, 2020, Fang et al, 2018, Jiang et al, 2021, Kjellström et al, 2011, Kokic et al, 2020, Lenz et al, 2015, Mandikal & Grauman, 2021, Montesano & Lopes, 2009, Nagarajan et al, 2019, Redmon & Angelova, 2015, Yang et al, 2020, manipulation affordance [Do et al, 2018, Mo et al, 2021a, Nagarajan et al, 2019, scene affordance , Piyathilaka & Kodagoda, 2015, Rhinehart & Kitani, 2016, etc. While these works mostly study single-object/scene or agent-object interaction scenarios, our work explores inter-object relationships.…”

Section: Related Workmentioning

confidence: 99%

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

Li¹,

Mo²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Building embodied intelligent agents that can interact with 3D indoor environments has received increasing research attention in recent years. While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -inter-object functional relationships (e.g., a switch on the wall turns on or off the light, a remote control operates the TV). Humans often spend little or no effort to infer these relationships, even when entering a new room, by using our strong prior knowledge (e.g., we know that buttons control electrical devices) or using only a few exploratory interactions in cases of uncertainty (e.g., multiple switches and lights in the same room). In this paper, we take the first step in building AI system learning inter-object functional relationships in 3D indoor environments with key technical contributions of modeling prior knowledge by training over large-scale scenes and designing interactive policies for effectively exploring the training scenes and quickly adapting to novel test scenes. We create a new benchmark based on the AI2Thor and PartNet datasets and perform extensive experiments that prove the effectiveness of our proposed method. Results show that our model successfully learns priors and fast-interactive-adaptation strategies for exploring inter-object functional relationships in complex 3D scenes. Several ablation studies further validate the usefulness of each proposed module.

show abstract

“…Affordance Reasoning. There is a growing interest in reasoning object affordances in the field of computer vision [8]- [14] and robotics [15]- [20]. Stoytchev [15] introduces an approach to ground tool affordances via dynamically applying different behaviors from a behavioral repertoire.…”

Section: Related Workmentioning

confidence: 99%

Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

Wu¹,

Meng²,

Ruan³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this letter, we study the problem of autonomously placing a teddy bear on a previously unseen chair for sitting. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it, respectively. Furthermore, to cope with the cases where the chair is not in an accessible pose for placing the bear, a human-robot interaction (HRI) framework is introduced in which a human follows language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously put the teddy bear on the 12 unseen chairs with a very high success rate. The HRI framework is also shown to be very effective in changing the accessibility of the chair. Source code will be available. Video demos are available at https: //chirikjianlab.github.io/putbearonchair/.

show abstract

Affordance-map: Mapping human context in 3D scenes using cost-sensitive SVM and virtual human models

Cited by 9 publications

References 14 publications

Where can i do this? Geometric Affordances from a Single Example with the Interaction Tensor

Where can i do this? Geometric Affordances from a Single Example with the Interaction Tensor

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

Contact Info

Product

Resources

About