Attribute Based Affordance Detection from Human-Object Interaction Images

Hassan, Mahmudul; Dharmaratne, Anuja

doi:10.1007/978-3-319-30285-0_18

Cited by 14 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(See Section IV for details.) et al [25] proposed a Bayesian network-based affordance detection method that exploits the attribute of the object, actor, and environment. Grabner et al [26] utilized a human skeleton 3D model to learn the action of sitting on a chair to infer whether an object can afford "sitting" action or not.…”

Section: A Affordance Learningmentioning

confidence: 99%

Phrase-Based Affordance Detection via Cyclic Bilateral Interaction

Lu¹,

Zhai²,

Luo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Lift dumbbell PA: lift, lift up, raise, grab, put down, pick up, take down, push, hold up, uplift, cause to raise, hold high, F: exercise, used for exercise of muscle-building E: indoor exercise Pick up chopsticks PA: take and lift upward, hold, grasp, move up and down, hold and lift F: pass food, kitchen utensil E: usually appears in kitchen or dining table AF: usually are made of wood Rolling baseball, croquet ball, golf ball, table tennis ball, tennis ball PA: rolling, move, can roll, move by rotating, roll over, rotate rapidly, turn round and round, rotate, move fast, spin, whirl, move around an axis or a center, cycle, revolve, change orientation or direction, twirl revolve AF: spherical Mix chopsticks, spoon, whisk

show abstract

Section: A Affordance Learningmentioning

confidence: 99%

Phrase-Based Affordance Detection via Cyclic Bilateral Interaction

Lu¹,

Zhai²,

Luo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This task will be complicated if one of those objects is unseen or occluded. [37]. The existence of a human in a scene makes it more complex in terms of actions, events and affordance learning in particular because the affordance is then dependent on the attributes of the human.…”

Section: Challengesmentioning

confidence: 99%

“…racket head detection obstruct [59] indicates the locations of obstructer wall detection break [59] indicates break-sensitive objects glass cups detection pinch-pull [59] indicates objects that should be pulled with punch knob detection hook-pull [59] indicates objects that should be pulled with hooking up handle detection tip-push [59] indicates objects that perform actions after pushing electricity buttons detection warmth [59] indicates warmth objects fireplaces detection illumination [59] indicates light objects lamps detection dry [59] indicates objects that absorb water towels detection walk [59] , [63] indicates places that allow walking gardens detection, segmentation lyable [63] refers to long free space that allow person to lie down bed segmentation reachable [63] refers to object in a scene that is reachable for a person to pick it water bottle from the fridge segmentation movable [64] refers to objects that can be moved around small objects like balls, mugs segmentation TABLE 1: Indoor affordance labels used to detect objects as in studies [57], [49], [58], [60], [61], [59], [63], [64], [65], [62] Offline Online images. Hassan and Dharmaratne [37] proposed an affordance detection method based on the object, human and the ambient environment. They used the objects attributes (physical, material, shape, etc), human attributes (poses) and object-to-object to train their scheme.…”

Section: Feature-engineering Approachesmentioning

confidence: 99%

Visual Affordance and Function Understanding

2021

View full text Add to dashboard Cite

Nowadays, robots are dominating the manufacturing, entertainment, and healthcare industries. Robot vision aims to equip robots with the capabilities to discover information, understand it, and interact with the environment, which require an agent to effectively understand object affordances and functions in complex visual domains. In this literature survey, first, “visual affordances” are focused on and current state-of-the-art approaches for solving relevant problems as well as open problems and research gaps are summarized. Then, sub-problems, such as affordance detection, categorization, segmentation, and high-level affordance reasoning, are specifically discussed. Furthermore, functional scene understanding and its prevalent descriptors used in the literature are covered. This survey also provides the necessary background to the problem, sheds light on its significance, and highlights the existing challenges for affordance and functionality learning.

show abstract

“…Grabner et al detected a surface suitable for sitting in 3D data [9], Kjellstrom et al classified the function of tools from a video [10], and Zhu et al proposed a task-oriented object model for learning construction patterns [11]. Hassan et al integrated human, object, and environmental aspects for attribute modeling and achieved accurate functional detection [12]. Kemp et al detected a tool tip that can be grasped by a robot [13].…”

Section: Releated Workmentioning

confidence: 99%

A New Semantic-Based Tool Detection Method for Robots

Chen

He²,

Chen³

et al. 2021

INT J COMPUT COMMUN, Int. J. Comput. Commun. Control

View full text Add to dashboard Cite

Home helper robots have become more acceptable due to their excellent image recognition ability. However, some common household tools remain challenging to recognize, classify, and use by robots. We designed a detection method for the functional components of common household tools based on the mask regional convolutional neural network (Mask-R-CNN). This method is a multitask branching target detection algorithm that includes tool classification, target box regression, and semantic segmentation. It provides accurate recognition of the functional components of tools. The method is compared with existing algorithms on the dataset UMD Part Affordance dataset and exhibits effective instance segmentation and key point detection, with higher accuracy and robustness than two traditional algorithms. The proposed method helps the robot understand and use household tools better than traditional object detection algorithms.

show abstract

Attribute Based Affordance Detection from Human-Object Interaction Images

Cited by 14 publications

References 23 publications

Phrase-Based Affordance Detection via Cyclic Bilateral Interaction

Phrase-Based Affordance Detection via Cyclic Bilateral Interaction

Visual Affordance and Function Understanding

A New Semantic-Based Tool Detection Method for Robots

Contact Info

Product

Resources

About