Functional Object Class Detection Based on Learned Affordance Cues

Abstract

Current approaches to visual object class detection mainly focus on the recognition of abstract object categories, such as cars, motorbikes, mugs and bottles. Although these approaches have demonstrated impressive performance in terms of recognition, their restriction to abstract categories seems artificial and inadequate in the context of embodied, cognitive agents. Here, distinguishing objects according to functional aspects based on object affordances is vital for a meaningful human-machine interaction. In this paper, we propose a complete system for the detection of functional object classes, based on a representation of visually distinct hints on object affordances (affordance cues). It spans the complete cycle from tutor-driven acquisition of affordance cues, one-shot learning of corresponding object models, and detecting novel instances of functional object classes in real images

Towards Grasp-Oriented Visual Perception for Humanoid Robots

“…The work by Stark et al 53 runs along the lines of the latter. Prehensile parts of objects are represented by k-Adjacent Segments (originally proposed for shape matching) that encode the relative geometric layout of distinct edge segments in an image.…”

“…Saxena et al, 51 and Morales et al 52 and Stark et al 53 apply monocular images to derive suitable grasps. Such 2D approaches avoid the difficult problem of 3D reconstruction, as also their applicability is supported by a number of articles in the field of neurophysiology.…”

An overview of 3D object grasp synthesis algorithms

“…Some approaches associate grasp parameters or hand shapes to object geometric features in order to find good grasps in terms of stability [61,62]. Other techniques learn to identify grasping regions in an object image [63,64]. These techniques are discussed in the following.…”

“…In a similar approach, Stark et al [64] developed a functional approach to affordance learning in which subcategories of the graspable affordance (such as handle-graspable and sidewallgraspable) are learned by observation of human-object interactions. Interaction with specific object parts leads to the development of detectors for specific affordance cues (such as handles).…”

Physically Grounded Spatio-temporal Object Affordances

“…instead of object identities (e.g., chairs, mugs, etc.). Most works take a recognition based approach where they first estimate physical attributes/parts and then jointly reasoned about them to come up with an object hypothesis [38]. Some works predict affordance-based or function-based object attributes.…”