Learning intermediate object affordances: Towards the development of a tool concept

Learning at the ends: From hand to tool affordances in humanoid robots

“…2. This structure is similar to the one that gave us the best generalization results in our previous work [20], thanks to its dimensionality reduction which reduced the number of edges, therefore the computational complexity of training and testing, and most importantly it reduces the amount of training data required to observe the emergence of some learning effect.…”

“…A recent overview is given in [2]. The idea behind robots learning object affordances, or action possibilities, is that knowledge acquired through self-exploration and interaction with the world can be used to make inferences in new scenarios and tasks, such as prediction [16], tool use [17]- [20], and planning [21], [22]. Recently, Schoeler and Wörgötter [23] introduced a framework for analyzing tools by modeling their dynamics and providing object graphs based on their parts, to support the conjecture of a cognitive hand-to-tool transfer (e.g., to understand that a helmet and a hollow skull can both be used to transport water).…”

“…The features that we extract are pre-categorical shape descriptors computed as geometric relationships between perimeter, area, convex hull and approximated shapes of the segmented silhouettes of the objects in front of the robot, and they are based on [32]. Our visual feature extraction is similar to our previous works [19], [20], [22], however it now incorporates a richer set of 13 features instead of 5: convexity, eccentricity, compactness, circularity, squareness, number of convexity defects (i.e., number of cavities along the contour, for example the "holes" between fingers in a hand image), and seven central normalized moments. It is worth noting that we do not classify or label our shapes into classes, but we merely reason about their shape-derived features: this gives our system some flexibility, making it able to process unknown situations not seen during training.…”

“…4b depicts the effect of using the elongated Figure 3. Structure of the Bayesian Network that we use to model the affordance relationships between robot manipulator, target object, motor action, and resulting effects, similarly to [20]. The raw visual features of manipulators and objects are real-valued and normalized between 0 and 1 (see Sec.…”

Detecting Features of Tools, Objects, and Actions from Effects in a Robot using Deep Learning

“…In each of these studies, robots did not consider tools, objects, and actions simultaneously, the way humans do. A study by Goncalves et al [10] focused on both tools and objects, but they aimed to determine features of tools and objects on the basis of categories, such as area and circularity, set in advance by the experimenters. Therefore, it was difficult for the robot to acquire the features autonomously and manipulate arbitrary objects with arbitrary tools without requiring human assistance.…”