Discover tool a ordances hooking, from RGBD observations of unknown objects reaching hammering & match expected task semantics across A ordance predictions and are similar to those of a human labeller, e.g., for hammering. human labeller Train an a ordance model and predict distributions over pairs of keypoints to argmax to detect sparse keypoints representing tool geometry by learning from the contact data of sampled trajectories. grasp and interact with for each task model predictions grasp interactFig. 1: Rather than relying on human labels, the GIFT framework discovers affordances from goal-directed interaction with a set of procedurally-generated tools. This interaction experience is collected with a simple sampling-based motion planner that does not require demonstrations or an expert policy. Since the affordances are not prespecified (either explicitly by labels or implicitly by predefined manipulation strategies), they are unbiased, i.e., they emerge only from the constraints of the task.

No abstract

No abstract