Learn to Grasp Via Intention Discovery and Its Application to Challenging Clutter

Abstract

Humans excel in grasping objects through diverse and robust policies, many of which are so probabilistically rare that exploration-based learning methods hardly observe and learn. Inspired by the human learning process, we propose a method to extract and exploit latent intents from demonstrations, and then learn diverse and robust grasping policies through selfexploration. The resulting policy can grasp challenging objects in various environments with an off-the-shelf parallel gripper.The key component is a learned intention estimator, which maps gripper pose and visual sensory to a set of sub-intents covering important phases of the grasping movement. Subintents can be used to build an intrinsic reward to guide policy learning. The learned policy demonstrates remarkable zero-shot generalization from simulation to the real world while retaining its robustness against states that have never been encountered during training, novel objects such as protractors and user manuals, and environments such as the cluttered conveyor.

Approximate Task Tree Retrieval in a Knowledge Network for Robotic Cooking

“…Various studies have explored the use of LLMs to generate step-bystep plans for long-horizon tasks. For instance, Erra et al [19] proposed an approach that employs LLMs to generate plans for complex tasks. [20], [21], [22] have also utilized LLMs for plan generation in different domains.…”