Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments

Abstract

Data efficiency in robotic skill acquisition is crucial for operating robots in varied small-batch assembly settings. To operate in such environments, robots must have robust obstacle avoidance and versatile goal conditioning acquired from only a few simple demonstrations. Existing approaches, however, fall short of these requirements. Deep reinforcement learning (RL) enables a robot to learn complex manipulation tasks but is often limited to small task spaces in the real world due to sample inefficiency and safety concerns. Motion planning (MP) can generate collision-free paths in obstructed environments, but cannot solve complex manipulation tasks and requires goal states often specified by a user or objectspecific pose estimator. In this work, we propose a system for efficient skill acquisition that leverages an object-centric generative model (OCGM) for versatile goal identification to specify a goal for MP combined with RL to solve complex manipulation tasks in obstructed environments. Specifically, OCGM enables one-shot target object identification and reidentification in new scenes, allowing MP to guide the robot to the target object while avoiding obstacles. This is combined with a skill transition network, which bridges the gap between terminal states of MP and feasible start states of a sampleefficient RL policy. The experiments demonstrate that our OCGM-based one-shot goal identification provides competitive accuracy to other baseline approaches and that our modular framework outperforms competitive baselines, including a stateof-the-art RL algorithm, by a significant margin for complex manipulation tasks in obstructed environments.

Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment

“…Meanwhile, Lee et al (2020) quantify uncertainty in pose estimation to determine a binary switching strategy using model-based or RL policies. Additionally, Yamada et al (2023) implemented an object-centric generative model to identify goals for motion planning and a skill transition network to facilitate the movement of the endeffector from its terminal state in motion planning to viable starting states of a sample-efficient RL policy. However, these methods require the model of the object, in particular, the manual specification of a goal state in the robot's frame and control policy design (Yamada et al, 2023).…”

“…Additionally, Yamada et al (2023) implemented an object-centric generative model to identify goals for motion planning and a skill transition network to facilitate the movement of the endeffector from its terminal state in motion planning to viable starting states of a sample-efficient RL policy. However, these methods require the model of the object, in particular, the manual specification of a goal state in the robot's frame and control policy design (Yamada et al, 2023). Additionally, they face difficulties in providing comprehensive guidance in both free space and contactrich regions due to the limited motion planning in tasks that require environmental interaction and the scarcity of visual servoing in addressing geometric constraints.…”

“…In response to these challenges, this study proposes a novel OEC task representation within imitation learning (IL) and residual RL frameworks, which are tailored to enable the learning of adaptive operations from minimal demonstrations and interactions. This approach builds upon these foundations of the prior vision model from the model-based methods (Lee et al, 2020;Shi et al, 2021a;Yamada et al, 2023) and the mathematical model from the imitation learning-based methods (Carvalho et al, 2022;Davchev et al, 2022). Our approach distinguishes itself by: (1) streamlining robot programming through extracting via-points from demonstrated end-effector trajectories for task representation, thereby simplifying the reconfiguration costs and improving adaptability.…”

APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation

“…This enables the models to better understand the underlying structure of a scene and capture the relationships between its constituent objects. Early works have conducted unsupervised scene inference and generation in 2D (MONet [22], Slot Attention [9], GENESIS [23], GENESIS-V2 [10]), and for robotics applications using APEX [24], [25]. In both [24], [25] the 2D OCGM, APEX, is utilised for object matching using the learned object-centric latent representation in an object rearrangement task in simulation and a peg-in-hole task in the real world respectively.…”

“…Early works have conducted unsupervised scene inference and generation in 2D (MONet [22], Slot Attention [9], GENESIS [23], GENESIS-V2 [10]), and for robotics applications using APEX [24], [25]. In both [24], [25] the 2D OCGM, APEX, is utilised for object matching using the learned object-centric latent representation in an object rearrangement task in simulation and a peg-in-hole task in the real world respectively. Nevertheless, the 2D reconstruction and 2D bounding boxes predicted by such OCGM are of limited use in a 3D world.…”