Hierarchical Policies for Cluttered-Scene Grasping With Latent Plans

Wang, Lirui; Meng, Xiangyun; Yu, Xi; Fox, Dieter

doi:10.1109/lra.2022.3143198

Cited by 20 publications

(8 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hierarchical RL [81], [139] [66], [71], [94], [95], [103], [129] Perfect simulator [40], [77], [90], [109] [42], [46], [137], [141] Domain randomization [84], [98], [108], [124] [91], [102], [123], [128] Domain adaptation [17], [43], [52], [64], [75], [104], [110], [143] adversarial network (GAN) capable of translating real tactile images to simulated depth images. Ning et al [97] introduce an autonomous robotic ultrasound imaging system, where the observation is a concatenated latent vector of two conventional autoencoders.…”

Section: Guided Rl Methods Sourcementioning

confidence: 99%

“…Nachum et al [94] employ a hierarchy to learn low-level goal reaching skills coordinated by a highlevel controller for coordinated multiagent object manipulation. Wang et al [129] apply a hierarchical policy in a cluttered-scene grasping cluttered scene grasping setting that learns an embedding space on expert plans and chooses sampled plans via a critic as well as appropriate options [122] via an option classifier. Finally, Li et al [71] adopt a hierarchical structure for interactive navigation tasks, where a high-level policy generates subgoals and selects low-level policies returning task phase-specific robot actions.…”

Section: Hierarchical Rlmentioning

confidence: 99%

See 1 more Smart Citation

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Eßer

Bach

Jestel

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

ecent successes aside, reinforcement learning (RL) still faces significant challenges in its application to the real-world robotics domain. Guiding the learning process with additional knowledge offers a potential solution, thus leveraging the strengths of data-and knowledge-driven approaches. However, this field of research encompasses several disciplines and hence would benefit from a structured overview.In this article, we propose a concept of guided RL that provides a systematic approach toward accelerating the training process and improving performance for real-world robotics settings. We introduce a taxonomy that structures guided RL approaches and shows how different sources of knowledge can be integrated into the learning pipeline in a practical way. Based on this, we describe available approaches in this field and quantitatively evaluate their specific impact in terms of efficiency, effectiveness, and sim-to-real transfer within the robotics domain. However, learning control policies in such a way naturally requires many interactions with the environment. This emphasizes the importance of both collecting highquality samples and exploring the search space in a sample-efficient manner. While directly learning on real robots is appealing, it comes along with substantial challenges, such as high sample cost, partial observability, and safety constraints [28]. Hence, simulators are often

show abstract

Section: Guided Rl Methods Sourcementioning

confidence: 99%

Section: Hierarchical Rlmentioning

confidence: 99%

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Eßer

Bach

Jestel

et al. 2023

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

show abstract

“…Initially designed for indoor human-computer interaction, it has been successfully applied in various automation scenarios. As described in [32], algorithms for contour and spatial positioning of planar shapes can be detected using Kinect. Figure 3 shows RGB and depth images captured at 640 × 480 px resolution in our work, used for extracting features such as obstacles, current location, and target positioning.…”

Section: End-to-end Ddpgmentioning

confidence: 99%

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Zhao,

Ding,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the rapid advancement of industrial automation and artificial intelligence technologies, particularly in the textile industry, robotic technology is increasingly challenged with intelligent path planning and executing high-precision tasks. This study focuses on the automatic path planning and yarn-spool-assembly tasks of textile robotic arms, proposing an end-to-end planning and control model that integrates deep reinforcement learning. The innovation of this paper lies in the introduction of a cascaded fuzzy reward system, which is integrated into the end-to-end model to enhance learning efficiency and reduce ineffective exploration, thereby accelerating the convergence of the model. A series of experiments conducted in a simulated environment demonstrate the model’s exceptional performance in yarn-spool-assembly tasks. Compared to traditional reinforcement learning methods, our model shows potential advantages in improving task success rates and reducing collision rates. The cascaded fuzzy reward system, a core component of our end-to-end deep reinforcement learning model, offers a novel and more robust solution for the automated path planning of robotic arms. In summary, the method proposed in this study provides a new perspective and potential applications for industrial automation, especially in the operation of robotic arms in complex and uncertain environments.

show abstract

“…closing a drawer quickly or slowly. We address this inherent multi-modality by auto-encoding contextual data through a latent plan space with a sequence-to-sequence conditional variational auto-encoder (seq2seq CVAE) [13,45]. Conditioning the action decoder on the latent plan allows the policy to use the entirety of its capacity for learning uni-modal behavior.…”

Section: Learning the Low-level Policymentioning

confidence: 99%

Latent Plans for Task-Agnostic Offline Reinforcement Learning

Rosete-Beas¹,

Mees²,

Kalweit³

et al. 2022

Preprint

View full text Add to dashboard Cite

Everyday tasks of long-horizon and comprising a sequence of multiple implicit subtasks still impose a major challenge in offline robot control. While a number of prior methods aimed to address this setting with variants of imitation and offline reinforcement learning, the learned behavior is typically narrow and often struggles to reach configurable long-horizon goals. As both paradigms have complementary strengths and weaknesses, we propose a novel hierarchical approach that combines the strengths of both methods to learn task-agnostic long-horizon policies from high-dimensional camera observations. Concretely, we combine a low-level policy that learns latent skills via imitation learning and a high-level policy learned from offline reinforcement learning for skill-chaining the latent behavior priors. Experiments in various simulated and real robot control tasks show that our formulation enables producing previously unseen combinations of skills to reach temporally extended goals by "stitching" together latent skills through goal chaining with an order-of-magnitude improvement in performance upon state-of-the-art baselines. We even learn one multi-task visuomotor policy for 25 distinct manipulation tasks in the real world which outperforms both imitation learning and offline reinforcement learning techniques.

show abstract

Hierarchical Policies for Cluttered-Scene Grasping With Latent Plans

Cited by 20 publications

References 45 publications

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Guided Reinforcement Learning: A Review and Evaluation for Efficient and Effective Real-World Robotics [Survey]

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Latent Plans for Task-Agnostic Offline Reinforcement Learning

Contact Info

Product

Resources

About