Robot Task Planning via Deep Reinforcement Learning: a Tabletop Object Sorting Application

Ceola, Federico; Tosello, Elisa; Tagliapietra, Luca; Nicola, Giorgio; Ghidoni, Stefano

doi:10.1109/smc.2019.8914278

Cited by 9 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saravanan et al [189] considered the minimization of traveling time and total energy and the maximization of manipulability to plan the trajectory for a robotic sorter with payload constraints. Currently, to remove the manual teaching process and hard-coded programs, several studies have used reinforcement learning [190] and active learning [191] methods to enable robots to learn sorting motions for static objects. To carefully manipulate the tangled and densely cluttered mixed waste items, such as in Fig.…”

Section: Planning and Executionmentioning

confidence: 99%

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Kiyokawa

Takamatsu

Koyanaka

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

To achieve recycling of mixed industrial waste toward an advanced sustainable society, waste sorting automation through robots is crucial and urgent. For this purpose, a robot is required to recognize the category, shape, pose, and condition of different waste items and manipulate them according to the category to be sorted. This survey considers three potential difficulties in the sorting automation: 1) End-effector: to robustly grasp and manipulate different waste items with dirt and deformations; 2) Sensor: to recognize the category, shape, and pose of existing objects to be manipulated and the wet and dirty conditions of their surfaces; and 3) Planner: to generate feasible and efficient sequences and trajectories. This survey includes 76 references to studies related to automatic waste sorting and 159 references to worldwide waste recycling attempts. This pioneering investigation reveals the possibility and limitations of conventional systems; thus, providing insights on open issues and potential technologies to achieve a robot-incorporated sorter for the chaotic mixed waste is one of its contributions. This paper further presents a system design policy for readers and discusses future advanced sorters, thereby contributing to the field of robotics and automation.Note to Practitioners-Most automated sorting systems operate for limited target waste items. This study is motivated by the automation of mixed industrial waste treatment facilities using advanced robotic sorters. Emerging advances and increasing functionalities of robot system components will widen system applicability and increase use cases in the chaotic mixed industrial waste domain. This paper surveys the research conducted to date, discusses open issues and potential approaches, and presents user guides that provide practitioners with a system design policy. The user guides created according to the strengths and weaknesses of each system configuration provide future researchers and developers with a useful a priori design policy that has been thus far validated on efficiency, quality, productivity, and reliability. A question-and-answer style guide Manuscript

show abstract

Section: Planning and Executionmentioning

confidence: 99%

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Kiyokawa

Takamatsu

Koyanaka

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…Learning multi-step tasks such as stacking [16,17,18], assembling [19,20,21,22,23] and sorting [5,24] are always more challenging because more primitive actions and action constraints are involved. In ref.…”

Section: Related Workmentioning

confidence: 99%

Learn multi-step object sorting tasks through deep reinforcement learning

et al. 2022

View full text Add to dashboard Cite

Robotic systems are usually controlled to repetitively perform specific actions for manufacturing tasks. The traditional control methods are domain-dependent and model-dependent with cost of much human efforts. They cannot meet the new requirements of generality and flexibility in many areas such as intelligent manufacturing and customized production. This paper develops a general model-free approach to enable robots to perform multi-step object sorting tasks through deep reinforcement learning. Taking projected heightmap images from different time steps as input without extra high-level image analysis and understanding, critic models are designed to produce a pixel-wise Q value map for each type of action. It is a new trial to apply pixel-wise Q value-based critic networks to solve multi-step sorting tasks that involve many types of actions and complex action constraints. The experimental validations on simulated and realistic object sorting tasks demonstrate the effectiveness of the proposed approach. Qualitative results (videos), code for simulated and realistic experiments, and pre-trained models are available at https://github.com/JiatongBao/DRLSorting

show abstract