Automatic Curriculum Design for Object Transportation Based on Deep Reinforcement Learning

Eoh, Gyuho; Park, Tae-Hyoung

doi:10.1109/access.2021.3118109

Cited by 5 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The other limitation is how much we modify the training environment that is built based on the validation environment, i.e., when generating a training environment, how much variance in the Gaussian distribution related to the number of obstacles, positions, and radii should be adjusted? Recent research has proposed to automatically adjust the difficulty of curriculum environments based on the robot's learning performance in the training environment [39]. Integrating such advances into our study by combining methods for generating appropriate subtasks and automatically generating curriculum environments for each task could address the limitations of our approach.…”

Section: Discussionmentioning

confidence: 99%

“…The robot can dynamically adjust the task difficulty based on the robot's performance, which provides a more appropriate curriculum design compared to the traditional methods [38]. Eoh and Park [39] proposed an automatic curriculum method for object transportation by generating the difficulty map. The curriculum is generated adaptively from easy to difficult environments for object transportation.…”

Section: B Curriculum Learningmentioning

confidence: 99%

See 1 more Smart Citation

Feedback-Based Curriculum Learning for Collision Avoidance

Choi,

Hwang,

Eoh

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

This paper proposes a novel curriculum learning approach for collision avoidance using feedback from the deep reinforcement learning (DRL) training process. Previous research on DRL-based collision avoidance algorithms has encountered challenges such as long training times and difficulty in convergence due to sparse rewards. To address these issues, curriculum learning has been used to divide the target task into multiple subtasks for training. However, manual or random curriculum design often generates unnecessary subtasks that do not improve performance. Furthermore, a standardized curriculum design method for collision avoidance has not yet been presented. Therefore, this paper introduces a curriculum-based collision avoidance learning method that utilizes feedback during the training phase. The proposed method differs from traditional curriculum learning in that the subtask is not predetermined before training. Instead, the curriculum is modified during training based on feedback obtained from validation environments. If a robot demonstrates high collision avoidance performance in a validation environment, it is then validated in more challenging environments for rigorous evaluation. Conversely, if collision avoidance performance is low in the validation environment, the robot is trained in a new environment to overcome frequent collision situations. Simulations and practical experiments were conducted for the proposed method, which showed better performance compared to the non-curriculum method.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Curriculum Learningmentioning

confidence: 99%

Feedback-Based Curriculum Learning for Collision Avoidance

Choi,

Hwang,

Eoh

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Zhang et al propose a deep reinforcement learning algorithm in which each robot use a deep Q-learning controller to transport the oversized object for a single robot [92]. Eoh et al find that a deep reinforcement learning algorithm takes a long time to train a policy, and random action is challenging to train a satisfactory policy [96]. They use curriculum-based learning methods and propose region-growing and single-to multi-robot curricula that raise the success rate of object transportation tasks.…”

Section: ) Reinforcement Learning Based Controlmentioning

confidence: 99%

Multi-Robot Systems and Cooperative Object Transport: Communications, Platforms, and Challenges

Lin

et al. 2023

IEEE Open J. Comput. Soc.

View full text Add to dashboard Cite

Multi-robot systems gain considerable attention due to lower cost, better robustness, and higher scalability as compared with single-robot systems. Cooperative object transport, as a well-known use case of multi-robot systems, shows great potential in real-world applications. The design and implementation of a multi-robot system involve many technologies, specifically, communication, coordination, task allocation methods, experimental platforms, and simulators. However, most of recent multi-robot system studies focus on coordination and task allocation problems, with little focus on communications among multiple robots. In this review, we focus on the communication, validation platform, and simulator of multi-robot systems, and discuss one of the important applications, cooperative object transport. First, we study the multi-robot system fundamentals and comprehensively review the multi-robot system communication technologies. Then, the multi-robot system validating platform, testbed, simulator, and middleware used in academia and industry are investigated. Finally, we discuss recent advances in cooperative object transport, and challenges and possible future research directions for multi-robot systems.

show abstract

“…Each decentralized Q-net is trained with the help of a centralized Q-net. Eoh and Park [ 29 ] proposed a curriculum-learning-based object transportation method using difficulty map generation and an adaptive determination of the episode size. Shibata et al [ 30 ] presented a DRL-based multi-robot transportation method using an event-triggered communication and consensus-based control.…”

Section: Related Workmentioning

confidence: 99%

Deep-Reinforcement-Learning-Based Object Transportation Using Task Space Decomposition

Eoh

2023

Sensors

Self Cite

View full text Add to dashboard Cite

This paper presents a novel object transportation method using deep reinforcement learning (DRL) and the task space decomposition (TSD) method. Most previous studies on DRL-based object transportation worked well only in the specific environment where a robot learned how to transport an object. Another drawback was that DRL only converged in relatively small environments. This is because the existing DRL-based object transportation methods are highly dependent on learning conditions and training environments; they cannot be applied to large and complicated environments. Therefore, we propose a new DRL-based object transportation that decomposes a difficult task space to be transported into simple multiple sub-task spaces using the TSD method. First, a robot sufficiently learned how to transport an object in a standard learning environment (SLE) that has small and symmetric structures. Then, a whole-task space was decomposed into several sub-task spaces by considering the size of the SLE, and we created sub-goals for each sub-task space. Finally, the robot transported an object by sequentially occupying the sub-goals. The proposed method can be extended to a large and complicated new environment as well as the training environment without additional learning or re-learning. Simulations in different environments are presented to verify the proposed method, such as a long corridor, polygons, and a maze.

show abstract

Automatic Curriculum Design for Object Transportation Based on Deep Reinforcement Learning

Cited by 5 publications

References 54 publications

Feedback-Based Curriculum Learning for Collision Avoidance

Feedback-Based Curriculum Learning for Collision Avoidance

Multi-Robot Systems and Cooperative Object Transport: Communications, Platforms, and Challenges

Deep-Reinforcement-Learning-Based Object Transportation Using Task Space Decomposition

Contact Info

Product

Resources

About