Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Luo, Sha; Kasaei, Hamidreza; Schomaker, Lambert

doi:10.1109/ijcnn48605.2020.9207427

Cited by 40 publications

(19 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning varying obstacle distribution and end-effector positions simultaneously is very difficult. Therefore, we design a two-stage learning strategy motivated by curriculum learning [ 27 ]. In Stage I, the manipulator starts from a fixed configuration and learns to avoid obstacles in null space.…”

Section: Methodsmentioning

confidence: 99%

Reinforcement Learning-Based Reactive Obstacle Avoidance Method for Redundant Manipulators

Shen

Jia

Huang

et al. 2022

Entropy

View full text Add to dashboard Cite

Redundant manipulators are widely used in fields such as human-robot collaboration due to their good flexibility. To ensure efficiency and safety, the manipulator is required to avoid obstacles while tracking a desired trajectory in many tasks. Conventional methods for obstacle avoidance of redundant manipulators may encounter joint singularity or exceed joint position limits while tracking the desired trajectory. By integrating deep reinforcement learning into the gradient projection method, a reactive obstacle avoidance method for redundant manipulators is proposed. We establish a general DRL framework for obstacle avoidance, and then a reinforcement learning agent is applied to learn motion in the null space of the redundant manipulator Jacobian matrix. The reward function of reinforcement learning is redesigned to handle multiple constraints automatically. Specifically, the manipulability index is introduced into the reward function, and thus the manipulator can maintain high manipulability to avoid joint singularity while executing tasks. To show the effectiveness of the proposed method, the simulation of 4 degrees of planar manipulator freedom is given. Compared with the gradient projection method, the proposed method outperforms in a success rate of obstacles avoidance, average manipulability, and time efficiency.

show abstract

Section: Methodsmentioning

confidence: 99%

Reinforcement Learning-Based Reactive Obstacle Avoidance Method for Redundant Manipulators

Shen

Jia

Huang

et al. 2022

Entropy

View full text Add to dashboard Cite

show abstract

“…In particular, they presented a recursive algorithm that receives environment information as point-clouds, as well as a robot's initial and desired goal configurations as input arguments and generates connectable path as an output. Some other researches also demonstrated segmenting motion tasks could reduce the complexity that the lower modules have to deal with [120]. Whether it is by distinguishing between individual tasks based on relations of touch [5] or embedding scores or attractors directly into a robotic joint feature space [74].…”

Section: Avoiding Collisions With the Environmentmentioning

confidence: 99%

The State of Lifelong Learning in Service Robots:

Kasaei

Melsen

Beers

et al. 2021

J Intell Robot Syst

View full text Add to dashboard Cite

Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming. In this paper, we review a set of previously published works and discuss advances in service robots from object perception to complex object manipulation and shed light on the current challenges and bottlenecks.

show abstract

“…It has also been shown that the development of a simple curriculum based on the accuracy requirements of a given task, on basis of a measure of a degree of competence like [20], leads to faster and more efficient training [21]. An extension to the work with Universal Value Function Approximators (UVFA) [22] is studied in [23]. While the current state of research mostly focusses on simulated objectives, notable efforts are being put into introducing simulation-trained intelligent and highly adaptive agents to the real-world [24][25][26].…”

Section: Curriculum Learningmentioning

confidence: 99%

Curriculum-Based Deep Reinforcement Learning for Adaptive Robotics: A Mini-Review

Gupta¹,

Najjaran²

2021

Int J Robot Eng

View full text Add to dashboard Cite

To facilitate the current and future automation needs, the research community constantly seeks to develop dynamic and efficient autonomous decision-making agents. These agents must not only be robust to modeling uncertainties, internal and external changes, but can adapt to a range of tasks also. Recent progress in deep reinforcement learning has corroborated to its potential to train such autonomous and robust agents. At the same time, the introduction of curriculum learning has made the reinforcement learning process significantly more efficient and allowed for training on much broader tasks. This combination, Curriculum-based Deep Reinforcement Learning (CDRL), presents a powerful solution to meet the increasing complexity of today's automation industry that demands highly intelligent machines. With this work we present a concise review of CDRL methods within the context of their application to the field of adaptive robotics.

show abstract

Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning

Cited by 40 publications

References 15 publications

Reinforcement Learning-Based Reactive Obstacle Avoidance Method for Redundant Manipulators

Reinforcement Learning-Based Reactive Obstacle Avoidance Method for Redundant Manipulators

The State of Lifelong Learning in Service Robots:

Curriculum-Based Deep Reinforcement Learning for Adaptive Robotics: A Mini-Review

Contact Info

Product

Resources

About