Curriculum Learning for Motor Skills

Abstract

Abstract. Humans and animals acquire their wide repertoire of motor skills through an incremental learning process, during which progressively more complex skills are acquired and subsequently integrated with prior abilities. The order in which the skills are learned and the progressive manner in which they are developed play an important role in developing a final skill set. Inspired by this general idea, we develop an approach for learning motor skills based on a two-level curriculum. At the high level, the curriculum specifies an order in which different skills should be learned. At the low level, the curriculum defines a process for learning within a skill. The method is used to develop an ensemble of highly dynamic integrated motor skills for a planar articulated figure capable of doing parameterized hops, flips, rolls, and acrobatic sequences. Importantly, we demonstrate that the same curriculum can be successfully applied to significant variations of the articulated figure to yield appropriately individualized motor skill sets.

Shaping in Practice: Training Wheels to Learn Fast Hopping Directly in Hardware

“…In practice, it is helpful to regularly estimate the local gradient of the next landscape before each transition. Ideally, we would want to design a sequence of training wheels that funnel into each other, similar in concept to [16] [20]. In this work, when to switch between environments was chosen heuristically.…”

Applying Deep Reinforcement Learning to Cable Driven Parallel Robots for Balancing Unstable Loads: A Ball Case Study

“…Alternatively, PG methods can operate in continuous or discrete action spaces ( DeepMind Technologies, 2014 ) and are becoming the preferred choice for reinforcement learning tasks ( Karpathy, 2016 ). Karpathy suggested that the reason PG methods are becoming favoured is because it is an end-to-end method: there’s an explicit policy and a principled approach that directly optimizes the expected reward ( Karpathy, 2016 ). Instead of estimating the future reward for every state-action pair based upon the data points collected, we estimate the future reward of the policy based on the policy parameters.…”

Curriculum-Based Deep Reinforcement Learning for Adaptive Robotics: A Mini-Review

“…In the last couple of decades, the field of Deep Reinforcement Learning (DRL) has established itself design process can be either domain-expert specified [8] or automatic [9]. Domain-expert specified curricula rely heavily on human knowledge and thus lack new knowledge discovery, scalability, and robustness to unseen scenarios and may exhibit bias towards certain tasks.…”