Mastering of a Task with Interaction between a Robot and Its Environment. "Kendama" Task.

Sato, Shuichi; Sakaguchi, Toshifumi; Masutani, Yasuhiro; Miyazaki, Fumio

doi:10.1299/kikaic.59.487

Cited by 10 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While we learn directly in the joint space of the robot, Takenaka et al [22] recorded planar human cup movements and determined the required joint movements for a planar, three degree of freedom (DoF) robot so that it could follow the trajectories while visual feedback was used for error compensation. Both Sato et al [23] and Shone [24] used motion planning approaches which relied on very accurate models of the ball while employing only one DoF in [24] or two DoF in [23] so that the complete state-space could be searched exhaustively. Interestingly, exploratory robot moves were used in [23] to estimate the parameters of the employed model.…”

Section: A Discrete Movement: Ball-in-a-cupmentioning

confidence: 99%

Learning motor primitives for robotics

Kober

Peters

2009

2009 IEEE International Conference on Robotics and Automation

187

135

View full text Add to dashboard Cite

Abstract-The acquisition and self-improvement of novel motor skills is among the most important problems in robotics. Motor primitives offer one of the most promising frameworks for the application of machine learning techniques in this context. Employing an improved form of the dynamic systems motor primitives originally introduced by Ijspeert et al.[2], we show how both discrete and rhythmic tasks can be learned using a concerted approach of both imitation and reinforcement learning. For doing so, we present both learning algorithms and representations targeted for the practical application in robotics. Furthermore, we show that it is possible to include a start-up phase in rhythmic primitives. We show that two new motor skills, i.e., Ball-in-a-Cup and Ball-Paddling, can be learned on a real Barrett WAM robot arm at a pace similar to human learning while achieving a significantly more reliable final performance.

show abstract

Section: A Discrete Movement: Ball-in-a-cupmentioning

confidence: 99%

Learning motor primitives for robotics

Kober

Peters

2009

2009 IEEE International Conference on Robotics and Automation

187

135

View full text Add to dashboard Cite

show abstract

“…While we learn directly in the joint space of the robot, Takenaka (1984) recorded planar human cup movements and determined the required joint movements for a planar, three degree of freedom (DoF) robot, so that it could follow the trajectories while visual feedback was used for error compensation. Both Sato et al (1993) and Shone et al (2000) used motion planning approaches which relied on very accurate models of the ball and the string while employing only one DoF in (Shone (Sato et al 1993) so that the complete state-space could be searched exhaustively. Interestingly, exploratory robot moves were used in (Sato et al 1993) to estimate the parameters of the employed model.…”

Section: Benchmark Comparison V: Castingmentioning

confidence: 99%

“…Both Sato et al (1993) and Shone et al (2000) used motion planning approaches which relied on very accurate models of the ball and the string while employing only one DoF in (Shone (Sato et al 1993) so that the complete state-space could be searched exhaustively. Interestingly, exploratory robot moves were used in (Sato et al 1993) to estimate the parameters of the employed model. Probably the most advanced preceding work on learning Kendama was carried out by Miyamoto et al (1996) who used a seven DoF anthropomorphic arm and recorded human motions to train a neural network to reconstruct via-points.…”

Section: Benchmark Comparison V: Castingmentioning

confidence: 99%

Policy search for motor primitives in robotics

2010

View full text Add to dashboard Cite

Many motor skills in humanoid robotics can be learned using parametrized motor primitives. While successful applications to date have been achieved with imitation learning, most of the interesting motor learning problems are high-dimensional reinforcement learning problems. These problems are often beyond the reach of current reinforcement learning methods. In this paper, we study parametrized policy search methods and apply these to benchmark problems of motor primitive learning in robotics. We show that many wellknown parametrized policy search methods can be derived from a general, common framework. This framework yields both policy gradient methods and expectation-maximization (EM) inspired algorithms. We introduce a novel EM-inspired algorithm for policy learning that is particularly well-suited for dynamical system motor primitives. We compare this algorithm, both in simulation and on a real robot, to several well-known parametrized policy search methods such as episodic REINFORCE, 'Vanilla' Policy Gradients with optimal baselines, episodic Natural Actor Critic, and episodic Reward-Weighted Regression. We show that the proposed method out-performs them on an empirical benchmark of learning dynamical system motor primitives both in simulation and on a real robot. We apply it in the context of motor learning and show that it can learn a complex Ball-in-a-Cup task on a real Barrett WAM™ robot arm.

show abstract

“…While we learn directly in the joint space of the robot, Takenaka [1984] recorded planar human cup movements and determined the required joint movements for a planar, three degree of freedom (DoF) robot, so that it could follow the trajectories while visual feedback was used for error compensation. Both Sato et al [1993] and Shone et al [2000] used motion planning approaches which relied on very accurate models of the ball and the string while employing only one DoF in [Shone et al, 2000] or two DoF in [Sato et al, 1993] so that the complete state-space could be searched exhaustively. Interestingly, exploratory robot moves were used in [Sato et al, 1993] to estimate the parameters of the employed model.…”

Section: Ball-in-a-cup On a Barrett Wammentioning

confidence: 99%

“…Both Sato et al [1993] and Shone et al [2000] used motion planning approaches which relied on very accurate models of the ball and the string while employing only one DoF in [Shone et al, 2000] or two DoF in [Sato et al, 1993] so that the complete state-space could be searched exhaustively. Interestingly, exploratory robot moves were used in [Sato et al, 1993] to estimate the parameters of the employed model. Probably the most advanced preceding work on learning Kendama was carried out by Miyamoto et al [1996] who used a seven DoF anthropomorphic arm and recorded human motions to train a neural network to reconstruct via-points.…”

Section: Ball-in-a-cup On a Barrett Wammentioning

confidence: 99%

Learning motor skills: from algorithms to robot experiments

Kober

Peters

2014

It - Information Technology

View full text Add to dashboard Cite

Die Veröffentlichung steht unter folgender Creative Commons Lizenz: Namensnennung -Keine kommerzielle Nutzung -Keine Bearbeitung 2.0 Deutschland http://creativecommons.org/licenses/by-nc-nd/2.0/de/ Abstract Ever since the word "robot" was introduced to the English language by KarelČapek's play "Rossum's Universal Robots" in 1921, robots have been expected to become part of our daily lives. In recent years, robots such as autonomous vacuum cleaners, lawn mowers, and window cleaners, as well as a huge number of toys have been made commercially available. However, a lot of additional research is required to turn robots into versatile household helpers and companions. One of the many challenges is that robots are still very specialized and cannot easily adapt to changing environments and requirements. Since the 1960s, scientists attempt to provide robots with more autonomy, adaptability, and intelligence. Research in this field is still very active but has shifted focus from reasoning based methods towards statistical machine learning. Both navigation (i.e., moving in unknown or changing environments) and motor control (i.e., coordinating movements to perform skilled actions) are important sub-tasks.In this thesis, we will discuss approaches that allow robots to learn motor skills. We mainly consider tasks that need to take into account the dynamic behavior of the robot and its environment, where a kinematic movement plan is not sufficient. The presented tasks correspond to sports and games but the presented techniques will also be applicable to more mundane household tasks. Motor skills can often be represented by motor primitives. Such motor primitives encode elemental motions which can be generalized, sequenced, and combined to achieve more complex tasks. For example, a forehand and a backhand could be seen as two different motor primitives of playing table tennis. We show how motor primitives can be employed to learn motor skills on three different levels. First, we discuss how a single motor skill, represented by a motor primitive, can be learned using reinforcement learning. Second, we show how such learned motor primitives can be generalized to new situations. Finally, we present first steps towards using motor primitives in a hierarchical setting and how several motor primitives can be combined to achieve more complex tasks.To date, there have been a number of successful applications of learning motor primitives employing imitation learning. However, many interesting motor learning problems are high-dimensional reinforcement learning problems which are often beyond the reach of current reinforcement learning methods. We review research on reinforcement learning applied to robotics and point out key challenges and important strategies to render reinforcement learning tractable. Based on these insights, we introduce novel learning approaches both for single and generalized motor skills.For learning single motor skills, we study parametrized policy search methods and introduce a framework of reward-weighted imi...

show abstract

Mastering of a Task with Interaction between a Robot and Its Environment. "Kendama" Task.

Cited by 10 publications

References 0 publications

Learning motor primitives for robotics

Learning motor primitives for robotics

Policy search for motor primitives in robotics

Learning motor skills: from algorithms to robot experiments

Contact Info

Product

Resources

About