Acquiring robot skills via reinforcement learning

Abstract

S performance. In this work we focus on two skills, namely robotic assembly and balancing and on two classic tasks to develop these skills via Ieaming: the peg-in-hole insertion task, and the ball balancing task. A stochastic real-valued (SRV) reinforcement leaming algorithm is described and used for leam-is with the Conipiiter Sc,ienc,e Depurtnietit, Utii\wsity of Massachusetts. Amhcr..>t. MA 01003. Enruil: ,.ijuyku-mar@cs.umass.edu. J.A. Fixnkliii und H . Benhrulrinr are Mith GTE Labor-utories Incorporated, 40 S y l~m Roud. Wulthuni. MA 02254. Email: ,$runkliri@gte.cwn and hhc~rihruhini~,~tc..c.om. The w w k of V: Gullapalli was siippor.ted by firriding to A. Barto by the AFOSR.Skilled behavior involves the effective use of knowledge in execution or performance. A skill may require dexterity or coordination, and generally develops over time through leaming. This work focuses on employing leaming to enable a robot to acquire skills, particularly physical skills where leaming control is required. The requirements are i) dealing with nonlinearity/complex dynamics, ii) achieving robust performance under

A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives

“…Peg-in-a-hole: Inserting a peg in a hole is one of the most basic manipulation skills that we discuss in this article. It is the most frequent assembly operation (Gullapalli et al 1994). The behavior can benefit from both visual (Levine et al 2016) and force sensors (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016), but it can also be done without any sensors (Chebotar et al 2017a).…”

“…It is the most frequent assembly operation (Gullapalli et al 1994). The behavior can benefit from both visual (Levine et al 2016) and force sensors (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016), but it can also be done without any sensors (Chebotar et al 2017a). While the most obvious application of this skill is found in assembly tasks (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016;Levine et al 2016), it can also be used to, for example, plug in a power plug (Chebotar et al 2017a).…”

“…The behavior can benefit from both visual (Levine et al 2016) and force sensors (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016), but it can also be done without any sensors (Chebotar et al 2017a). While the most obvious application of this skill is found in assembly tasks (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016;Levine et al 2016), it can also be used to, for example, plug in a power plug (Chebotar et al 2017a). The problem can be solved end-to-end from visual data to motor torques (Levine et al 2016) or from force measurements to Cartesian positions (Gullapalli et al 1994) as a purely reactive behavior.…”

“…While the most obvious application of this skill is found in assembly tasks (Gullapalli et al 1994;Ellekilde et al 2012;Kramberger et al 2016;Levine et al 2016), it can also be used to, for example, plug in a power plug (Chebotar et al 2017a). The problem can be solved end-to-end from visual data to motor torques (Levine et al 2016) or from force measurements to Cartesian positions (Gullapalli et al 1994) as a purely reactive behavior. Alternatively, learning can be combined with search heuristics for the hole based on force measurements (Ellekilde et al 2012;Kramberger et al 2016).…”

“…Kober et al (2008) learn a coupling of perception and action to handle perturbations of trajectories. Gullapalli et al (1994) learn peg-in-a-hole insertion. They have sensor noise in position encoders and in a wrist force sensor and demonstrate that reinforcement learning can be used to generate robust insertion behavior.…”