Imitation Learning for Locomotion and Manipulation

Abstract

Decision making in robotics often involves computing an optimal action for a given state, where the space of actions under consideration can potentially be large and state dependent. Many of these decision making problems can be naturally formalized in the multiclass classification framework, where actions are regarded as labels for states. One powerful approach to multiclass classification relies on learning a function that scores each action; action selection is done by returning the action with maximum score. In this work, we focus on two imitation learning problems in particular that arise in robotics. The first problem is footstep prediction for quadruped locomotion, in which the system predicts next footstep locations greedily given the current four-foot configuration of the robot over a terrain height map. The second problem is grasp prediction, in which the system must predict good grasps of complex free-form objects given an approach direction for a robotic hand. We present experimental results of applying a recently developed functional gradient technique for optimizing a structured margin formulation of the corresponding large non-linear multiclass classification problems.

Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges

“…The use of imitation learning for solving problems of manipulation like picking, dropping, etc. [139,140] where we can exploit the benefits of soft robotics over hard ones have become essential. Controls tasks in such situations generally have tough to compute cost functions due to the high dimension of action space caused by the flexibility of the soft structure of the robot introduced in the motion of the actuator/end-effector.…”

Deep Reinforcement Learning for Soft Robotic Applications: Brief Overview with Impending Challenges

“…The use of imitation learning for solving problems of manipulation like picking, dropping, etc. [123,124] where we can exploit several benefits of soft robotics over hard ones have become extremely essential. Controls tasks in such situations generally have tough to compute cost functions due to high dimension of action space caused by the flexibility introduced in the motion of the actuator/end-effector because of the soft structure of the robot leading to increased difficulty of applying DRL techniques.…”

Contextual classification with functional Max-Margin Markov Networks

“…We discuss our contribution of learning M 3 Ns using functional gradient boosting as described by Ratliff et al [18], now in the context of multi-label structured prediction.…”

“…A variant of this algorithm is to perform exponentiated functional gradient descent, as described in [18]. When evaluating the functional, these potentials are of the form ψ Ci (x c , y c ) = exp( t α t h t (f Ci (x c , y c ))).…”

“…Functional gradient learning has recently been used to learn binary score functions to estimate the quality of in- puts for footstep and grasp planning [18] and path planning [22]. In this work, we adapt these functional gradient techniques to learn the potential functions of Markov Random Fields for discrete multi-label contextual classification.…”