Information theoretic MPC for model-based reinforcement learning

Williams, Grady; Wagener, Nolan; Goldfain, Brian; Drews, Paul; Rehg, James M.; Boots, Byron; Theodorou, Evangelos A.

doi:10.1109/icra.2017.7989202

Cited by 400 publications

(398 citation statements)

References 16 publications

Supporting

Mentioning

395

Contrasting

Unclassified

Order By: Relevance

“…This platform is approximately 1 meter long, weighs over 20 kilograms, and has a top speed over 20 m/s. Previous works have demonstrated that the MPPI controller (with tuned soft cost terms) is capable of navigating this type of vehicle around a simple elliptical track [25,26], which we did our best to match in our simulation experiments. Our real-world experiments use the same type of vehicle as these prior works, but in a more challenging environment (Fig.…”

Section: /5 Scale Autonomous Racing Experimentsmentioning

confidence: 80%

“…There are then 3 components of the Tube-MPC algorithm that we need: (1) a nominal controller, (2) a method for setting the nominal state, and (3) an ancillary controller. We use an information theoretic interpretation of model predictive path integral control (MPPI) [26], so we will hereon refer to our method as Tube-MPPI.…”

Section: Robust Sampling Based Mpcmentioning

confidence: 99%

“…Then, by using an information theoretic lower bound, it is possible to show that there exists an "optimal" distribution over controls, in the sense that trajectories sampled from that distribution have a lower expected cost than any other distribution. It can be shown [26] that this takes the form:…”

Section: A Nominal Controller -Model Predictive Path Integralmentioning

confidence: 99%

See 2 more Smart Citations

Robust Sampling Based Model Predictive Control with Sparse Objective Information

Williams¹,

Goldfain²,

Drews³

et al. 2018

Robotics: Science and Systems XIV

Self Cite

View full text Add to dashboard Cite

Abstract-We present an algorithmic framework for stochastic model predictive control that is able to optimize non-linear systems with cost functions that have sparse, discontinuous gradient information. The proposed framework combines the benefits of sampling-based model predictive control with linearization-based trajectory optimization methods. The resulting algorithm consists of a novel utilization of Tube-based model predictive control. We demonstrate robust algorithmic performance on a variety of simulated tasks, and on a real-world fast autonomous driving task.

show abstract

Section: /5 Scale Autonomous Racing Experimentsmentioning

confidence: 80%

Section: Robust Sampling Based Mpcmentioning

confidence: 99%

See 1 more Smart Citation

Robust Sampling Based Model Predictive Control with Sparse Objective Information

Williams¹,

Goldfain²,

Drews³

et al. 2018

Robotics: Science and Systems XIV

Self Cite

View full text Add to dashboard Cite

show abstract

“…Model Predictive Control (MPC)-based optimal controllers (e.g. Model Predictive Path Integral (MPPI) [19]) provide planned control trajectories given an initial state and a cost function by solving the optimal control problem. An optimal control problem whose objective is to minimize a task-specific cost function J(X, U) can be formulated as follows:…”

Section: A Model Predictive Optimal Controlmentioning

confidence: 99%

“…This can be solved in a receding horizon fashion in an MPC framework and it allows us to have a real-time optimal controller with feedback. In our work, a sampling-based receding-horizon stochastic optimization algorithm, MPPI controller [19] is used as an MPC controller. We chose MPPI for several reasons, first off being the generality of cost functions and dynamics allowed.…”

Section: A Model Predictive Optimal Controlmentioning

confidence: 99%

Aggressive Perception-Aware Navigation Using Deep Optical Flow Dynamics and PixelMPC

Lee

Gibson

Theodorou

2020

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Recently, vision-based control has gained traction by leveraging the power of machine learning. In this work, we couple a model predictive control (MPC) framework to a visual pipeline. We introduce deep optical flow (DOF) dynamics, which is a combination of optical flow and robot dynamics. Using the DOF dynamics, MPC explicitly incorporates the predicted movement of relevant pixels into the planned trajectory of a robot. Our implementation of DOF is memory-efficient, data-efficient, and computationally cheap so that it can be computed in real-time for use in an MPC framework. The suggested Pixel Model Predictive Control (PixelMPC) algorithm controls the robot to accomplish a high-speed racing task while maintaining visibility of the important features (gates). This improves the reliability of vision-based estimators for localization and can eventually lead to safe autonomous flight. The proposed algorithm is tested in a photorealistic simulation with a high-speed drone racing task. Supplementary video: https://youtu.be/NzL2YRcOh I

show abstract

Data‐Driven Intelligent Manipulation of Particles in Microfluidics

2022

View full text Add to dashboard Cite

Automated manipulation of small particles using external (e.g., magnetic, electric and acoustic) fields has been an emerging technique widely used in different areas. The manipulation typically necessitates a reduced-order physical model characterizing the field-driven motion of particles in a complex environment. Such models are available only for highly idealized settings but are absent for a general scenario of particle manipulation typically involving complex nonlinear processes, which has limited its application. In this work, the authors present a data-driven architecture for controlling particles in microfluidics based on hydrodynamic manipulation. The architecture replaces the difficult-to-derive model by a generally trainable artificial neural network to describe the kinematics of particles, and subsequently identifies the optimal operations to manipulate particles. The authors successfully demonstrate a diverse set of particle manipulations in a numerically emulated microfluidic chamber, including targeted assembly of particles and subsequent navigation of the assembled cluster, simultaneous path planning for multiple particles, and steering one particle through obstacles. The approach achieves both spatial and temporal controllability of high precision for these settings. This achievement revolutionizes automated particle manipulation, showing the potential of data-driven approaches and machine learning in improving microfluidic technologies for enhanced flexibility and intelligence.

show abstract

Information theoretic MPC for model-based reinforcement learning

Cited by 400 publications

References 16 publications

Robust Sampling Based Model Predictive Control with Sparse Objective Information

Robust Sampling Based Model Predictive Control with Sparse Objective Information

Aggressive Perception-Aware Navigation Using Deep Optical Flow Dynamics and PixelMPC

Data‐Driven Intelligent Manipulation of Particles in Microfluidics

Contact Info

Product

Resources

About