Stochastic Spatio-Temporal Optimization for Control and Co-Design of Systems in Robotics and Applied Physics

Abstract

Correlated with the trend of increasing degrees of freedom in robotic systems is a similar trend of rising interest in Spatio-Temporal systems described by Partial Differential Equations (PDEs) among the robotics and control communities. These systems often exhibit dramatic under-actuation, high dimensionality, bifurcations, and multimodal instabilities. Their control represents many of the current-day challenges facing the robotics and automation communities. Not only are these systems challenging to control, but the design of their actuation is an NP-hard problem on its own. Recent methods either discretize the space before optimization, or apply tools from linear systems theory under restrictive linearity assumptions in order to arrive at a control solution. This manuscript provides a novel sampling-based stochastic optimization framework based entirely in Hilbert spaces suitable for the general class of semi-linear SPDEs which describes many systems in robotics and applied physics. This framework is utilized for simultaneous policy optimization and actuator co-design optimization. The resulting algorithm is based on variational optimization, and performs joint episodic optimization of the feedback control law and the actuation design over episodes. We study first and second order systems, and in doing so, extend several results to the case of second order SPDEs. Finally, we demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs in robotics and applied physics including an infinite degree-of-freedom soft robotic manipulator.

Stochastic optimization for learning quantum state feedback control

“…This is discussed with greater detail in the supplemental material, including some common instances of degeneracy. These degeneracies prove prohibitive for a variety of methods introduced in the stochastic optimal control literature, including path integral control [56][57][58][59], forwardbackward stochastic differential equations using importance sampling [60,61], and recently spatio-temporal stochastic optimization [62,63]. In each case, such degeneracies must be carefully addressed.…”

“…In the context of [63], policies without explicit time dependence have been shown to effectively control a number of SPDE systems for reaching and stabiliziation tasks, however these policies can fail for tracking tasks. Both of these approaches are algorithmically quite similar, and may have theoretic connections if one can connect the objective in GASS to an analogous free-energy relative entropy relationship.…”

“…As stated in the main paper, the cost function is passed through a differentiable and non-increasing shape function S(•) : R → R. Many different shape functions can be used, as explained in greater detail in [69]. In our simulated experiments, we used the function S(J) := exp − κJ), (63) where κ = 1.0 affects the slope, and may be tuned for greater performance.…”