Discrete-Time Linear-Quadratic Regulation via Optimal Transport

Abstract

In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.

Covariance Steering of Discrete-Time Stochastic Linear Systems Based on Wasserstein Distance Terminal Cost

“…All numerical computations were conducted on a Mac M1 with 8GB of RAM. The first numerical example in Section V-A is similar to the one solved in [7], for comparison purposes. The second example in Section V-B is provided to illustrate that our approach can be used to steer general probability distributions.…”

“…The parameters of the system dynamics and the cost function are taken from Example V-A in [7]. In this example, x k ∈ R 2 and u k ∈ R 1 for all k ∈ {0, .…”

“…Lastly, optimal mass transport-based approaches treat the dynamic problem as a static mass transport problem using transition costs between initial and terminal states. This associated problem is solved using well-known optimal mass transport algorithms [1], [7]- [9].…”

“…Literature Review: The problem of density steering has received significant attention in previous research. In [7], the authors propose a similar problem and re-frame it as an optimal mass transport problem. Meanwhile, in [13], the authors tackle the density steering problem using dynamic programming over probability spaces.…”

Entropic Model Predictive Optimal Transport over Dynamical Systems

“…In this approach, a large population limit is considered, and infinitely many agents are represented as a probability density of the state of a single system. Then, the dynamical assignment problem boils down to a density control problem [14][15][16] finding a feedback control law that steers an initial state density to a target density with minimum cost. Consequently, this approach can avoid the difficulty due to the large scale of the collective dynamics.…”