Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions

Abstract

We study the problem of learning-augmented predictive linear quadratic control. Our goal is to design a controller that balances consistency, which measures the competitive ratio when predictions are accurate, and robustness, which bounds the competitive ratio when predictions are inaccurate. We propose a novel 𝜆-confident controller and prove that it maintains a competitive ratio upper bound of 1 + min{𝑂 (𝜆 2 𝜀) + 𝑂 (1 − 𝜆) 2 , 𝑂 (1) + 𝑂 (𝜆 2 )} where 𝜆 ∈ [0, 1] is a trust parameter set based on the confidence in the predictions, and 𝜀 is the prediction error. Further, we design a self-tuning policy that adaptively learns the trust parameter 𝜆 with a competitive ratio that depends on 𝜀 and the variation of system perturbations and predictions.CCS Concepts: • Theory of computation → Online learning algorithms; Regret bounds.

Certifying Black-Box Policies With Stability for Nonlinear Control

“…Before proceeding to our policy, to highlight the challenge of combining modelbased advice with model-free policies in this setting we first consider a simple strategy for combining the two via a convex combination. This is an approach that has been proposed and studied previously, e.g., [7], [21]. However, we show that it can be problematic in that it can yield an unstable policy even when the two policies are stabilizing individually.…”

“…Our results imply an interesting trade-off between stability and optimality, in the sense that if λ is smaller, it is guaranteed to stabilize with a higher rate and if λ becomes larger, it is able to have a smaller competitive ratio bound when provided with a high-quality black-box policy. Different from the linear case, where a cost characterization lemma can be directly applied to bound the difference between the policy costs and optimal costs in terms of the difference between their actions [21], for the case of nonlinear dynamics (1), we introduce an auxiliary linear problem to derive an upper bound on the dynamic regret, whose value can be decomposed into a quadratic term and a term induced by the nonlinearity. The first term can be bounded via a generalized characterization lemma and becomes the model-based bound and model-free error in (2).…”

“…Learning-augmented online problems. Recently, the idea of augmenting competitive/robust online algorithms with machine-learned advice has attracted attention in online problems in settings like online caching [26], ski-rental [27], [28], smoothed online convex optimization [29], [30] and linear quadratic control [21]. In many of these learning-augmented online algorithms, a convex combination of machine-learned (untrusted) predictions and robust decisions is involved.…”

“…In many of these learning-augmented online algorithms, a convex combination of machine-learned (untrusted) predictions and robust decisions is involved. For instance, in [21], competitive ratio upper bounds of a λ-confident policy are given for a linear quadratic control problem. The policy λπ MPC +(1−λ)π LQR combines linearly a linear quadratic regulator π LQR and an MPC policy π MPC with machine-learned predictions where λ ∈ [0, 1] measures the confidence of the machine-learned predictions.…”

Optimal Competitive-Ratio Control

“…Bounds on the competitive ratio has been the focus of many prior works in control [28][29][30][31]. The competitive-ratio control problem has been proposed in [5], where its sub-optimal solution was derived under certain conditions.…”