Online Optimization and Learning in Uncertain Dynamical Environments with Performance Guarantees

Abstract

We propose a new framework to solve online optimization and learning problems (P) in unknown and uncertain dynamical environments. This framework enables us to simultaneously learn the uncertain dynamical environment while making online decisions in a quantifiably robust manner. The main technical approach relies on the theory of distributional robust optimization that leverages adaptive probabilistic ambiguity sets. However, as defined, the ambiguity set usually leads to online intractable problems, and the first part of our work is directed to find reformulations in the form of online convex problems for two subclasses of objective functions. To solve the resulting problems in the proposed framework, we further introduce an online version of the Nesterov's accelerated-gradient algorithm. We determine how the proposed solution system achieves a probabilistic regret bound under certain conditions. Two applications illustrate the applicability of the proposed framework.

Online Stochastic Optimization for Unknown Linear Systems: Data-Driven Synthesis and Controller Analysis

“…Several works applied online convex optimization to control plants modeled as algebraic maps [25]- [27] (corresponding to cases where the dynamics are infinitely fast). When the dynamics are non-negligible, LTI systems are considered in [4], [5], [7], [10], stable nonlinear systems in [6], [28], switching systems in [12], and distributed multi-agent systems in [3], [29]. All these works consider continuoustime dynamics and deterministic optimization problems, and derive results in terms of asymptotic or exponential stability.…”

Data-Driven Synthesis of Optimization-Based Controllers for Regulation of Unknown Linear Systems

“…Online optimization approaches aim to optimize loss functions in connection with an underlying and uncertain dynamical system. Linear time-invariant systems are considered in e.g., [2]- [4], [9], stable nonlinear systems in [8], [23], and switched systems in [10]. In contrast with the above line of work, which considers continuous-time dynamics, the focus of this paper is on systems and controllers that operate at discrete time.…”