On Controller Tuning with Time-Varying Bayesian Optimization

Abstract

Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. When the underlying cause and time of change is unknown, we need to rely on online data for this adaptation. In this paper, we will use time-varying Bayesian optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. Two properties are characteristic of many online controller tuning problems: First, they exhibit incremental and lasting changes in the objective due to changes to the system dynamics, e.g., through wear and tear. Second, the optimization problem is convex in the tuning parameters. Current TVBO methods do not explicitly account for these properties, resulting in poor tuning performance and many unstable controllers through over-exploration of the parameter space. We propose a novel TVBO forgetting strategy using Uncertainty-Injection (UI), which incorporates the assumption of incremental and lasting changes. The control objective is modeled as a spatio-temporal Gaussian process (GP) with UI through a Wiener process in the temporal domain. Further, we explicitly model the convexity assumptions in the spatial dimension through GP models with linear inequality constraints. In numerical experiments, we show that our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and fewer unstable parameter configurations.

Safe Risk-Averse Bayesian Optimization for Controller Tuning

“…Our approach considers safety of the underlying system and combines aspects of safe BO and risk-averse BO. This ensures that there are no constraint violations during the optimization, and enables the application of the approach to continuous optimization, e.g., [11], [12] concerning controller parameter adaptation for systems operating under changing conditions.…”

“…Especially in high-precision motion systems, even one iteration with excessive vibrations is not allowed during operation. Thus, learning the parameters of cascaded controllers with safety and stability constraints is needed to ensure that only safe parameters are explored [11], [12]. The SafeOpt algorithm [18] achieves safety, but is inefficient due to its exploration strategy [9].…”

Improving the Performance of Robust Control through Event-Triggered Learning

“…Herein, we discuss how to automate controller re-tuning to improve performance while retaining robustness to model uncertainty. More generally, finding time-dependent controllers can also be phrased as a timevarying convex optimization problem [4] or a sequential decision problem [5].…”