Safe controller optimization for quadrotors with Gaussian processes

Abstract

Abstract-One of the most fundamental problems when designing controllers for dynamic systems is the tuning of the controller parameters. Typically, a model of the system is used to obtain an initial controller, but ultimately the controller parameters must be tuned manually on the real system to achieve the best performance. To avoid this manual tuning step, methods from machine learning, such as Bayesian optimization, have been used. However, as these methods evaluate different controller parameters on the real system, safety-critical system failures may happen. In this paper, we overcome this problem by applying, for the first time, a recently developed safe optimization algorithm, SAFEOPT, to the problem of automatic controller parameter tuning. Given an initial, low-performance controller, SAFEOPT automatically optimizes the parameters of a control law while guaranteeing safety. It models the underlying performance measure as a Gaussian process and only explores new controller parameters whose performance lies above a safe performance threshold with high probability. Experimental results on a quadrotor vehicle indicate that the proposed method enables fast, automatic, and safe optimization of controller parameters without human intervention. SUPPLEMENTARY MATERIALA video demonstrating the proposed safe, automatic controller optimization on a quadrotor vehicle can be found at http://tiny.cc/icra16 video. A Python implementation of the algorithm is available in [1].

Interactive Trajectory Adaptation through Force-guided Bayesian Optimization

“…If human intervention is detected, a local adaptation process is triggered by running the forge-guided local search described in Section III-B. This local search is implemented by proposing new local parameters θ i,t+1 at each time step t according to the acquisition function (17). Once the optimal set of local parameters θ * i has been found, the reference trajectory distribution is recomputed using the updated observation and duration probabilities of state i with new means µ O * i and µ D * i .…”

“…In contrast, our approach allows for both spatial and temporal adaptation of the nominal task plan, and provides data-efficient adaptation by confining BayesOpt to carry out local searches at the level of the model states distribution, which significantly reduces the parameter space dimensionality. Moreover, the search space is automatically defined from the learning model, sharing some similarities with safe BayesOpt approaches [17]. The proposed framework is evaluated in Section IV for different instances of a simulated 2D pick-and-place task, where data-efficient trajectory adaptations exploiting force-based guidance are successfully reported.…”

Data-Efficient Autotuning With Bayesian Optimization: An Industrial Control Study

“…BO for controller learning has recently also been suggested in [12], [20], [21], which include successful demonstrations in laboratory experiments. A discrete event controller is optimized for a walking robot in [12], and state-feedback controllers are tuned in [20] for a quadrotor and in [21] for a humanoid robot balancing a pole.…”

“…BO for controller learning has recently also been suggested in [12], [20], [21], which include successful demonstrations in laboratory experiments. A discrete event controller is optimized for a walking robot in [12], and state-feedback controllers are tuned in [20] for a quadrotor and in [21] for a humanoid robot balancing a pole. Herein, we present results of applying BO for a typical control problem in the automotive industry (throttle valve control) and consider two types of control objectives, different from those in [12], [20], [21].…”

“…A discrete event controller is optimized for a walking robot in [12], and state-feedback controllers are tuned in [20] for a quadrotor and in [21] for a humanoid robot balancing a pole. Herein, we present results of applying BO for a typical control problem in the automotive industry (throttle valve control) and consider two types of control objectives, different from those in [12], [20], [21]. The proposed controller learning framework, which combines BO with ADRC, is different from the controllers in the mentioned references.…”