Trajectory-control using deep System Identification and Model Predictive Control for Drone Control under Uncertain Load

Abstract

Machine learning allows to create complex models if provided with enough data, hence challenging more traditional system identification methods. We compare the quality of neural networks and an ARX model when use in an model predictive control to command a drone in a simulated environment. The training of neural networks can be challenging when the data is scarce or datasets are unbalanced. We propose an adaptation of prioritized replay to system identification in order to mitigate these problems. We illustrate the advantages and limits of this training method on the control task of a simulated drone.Index Terms-identification, model predictive control, neural networks, learning

Importance Sampling for Deep System Identification

“…Indeed, prioritization forces the training on harder samples even if they are scarce. The adaptation of this sampling strategy to system identification yielded encouraging results illustrated in [7]. In practice, we use the loss of the network prediction to estimate the training value of a sample.…”

“…As far as we are aware, there is only one prior attempt to perform importance sampling in neural-network-based model identification [7]. This work relies on the Prioritized Experience Replay [3] which demonstrated great results in Reinforcement Learning (RL).…”

“…Those approaches are evaluated on standard datasets [15] and custom datasets that exhibit various degrees of non-linearity, unbalancing and noise. Building on the promising simulated result of [7] we expand the prioritize experience replay method to real world data and show its applicability on standard datasets. Moreover, we propose to use an other prioritization scheme that alleviate the hyperparameter complexity of the previous algorithm.…”

“…In this paper, prioritization using the upper-bound gradient form [6] is applied to system identification. The result from [7] are extended to standard and real world datasets. We demonstrate the method on different conditions and systems and compare the different methods : ARMA, standard NN training, prioritized experienced replay NN training, and gradient upper-bound NN training.…”

Evaluation of Prioritized Deep System Identification on a Path Following Task

“…To address these problems, prior works performed importance sampling in NN-based model identification [21]. This work relies on the PER [28] algorithm, which demonstrated great results in RL.…”