Learning-Based vs Model-Free Adaptive Control of a MAV Under Wind Gust

Chaffre, Thomas; Moras, Julien; Chan-Hon-Tong, Adrien; Marzat, Julien; Sammut, Karl; Chenadec, Gilles Le; Clément, Benoı̂t

doi:10.1007/978-3-030-92442-3_19

Cited by 5 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to our previous work [30], the control objective treated here is fundamentally more complex since we are not trying to reach one singular state but rather a sequence of desired successive states. We propose the following terminal reward signal to take this into account:…”

Section: B Reward Shapingmentioning

confidence: 99%

See 1 more Smart Citation

Learning Stochastic Adaptive Control using a Bio-Inspired Experience Replay

CHAFFRE¹,

SANTOS²,

CHENADEC³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

<p>Deep Reinforcement Learning (DRL) methods are dominating the field of adaptive control where they are used to adapt the controller response to disturbances. Nevertheless, the usage of these methods on physical platforms is still limited due to their data inefficiency and the performance drop when facing unseen process variations. This is particularly perceived in the Autonomous Underwater Vehicles (AUVs) context as studied here, where the process observability is limited. To be effective, DRL-based AUV control systems require the use of methods that are data-efficient (in order to reach a satisfactory behavior with a sufficiently fast response time) and are resilient (to ensure robustness to severe changes in operating conditions). With this ambition, we study in this paper the effect of the Experience Replay (ER) mechanism on the performance variation of a DRL-based stochastic adaptive controller. We propose a new ER method (denoted as BIER) that takes inspiration from the biological Replay Mechanism and compare it to the standard method denoted as CER. We apply it to the Soft Actor-Critic, a maximum entropy DRL algorithm, for use with an AUV maneuvering task that consists in stabilizing the vehicle at a given velocity and pose. The training results show that our BIER method exceeds the performance of the nonadaptive optimal model-based counterpart of the controller in less than half the number of episodes compared to CER. We proposed different evaluation scenarios of increasing complexity as measured by desired velocity value and amplitude of current disturbance. Our results suggest that the BIER method achieves improved learning stability and better generalization abilities.</p>

show abstract

Section: B Reward Shapingmentioning

confidence: 99%

“…where a t−1 ∈ R (30), the current disturbance characteristics are not included. In order to improve the process observability and following our previous results [32], we construct our state vector s t out of the current and past observation vectors along with their two-by-two difference.…”

Section: Process Observabilitymentioning

confidence: 99%

Learning Stochastic Adaptive Control using a Bio-Inspired Experience Replay

CHAFFRE¹,

SANTOS²,

CHENADEC³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another model-free approach is data-driven control which requires prior offline learning with the use of previously collected input–output data and is based on the use of regression techniques 8 . However, despite their advantages, model-free control algorithms may have difficulties with quick convergence, as demonstrated in 3 .…”

Section: Introductionmentioning

confidence: 99%

Linear matrix genetic programming as a tool for data-driven black-box control-oriented modeling in conditions of limited access to training data

Praczyk,

Szymkowiak

2024

Sci Rep

View full text Add to dashboard Cite

In the paper, a new evolutionary technique called Linear Matrix Genetic Programming (LMGP) is proposed. It is a matrix extension of Linear Genetic Programming and its application is data-driven black-box control-oriented modeling in conditions of limited access to training data. In LMGP, the model is in the form of an evolutionarily-shaped program which is a sequence of matrix operations. Since the program has a hidden state, running it for a sequence of input data has a similar effect to using well-known recurrent neural networks such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU). To verify the effectiveness of the LMGP, it was compared with different types of neural networks. The task of all the compared techniques was to reproduce the behavior of a nonlinear model of an underwater vehicle. The results of the comparative tests are reported in the paper and they show that the LMGP can quickly find an effective and very simple solution to the given problem. Moreover, a detailed comparison of models, generated by LMGP and LSTM/GRU, revealed that the former are up to four times more accurate than the latter in reproducing vehicle behavior.

show abstract