Data Driven Linear Quadratic Gaussian Control Design

Putri, Adi Novitarini; Machbub, Carmadi; Mahayana, Dimitri; Hidayat, Egi

doi:10.1109/access.2023.3254879

Cited by 4 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The implementation of the DRQN based on LSTM also proves that the resulting output signal holds the fastest convergence time (see the magenta graph). In this case study, comparisons can be made with previous research [22]. In this research, the design of the control method is based on the information model of the system.…”

Section: ) 2nd Case Study : Batch Distillation Systemmentioning

confidence: 99%

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-Networks

Novitarini

2023

Preprint

View full text Add to dashboard Cite

<p>The current application of control theory is commonly carried out in systems with a model or known system dynamics. However, in practice this is a formidable task to achieve as not all state information can be known. The use of the Output Feedback (OPFB) scheme in the field of control systems also possesses a weakness because it requires the use of an observer. This appears rather contradictory as the use of an observer requires system dynamics information. This research proposes an optimal control scheme using Deep Recurrent Q-Networks (DRQN) to generate an optimal control signal trajectory based on a collection of input and output data from the system itself. The approach proposed in this study is based on the Q-Learning method from the Reinforcement Learning (RL) scheme. The Long-Short Term Memory (LSTM) is used to approximate the Q-function and determine the control signals for a system without a known model. The method that we proposed in this study has been tested on four case studies. The control signal trajectory generated from our proposed algorithm, is much smaller than the control signal that generated from classical Q-Learning scheme. The results of this research are certainly relevant to the aim of OPFB, namely that the controller is designed to be able to regulate (bring the state trajectory to zero) and minimize control signal energy.</p> <p>It is empirically discovered that the same result is proven by the norm values resulting from the Q-function trajectory. The norm of Q-function trajectory for our proposed algorithm on the 1st, 2nd, 3rd, and 4th case studies are 2.11E-08, 3.15E-06, 3.79E-09, and 1.59E-13, respectively.</p>

show abstract

Section: ) 2nd Case Study : Batch Distillation Systemmentioning

confidence: 99%

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-Networks

Novitarini

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The proposed method for the first and second case studies have been published on [22]. The third case study is inspired by [17].…”

Section: Simulation Studymentioning

confidence: 99%

“…This paper is a further research from [22]. In [22], we proposed the combination of model-based RL and Kalman-Net to adapt the conventional Linear Quadratic Gaussian (LQG) scheme.…”

Section: Introductionmentioning

confidence: 99%

“…This paper is a further research from [22]. In [22], we proposed the combination of model-based RL and Kalman-Net to adapt the conventional Linear Quadratic Gaussian (LQG) scheme. The proposed algorithm in [22] is formulated to solve the regulator problem for stochastic and DT linear systems.…”

Section: Introductionmentioning

confidence: 99%

“…• Implementing the LSTM network to adapt the OPFB based Q-Learning algorithm to carry out the policy evaluation and policy improvement stage to obtain the control signal trajectories. • The advantage of the control scheme proposed in this study, when compared to [22], is that it no longer requires information regarding the dynamical model of the plant (A, B, C, D) in designing the optimal control. Additionally, it is no longer necessary to assign an observer as a state estimation method.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System Using Deep Recurrent Q-Networks

Putri,

Hidayat,

Mahayana

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

The current application of control theory is commonly carried out in systems with a model or known system dynamics. However, in practice this is a formidable task to achieve as not all state information can be known. The use of the Output Feedback (OPFB) scheme in the field of control systems also possesses a weakness because it requires the use of an observer. This appears rather contradictory as the use of an observer requires system dynamics information. This research proposes an optimal control scheme using Deep Recurrent Q-Networks (DRQN) to generate an optimal control signal trajectory based on a collection of input and output data from the system itself. The approach proposed in this study is based on the Q-Learning method from the Reinforcement Learning (RL) scheme. The Long-Short Term Memory (LSTM) is used to approximate the Q-function and determine the control signals for a system without a known model. The method that we proposed in this study has been tested on four case studies. The control signal trajectory generated from our proposed algorithm, is much smaller than the control signal that generated from classical Q-Learning scheme. The results of this research are certainly relevant to the aim of OPFB, namely that the controller is designed to be able to regulate (bring the state trajectory to zero) and minimize control signal energy. It is empirically discovered that the same result is proven by the norm values resulting from the Q-function trajectory. The norm of Q-function trajectory for our proposed algorithm on the 1st, 2nd, 3rd, and 4th case studies are 2.11E-08, 3.15E-06, 3.79E-09, and 1.59E-13, respectively.

show abstract

Design and Experimental Validation of a Model-Free Controller for Beam Stabilization in Adaptive Optics Systems

Guo,

Cheng,

Yang

et al. 2024

IEEE Photonics J.

View full text Add to dashboard Cite

Data Driven Linear Quadratic Gaussian Control Design

Cited by 4 publications

References 24 publications

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-Networks

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System using Deep Recurrent Q-Networks

Output Feedback Control for Deterministic Unknown Dynamics Discrete-Time System Using Deep Recurrent Q-Networks

Design and Experimental Validation of a Model-Free Controller for Beam Stabilization in Adaptive Optics Systems

Contact Info

Product

Resources

About