One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares

Min, Young-Jae; Ahn, Kyu-Joong; Azizan, Navid

doi:10.1109/cdc51059.2022.9992939

Cited by 5 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the agent should be able to learn with bounded memory to meet the computational requirements of embedded systems. While we used a rather naive online learning algorithm, recent advances in the communities of machine learning and control are promising for learning dynamics online (Min et al, 2022). The computational cost of our method is dominated by the calculation of derivatives.…”

Section: Discussionmentioning

confidence: 99%

Online greedy identification of linear dynamical systems

Blanke

Lelarge

2022

2022 IEEE 61st Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Model-based reinforcement learning is a powerful tool, but collecting data to fit an accurate model of the system can be costly. Exploring an unknown environment in a sample-efficient manner is hence of great importance. However, the complexity of dynamics and the computational limitations of real systems make this task challenging. In this work, we introduce FLEX, an exploration algorithm for nonlinear dynamics based on optimal experimental design. Our policy maximizes the information of the next step and results in an adaptive exploration algorithm, compatible with generic parametric learning models and requiring minimal resources. We test our method on a number of nonlinear environments covering different settings, including time-varying dynamics. Keeping in mind that exploration is intended to serve an exploitation objective, we also test our algorithm on downstream model-based classical control tasks and compare it to other state-of-theart model-based and model-free approaches. The performance achieved by FLEX is competitive and its computational cost is low.

show abstract

Section: Discussionmentioning

confidence: 99%