Quadratic approximate dynamic programming for input‐affine systems

Keshavarz, Arezou; Boyd, Stephen

doi:10.1002/rnc.2894

Cited by 26 publications

(20 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another tool used in portfolio optimization applications of this paper is a special class of approximate value function methods [27][28][29][30]. In general, stochastic optimal control problems can be solved by utilizing state value functions, which estimate performance at a given state.…”

Section: Initialize Parameter θ Of the Search Distribution π(·|θ)mentioning

confidence: 99%

“…in state-space format, it is necessary to define the state and control input together with the performance index that is used as an optimization criterion. To do this, we follow the research of Boyd et al [1,28,30]. We define the state vector as the collection of the portfolio positions.…”

Section: Machine Learning and Control Based Portfolio Optimizationmentioning

confidence: 99%

“…In this paper, we consider a solution to the trend following trading problem based on the natural evolution strategy (NES) [21][22][23]25] and a risk-adjusted expected profit maximization problem based on an approximate value function (AVF) method [27][28][29][30].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Modern Probabilistic Machine Learning and Control Methods for Portfolio Optimization

Park¹,

Lim²,

Lee³

et al. 2014

International Journal of Fuzzy Logic and Intelligent Systems

View full text Add to dashboard Cite

Many recent theoretical developments in the field of machine learning and control have rapidly expanded its relevance to a wide variety of applications. In particular, a variety of portfolio optimization problems have recently been considered as a promising application domain for machine learning and control methods. In highly uncertain and stochastic environments, portfolio optimization can be formulated as optimal decision-making problems, and for these types of problems, approaches based on probabilistic machine learning and control methods are particularly pertinent. In this paper, we consider probabilistic machine learning and control based solutions to a couple of portfolio optimization problems. Simulation results show that these solutions work well when applied to real financial market data.

show abstract

Section: Initialize Parameter θ Of the Search Distribution π(·|θ)mentioning

confidence: 99%

Section: Machine Learning and Control Based Portfolio Optimizationmentioning

confidence: 99%

See 1 more Smart Citation

Modern Probabilistic Machine Learning and Control Methods for Portfolio Optimization

Park¹,

Lim²,

Lee³

et al. 2014

International Journal of Fuzzy Logic and Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…For the special case when the dynamics of the system are linear, Dynamic Programming (DP) gives a complete and explicit solution to the problem, because the one-step state cost and the value/cost function in this case are quadratic. 16 For the general nonlinear control problem, DP is difficult to carry out and ADP designs are not systematic.…”

mentioning

confidence: 99%

“…This is called a model-free approach, because it does not need any a priori model information at the beginning of the algorithm nor on-line identification of nonlinear systems, but only the on-line identified linear model. This control approach was inspired by the ideas and solutions given by several articles [16][17][18][19][20] . It starts with the selection of the value/cost function in a systematic way, 16 and follows by the Linear Approximate Dynamic Programming (LADP) model-free adaptive control approach.…”

mentioning

confidence: 99%

An Incremental Approximate Dynamic Programming Flight Controller Based on Output Feedback

Zhou

Kampen

Chu

2016

AIAA Guidance, Navigation, and Control Conference

View full text Add to dashboard Cite

A self-learning adaptive flight control for nonlinear systems allows a reliable, faulttolerant and effective operation of complex flight vehicles in a dynamic environment. Approximate dynamic programming provides a model-free control design for nonlinear systems with complex design processes and non-guaranteed closed-loop convergence properties. Linear approximate dynamic programming systematically applies a quadratic cost-togo function and greatly simplifies the design process of approximate dynamic programming. This paper presents a newly developed self-learning adaptive control method called incremental approximate dynamic programming for nonlinear unknown systems. It combines the advantages of linear approximate dynamic programming methods and incremental control techniques to generate a near-optimal control without a priori knowledge of the system model. In this paper, two incremental approximate dynamic programming algorithms with the direct availability of full states and with only the availability of system outputs have been developed. Both algorithms have been applied to a nonlinear aerospace related simulation model. The simulation results demonstrate that both model-free adaptive control algorithms improve the closed-loop performance of the nonlinear system, while keeping the design process simple and systematic as compared to conventional approximate dynamic programming algorithms.

show abstract

A stabilizing reinforcement learning approach for sampled systems with partially unknown models

Beckenbach,

Osinenko,

Streif

2024

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Reinforcement learning is commonly associated with training of reward‐maximizing (or cost‐minimizing) agents, in other words, controllers. It can be applied in model‐free or model‐based fashion, using a priori or online collected system data to train involved parametric architectures. In general, online reinforcement learning does not guarantee closed loop stability unless special measures are taken, for instance, through learning constraints or tailored training rules. Particularly promising are hybrids of reinforcement learning with classical control approaches. In this work, we suggest a method to guarantee practical stability of the system‐controller closed loop in a purely online learning setting, in other words, without offline training. Moreover, we assume only partial knowledge of the system model. To achieve the claimed results, we employ techniques of classical adaptive control. The implementation of the overall control scheme is provided explicitly in a digital, sampled setting. That is, the controller receives the state of the system and computes the control action at discrete, specifically, equidistant moments in time. The method is tested in adaptive traction control and cruise control where it proved to significantly reduce the cost.

show abstract

Quadratic approximate dynamic programming for input‐affine systems

Cited by 26 publications

References 39 publications

Modern Probabilistic Machine Learning and Control Methods for Portfolio Optimization

Modern Probabilistic Machine Learning and Control Methods for Portfolio Optimization

An Incremental Approximate Dynamic Programming Flight Controller Based on Output Feedback

A stabilizing reinforcement learning approach for sampled systems with partially unknown models

Contact Info

Product

Resources

About