Low-discrepancy sampling for approximate dynamic programming with local approximators

Cervellera, Cristiano; Gaggero, Mauro; Macciò, Danilo

doi:10.1016/j.cor.2013.09.006

Cited by 19 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ξ (4) ξ (5) ξ (6) ξ (7) ξ (8) ξ (9) ξ (10) ξ (11) ξ (12) ξ (13) ξ (14) ξ (15) (in terms of number of steps to reach the goal). For each ξ ∈ Σ, let N (ξ) be the number of steps that were needed to reach the goal (i.e., N (ξ (j) ) = T j − t j in the previous expression).…”

Section: A Problem Statementmentioning

confidence: 99%

“…They are commonly employed in the RL and ADP contexts [8], [27]. In this paper, the main advantage of their use with respect to models like neural networks lies in the fact that their output depends structurally on the available data samples, and only marginally on the vector of parameters w (for this reason, they are commonly called "non-parametric" approximators).…”

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

“…The bandwidth can still be derived offline through a cross validation approach using a least squares criterion similar to (6) (see, e.g., [8]). However, suitable heuristics can be applied to derive a reasonable value for the parameter directly online, without offline optimization.…”

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

“…However, suitable heuristics can be applied to derive a reasonable value for the parameter directly online, without offline optimization. In the literature it is possible to find approaches in this sense (see, e.g., [8], [28], [29]) relying on suitable hypotheses on the available data samples. In general, the idea is to choose a value of the widths ensuring that only kernels centered in a proximity of z t = (x t , u t ) actually contribute to the output.…”

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

“…This algorithm relies on the definition of a value function (or cost-to-go function) at each time step, which corresponds to the optimal cost to operate the system from the current time instant through the end of the time horizon. The value functions are then approximated offline via suitable approximation schemes [2]- [8].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

An approach to exploit non-optimized data for efficient control of unknown systems through neural and kernel models

Cervellera

Gaggero

Macciò

et al. 2014

2014 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

In this paper, efficient real time control strategies are devised for systems with unknown state equation, based only on a set of data inherited from non-optimized, possibly inefficient, operation of the system, in the case in which experimenting online with the latter is impossible or costly. Neural networks and kernel smoothing models are employed as architectures for learning the system dynamics. The former require an offline training phase to learn the state equation, whereas the latter exploit the available data in a direct fashion, thus making the proposed approach directly applicable online and able to exploit new available data without the need of an offline training. Convergence properties of the proposed algorithm for generating the control strategies are provided under suitable hypotheses. Simulation results on classic benchmark systems are reported for performance evaluation, also through a comparison with the SARSA reinforcement learning algorithm.

show abstract

Section: A Problem Statementmentioning

confidence: 99%

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

Section: B Local Approximators Based On Nadaraya-watson Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An approach to exploit non-optimized data for efficient control of unknown systems through neural and kernel models

Cervellera

Gaggero

Macciò

et al. 2014

2014 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

Data‐driven performance metrics for neural network learning

Alessandri,

Gaggero,

Sanguineti

2023

Adaptive Control & Signal

View full text Add to dashboard Cite

SummaryEffectiveness of data‐driven neural learning in terms of both local mimima trapping and convergence rate is addressed. Such issues are investigated in a case study involving the training of one‐hidden‐layer feedforward neural networks with the extended Kalman filter, which reduces the search for the optimal network parameters to a state estimation problem, as compared to descent‐based methods. In this respect, the performances of the training are assessed by using the Cramér‐Rao bound, along with a novel metric based on an empirical criterion to evaluate robustness with respect to local minima trapping. Numerical results are provided to illustrate the performances of the training based on the extended Kalman filter in comparison with gradient‐based learning.

show abstract

Lattice point sets for state sampling in approximate dynamic programming

Cervellera

Gaggero

Macciò

2017

Optim Control Appl Methods

View full text Add to dashboard Cite

Summary This paper investigates the use of lattice point sets as an efficient method to sample uniformly the state space of discrete‐time dynamic systems for the solution of finite‐horizon optimal control problems using approximate dynamic programming. Lattice point sets are a kind of discretization method, commonly employed for efficient numerical integration, providing a regular and balanced sampling of the state space based on the repetition of elementary unit cells. A convergence analysis of the approximate solution of the control problem to the optimal one is provided, pointing out that such sampling schemes allow one to efficiently exploit possible regularities of the cost‐to‐go functions. Furthermore, it is shown that a higher accuracy may be obtained through suitable transformations of the state vector of the dynamic system. Another advantage of lattice point sets over other sampling schemes is the possibility of evaluating a priori the goodness of a given set over another through the explicit computation of a specific parameter. Simulation results concerning the optimal control of a water reservoirs system are presented to show the effectiveness of the proposed approach.

show abstract

Low-discrepancy sampling for approximate dynamic programming with local approximators

Cited by 19 publications

References 13 publications

An approach to exploit non-optimized data for efficient control of unknown systems through neural and kernel models

An approach to exploit non-optimized data for efficient control of unknown systems through neural and kernel models

Data‐driven performance metrics for neural network learning

Lattice point sets for state sampling in approximate dynamic programming

Contact Info

Product

Resources

About