The Kernel Recursive Least-Squares Algorithm

Engel, Yaakov; Mannor, Shie; Meir, Ron

doi:10.1109/tsp.2004.830985

Cited by 961 publications

(671 citation statements)

References 9 publications

Supporting

Mentioning

649

Contrasting

Unclassified

Order By: Relevance

“…The Kernel Recursive Least Squares (KRLS) algorithm was introduced in [4] and has a conceptual foundation related to Principal Component Analysis (PCA) and Support Vector Machines (SVM). KRLS algorithm produces much sparser solutions with higher robustness to noise.…”

Section: Online Krls-svm Learningmentioning

confidence: 99%

“…We use KRLS algorithm as proposed in [4] for reinforcement learning. Q-learning method based on KRLS-SVM can be summarized as follows:…”

Section: Online Krls-svm Learningmentioning

confidence: 99%

“…The objective in hand is to explore the use of a support vector machine with sparse support vectors, low computational cost and satisfactory accuracy. The kernel recursive least-squares (KRLS) -SVM [4], is a strong candidate to achieve this objective. We develop a KRLS-SVM algorithm for reinforcement learning and demonstrate its potential through case study -two-link robot manipulator.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reinforcement Learning with Kernel Recursive Least-Squares Support Vector Machine

Shah¹,

Gopal²

2012

IJMLC

View full text Add to dashboard Cite

Abstract-A reinforcement learning system based on the kernel recursive least-squares algorithm for continuous state-space is proposed in this paper. A kernel recursive least-squares-support vector machine is used to realized a mapping from state-action pair to Q-value function. An online sparsification process that permits the addition of training sample into the Q-function approximation only if it is approximately linearly independent of the preceding training samples. Simulation result of two-link robot manipulator show that the proposed method has high learning efficiency -better accuracy measured in terms of mean square error, and lesser computation time compare to the least-squares support vector machine.Index Terms-Kernel methods, least-squares support vector machine, recursive least squares, reinforcement learning. I. INTRODUCTIONSupport vector machine (SVM), which is based on Vapnik's structural risk minimization (SRM) [1], has become one of the most popular methods in solving classification and regression problems. Conventional SVMs have properties of global optimization, and good adaptability. However, the optimal solutions are obtained by solving standard quadratic programming which results in high computational cost. In order to reduce the computational cost, a least square support vector (LS-SVM) was proposed in [2] by converting inequality constraints to linear equations. LS-SVM has been successfully applied to reinforcement learning (RL) problems [3]. A RL problem is converted into a regression problem, wherein the observed states and actions are considered as inputs and Q-value functions as output. All the training samples may be support vectors in LS-SVM, and thus the support vectors are no longer sparse. It may lead to poor generalization. In addition, with the number of the input training pairs increasing, the number of equations will increase which may result in higher computational cost.Our focus in this paper is on reinforcement learning problems. The objective in hand is to explore the use of a support vector machine with sparse support vectors, low computational cost and satisfactory accuracy. The kernel recursive least-squares (KRLS) -SVM [4], is a strong candidate to achieve this objective. We develop a KRLS-SVM algorithm for reinforcement learning and demonstrate its potential through case study -two-link robot manipulator. The mean square error accuracy, computational Manuscript received

show abstract

Section: Online Krls-svm Learningmentioning

confidence: 99%

“…We use KRLS algorithm as proposed in [4] for reinforcement learning. Q-learning method based on KRLS-SVM can be summarized as follows:…”

Section: Online Krls-svm Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning with Kernel Recursive Least-Squares Support Vector Machine

Shah¹,

Gopal²

2012

IJMLC

View full text Add to dashboard Cite

show abstract

“…There also exists an online sparsification process used in kernel recursive least squares, [75], that reduces computational complexity. The evaluation of a kernel density at a point can be considered as a sum of weights and a set of feature vectors.…”

Section: Related Workmentioning

confidence: 99%

“…In the KDE literature, methods that tackle overall computational complexity primarily approach from the perspective of reducing pairwise kernel evaluations; examples include Nystrom approximation [72], Fast Gauss Transform [73,74] and sparse dictionary learning methods [75].…”

Section: Introductionmentioning

confidence: 99%

Manifold learning and unwrapping using density ridges

Shaker¹

View full text Add to dashboard Cite

xii 3.6 Unwrapped mnist-0 digits with a selection of images of the actual digits on top. Orientation and thickness of digits smoothly vary across horizontal and vertical directions, respectively. . 34 3.7 Unfolding mnist-0 digits using two benchmark manifold learning methods. Unlike the proposed method, no smooth variation of structure is preserved using these methods. . . . . 35 3.8 Unfolding mnist-2 digits using the proposed method and t-SNE. Similar to Figure 3.7(b) for mnist-0, no smooth variation of structure is preserved using t-SNE, while proposed method shows smooth change of thickness and shape in horizontal and vertical coordinates, respectively. 36 3.9 Unfolding mnist-9 digits using the proposed method, which shows smooth change of thickness Sylvain Bouix, who were also my mentors, for all of their guidance through this process; your discussion, ideas, and feedback have been absolutely invaluable.

show abstract