Convergence of Learning Algorithms in Neural Networks for Adaptive Identification of Nonlinearly Parameterized Systems

Zhiteckii, Leonid S.; Azarskov, Valerii N.; Nikolaienko, Sergii A.

doi:10.3182/20120711-3-be-2027.00150

Cited by 4 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It turned out that, at least, in the ideal case, the set , W * containing these s w * becomes not one-point [26,27] …”

Section: Preliminariesmentioning

confidence: 99%

“…This paper is an extension of [26,27]. The main efforts is focused on establishing sufficient conditions under which the global convergence of gradient algorithm for learning neural networks models in the stochastic environments is ensured.…”

Section: Introductionmentioning

confidence: 99%

“…This approach has been exploited in [26,27] to derive some local convergence in stochastic framework for standard online gradient algorithms with the constant learning rate.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems

Azarskov¹,

Kucherov²,

Nikolaienko³

et al. 2015

AJNNA

View full text Add to dashboard Cite

This paper deals with studying the asymptotical properties of multilayer neural networks models used for the adaptive identification of wide class of nonlinearly parameterized systems in stochastic environment. To adjust the neural network's weights, the standard online gradient type learning algorithms are employed. The learning set is assumed to be infinite but bounded. The Lyapunov-like tool is utilized to analyze the ultimate behaviour of learning processes in the presence of stochastic input variables. New sufficient conditions guaranteeing the global convergence of these algorithms in the stochastic frameworks are derived. The main their feature is that they need no a penalty term to achieve the boundedness of weight sequence. To demonstrate asymptotic behaviour of the learning algorithms and support the theoretical studies, some simulation examples are also given.

show abstract

“…It turned out that, at least, in the ideal case, the set , W * containing these s w * becomes not one-point [26,27] …”

Section: Preliminariesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems

Azarskov¹,

Kucherov²,

Nikolaienko³

et al. 2015

AJNNA

View full text Add to dashboard Cite

show abstract

“…This approach has been exploited by the authors in [24] to derive some local convergence in stochastic framework for standard online gradient algorithms with the constant learning rate.…”

Section: Introductionmentioning

confidence: 99%

“…Several of these authors assumed that training set must be finite whereas in online identification schemes, this set is theoretically infinite. Nevertheless, recently we observed a non-stochastic learning process when this procedure did not converge for certain infinite sequence of training examples [24].…”

Section: Introductionmentioning

confidence: 99%

Some Features of Neural Networks as Nonlinearly Parameterized Models of Unknown Systems Using an Online Learning Algorithm

Zhiteckii¹,

Azarskov²,

Nikolaienko³

et al. 2018

JAMP

Self Cite

View full text Add to dashboard Cite

This paper deals with deriving the properties of updated neural network model that is exploited to identify an unknown nonlinear system via the standard gradient learning algorithm. The convergence of this algorithm for online training the three-layer neural networks in stochastic environment is studied. A special case where an unknown nonlinearity can exactly be approximated by some neural network with a nonlinear activation function for its output layer is considered. To analyze the asymptotic behavior of the learning processes, the so-called Lyapunov-like approach is utilized. As the Lyapunov function, the expected value of the square of approximation error depending on network parameters is chosen. Within this approach, sufficient conditions guaranteeing the convergence of learning algorithm with probability 1 are derived. Simulation results are presented to support the theoretical analysis.

show abstract

Convergence properties of an online learning algorithm in neural network models of complex systems

Azarskov

Nikolaienko

Zhiteckii

2013

2013 IEEE 2nd International Conference Actual Problems of Unmanned Air Vehicles Developments Proceedings (APUAVD)

View full text Add to dashboard Cite

Convergence of Learning Algorithms in Neural Networks for Adaptive Identification of Nonlinearly Parameterized Systems

Cited by 4 publications

References 21 publications

Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems

Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems

Some Features of Neural Networks as Nonlinearly Parameterized Models of Unknown Systems Using an Online Learning Algorithm

Convergence properties of an online learning algorithm in neural network models of complex systems

Contact Info

Product

Resources

About