Uniformly Stable Backpropagation Algorithm to Train a Feedforward Neural Network

Abstract

Neural networks (NNs) have numerous applications to online processes, but the problem of stability is rarely discussed. This is an extremely important issue because, if the stability of a solution is not guaranteed, the equipment that is being used can be damaged, which can also cause serious accidents. It is true that in some research papers this problem has been considered, but this concerns continuous-time NN only. At the same time, there are many systems that are better described in the discrete time domain such as population of animals, the annual expenses in an industry, the interest earned by a bank, or the prediction of the distribution of loads stored every hour in a warehouse. Therefore, it is of paramount importance to consider the stability of the discrete-time NN. This paper makes several important contributions. 1) A theorem is stated and proven which guarantees uniform stability of a general discrete-time system. 2) It is proven that the backpropagation (BP) algorithm with a new time-varying rate is uniformly stable for online identification and the identification error converges to a small zone bounded by the uncertainty. 3) It is proven that the weights' error is bounded by the initial weights' error, i.e., overfitting is eliminated in the proposed algorithm. 4) The BP algorithm is applied to predict the distribution of loads that a transelevator receives from a trailer and places in the deposits in a warehouse every hour, so that the deposits in the warehouse are reserved in advance using the prediction results. 5) The BP algorithm is compared with the recursive least square (RLS) algorithm and with the Takagi-Sugeno type fuzzy inference system in the problem of predicting the distribution of loads in a warehouse, giving that the first and the second are stable and the third is unstable. 6) The BP algorithm is compared with the RLS algorithm and with the Kalman filter algorithm in a synthetic example.

Feasibility of an AI-Based Measure of the Hand Motions of Expert and Novice Surgeons

“…These data will be input as training data, and we plan to perform the AI learning cycle again. The trial and improvement method is the best way to construct a high-accuracy neural network [ 13 , 14 ].…”

“…To optimize its ability to correctly distinguish between experienced and novice surgeons, the neural network learned via machine learning from datasets consisting of input factors from 38 participants (expert group: 11; novice group: 27) ( Figure 1 ). The backpropagation algorithm was employed as a learning strategy [ 13 ]. Learning by the system was repeated until the two groups of surgeons were correctly distinguished.…”

Inversion of Regional Economic Trend from NPP-VIIRS Nighttime Light Data Based on Adaptive Clustering Algorithm

“…en, we put the samples into the three-layer BP neural network with only one hidden layer to train a feed-forward neural network [15] and get the classification result. e logarithm was selected as activation function and the other parameters of BP neural network were set as follows: initial weight σ � 0.9, weight adjustment speed η � 0.2, momentum factor α � 0.9, and global network error E � 0.1.…”

An evolutionary autoencoder for dynamic community detection

“…Eq. 5is solved by a back propagation with a stochastic gradient descent [33]. In every iteration, a set of parameters (i.e., θ…”