Sliding mode algorithm for training multilayerartificial neural networks

Parma, G.G.; Menezes, B.R.; Braga, Antônio de Pádua

doi:10.1049/el:19980062

Cited by 75 publications

(29 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the case of Backpropagation [18] and its variations, such as Quickprop [16], Rprop [15], Levenberg Marquardt [14], and sliding model training [13].…”

Section: Introductionmentioning

confidence: 99%

New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data

Costa

Rodrigues

Horta

et al. 2009

Studies in Computational Intelligence

View full text Add to dashboard Cite

Abstract. One of the most used Artificial Neural Networks models is the MultiLayer Perceptron, which is capable to fit any function as long as they have enough number of neurons and network layers. The process of obtaining a properly trained Artificial Neural Network usually requires a great effort in determining the parameters that will make it to learn. Currently there are a variety of algorithms for Artificial Neural Networks's training working, simply, in order to minimize the sum of mean square error. However, even if the network reaches the global minimum error, it does not imply that the model response is optimal. Basically, a network with large number of weights but with small amplitudes behaves as an underfitted model that gradually overfits data during training. Solutions that have been overfitting are unnecessary complexity solutions. Moreover, solutions with low norm of the weights are those that present underfitting, with low complexity. The Multi-Objective Algorithm controls the weights amplitude by optimizing two objective functions: the error function and norm function. The high generalization capability of the Multi-Objective Algorithm and an automatic weight selection is aggregated by the LASSO approach, which generates networks with reduced number of weights when compared with Multi-Objective Algorithm solutions. Four data sets were chosen in order to compare and evaluate MOBJ, LASSO and Early-Stopping solutions. One generated from a function and tree available from a Machine Learning Repository. Additionally, the MOBJ and LASSO algorithms are applied to a microarray data set, which samples correspond to a genetic expression profile from DNA microarray technology of neoadjuvant chemotherapy (treatment given prior to surgery) for patients with breast cancer. Originally, the dataset is composed of 133 samples with 22283 attributes. By applying e probe section method described in the literature, 30 attributes were selected and used to train the Artificial Neural Networks. In average, the MOBJ and LASSO solutions were the same, the main difference is the simplified topology achieve by LASSO training method.

show abstract

“…As the case of Backpropagation [18] and its variations, such as Quickprop [16], Rprop [15], Levenberg Marquardt [14], and sliding model training [13].…”

Section: Introductionmentioning

confidence: 99%

New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data

Costa

Rodrigues

Horta

et al. 2009

Studies in Computational Intelligence

View full text Add to dashboard Cite

show abstract

“…Some examples utilizing VSS theory have successfully demonstrated that the approach can be utilized for tracking control of uncertain systems and identification purposes [14,15]. The underlying idea is to integrate the robustness and invariance properties of SMC technique with the power of knowledge based systems like neural networks and fuzzy inference systems.…”

Section: Introductionmentioning

confidence: 99%

A robust on‐line learning algorithm for intelligent control systems

Efe

Kaynak

Wilamowski

et al. 2003

Adaptive Control & Signal

View full text Add to dashboard Cite

This paper describes a novel error extraction approach for exploiting the strength of LevenbergMarquardt (LM) optimization technique in intelligent control systems. Since the target value of the control signal is unknown, tuning of the controller parameters becomes a tedious task if the knowledge about the system and the environment is limited. The suggested methodology utilizes the sliding model control (SMC) technique. The error extraction scheme postulates the form of error on the applied control signal using the discrepancy from the prescribed reaching dynamics. The devised approach has been tested on the non-linear Duffing oscillator, which has been forced to follow a periodic orbit radically different from the natural one. The results obtained through a series of simulations have confirmed the high precision and robustness advantages without knowing the analytical details of the system under investigation. The issues of observation noise and the stability in the parametric space have approximately been addressed from the point of SMC perspective.

show abstract

“…Besides the problems mentioned above, it can be verified that the learning strategy of training algorithms based on the principle of backpropagation is not protected against external disturbances associated with excitation signals . The high performance of variable structure system control (Itkis, 1976) in dealing with uncertainties and imprecision have motivated the use of the sliding mode control (SMC) (Utkin, 1978) in training ANN (Parma et al, 1998a). This approach was chosen for three reasons: because it is a well established theory, it allows for the adjustment of parameters (weights) of the network, and it allows an analytical study of the gains involved in training.…”

mentioning

confidence: 99%

“…In , the sliding mode strategy for the learning of analog Adaline networks, proposed by Sira-Ramirez & Colina-Morles (1995), was extended to a more general class of multilayer networks with a scalar output. The first SMC learning algorithm for training multilayer perceptron (MLP) networks was proposed by Parma et al (1998a). Besides the speed up achieved with the proposed algorithm, control theory is actually used to guide neural network learning as a system to be controlled.…”

mentioning

confidence: 99%

“…A comprehensive review of VSS and SMC can be seen in Hung et al (1993), and a survey about the fusion of computationally intelligent methodologies and SMC can be found in Kaynak et al (2001). Although the methodology used by Parma et al (1998a) makes it possible to determine the limits of parameters involved in the training of MLP networks, their complexity still makes it necessary to use heuristic methods to determine the most appropriate gain to be used in order to ensure the best network performance for a particular training. In this chapter, an algorithm for on-line ANN training based on SMC is presented.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Sliding Mode Control Approach for Training On-line Neural Networks with Adaptive Learning Rate

Nied¹,

Oliveira²

2011

Sliding Mode Control

View full text Add to dashboard Cite

This chapter includes contributions to the theory of on-line training of artificial neural networks (ANN), considering the multilayer perceptrons (MLP) topology. By on-line training, we mean that the learning process is conducted while the signal processing is being executed by the system, i.e., the neural network continuously adjusts its free parameters from the variations in the incident signal in real time (Haykin, 1999). An artificial neural network is a massively parallel distributed processor made up of simple processing units, which have a natural tendency to store experimental knowledge and make it available for use (Haykin, 1999). These units (also called neurons) are non-linear adaptable devices, although very simple in terms of computing power and memory. However, when linked, they have enormous potential for nonlinear mappings. The learning algorithm is the procedure used to do the learning process, whose function is to modify the synaptic weights of the network in an orderly manner to achieve a desired goal of the project (Haykin, 1999). Although initially used only in problems of pattern recognition and signal processing and image, today, the ANN are used to solve various problems in several areas of human knowledge. An important feature of ANN is its ability to generalize, i.e., the ability of the network to provide answers in relation to standards unknown or not presented during the training phase. Among the factors that influence the generalization ability of ANN, we cite the network topology and the type of algorithm used to train the network. The network topology refers to the number of inputs, outputs, number of layers, number of neurons per layer and activation function. From the work of Cybenko (1989), networks with the MLP topology had widespread use, because they possessed the characteristic of universal approximator of continuous functions. Basically, an MLP network is subdivided into the following layers: input layer, intermediate or hidden layer(s) and output layer. The operation of an MLP network is synchronous, i.e., given an input vector, it is propagated to the output by multiplying by the weights of each layer, applying the activation function (the model of each neuron of the network includes a non-linear activation function, being the non-linearity differentiable at any point) and propagating this value to the next layer until the output layer is reached. Issues such as flexibility of the system to avoid biased solutions (under tting)and,conversely , limiting the complexity of network topology, thus avoiding the variability of solutions

show abstract

Sliding mode algorithm for training multilayerartificial neural networks

Cited by 75 publications

References 2 publications

New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data

New Multi-Objective Algorithms for Neural Network Training Applied to Genomic Classification Data

A robust on‐line learning algorithm for intelligent control systems

Sliding Mode Control Approach for Training On-line Neural Networks with Adaptive Learning Rate

Contact Info

Product

Resources

About