Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights

Nguyen, Dat Thanh; Widrow, Bernard

doi:10.1109/ijcnn.1990.137819

Cited by 1,044 publications

(554 citation statements)

References 2 publications

Supporting

Mentioning

523

Contrasting

Unclassified

Order By: Relevance

“…We cannot guarantee that these results are optimal for every considered dataset. However, the resulting intervals roughly confirm the findings for the weight intervals reported in Nguyen & Widrow (1990) and Thimm & Fiesler (1994). To the best of our knowledge there is no specific study on this matter in the literature and in light of these results this should constitute an interesting point for deeper investigation.…”

Section: Initializing Hidden-to-hidden and Hidden-to-output Layer Consupporting

confidence: 85%

“…A number of approaches such as those presented in this section claim the reputation to provide improvement in BP convergence speed and avoidance of bad local minima, (Nguyen & Widrow, 1990;Wessels & Barnard, 1992).…”

Section: Random Selection Of Initial Weightsmentioning

confidence: 99%

“…Here, n i denotes the number of inputs to the network and n h is the number of nodes in the hidden layer. In addition v 2 is the mean of the expectation of the quadratic values of the inputs, Nguyen & Widrow (1990) proposed a simple modification of the widely used random initialization process of Fahlman (1988). The weights connecting the output units In addition to the above, a number of interesting methods related to this context have been formulated by Osowski (1993); Chen & Nutter (1991); Yam & Chow (1995; LeCun (1993); Schmidhuber & Hochreiter (1996), as well as by others researchers.…”

Section: Random Selection Of Initial Weightsmentioning

confidence: 99%

See 2 more Smart Citations

Solving the linear interval tolerance problem for weight initialization of neural networks

Adam¹,

Karras²,

Magoulas³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval Analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed Linear Interval Tolerance Approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.

show abstract

Section: Initializing Hidden-to-hidden and Hidden-to-output Layer Consupporting

confidence: 85%

Section: Random Selection Of Initial Weightsmentioning

confidence: 99%

Section: Random Selection Of Initial Weightsmentioning

confidence: 99%

See 1 more Smart Citation

Solving the linear interval tolerance problem for weight initialization of neural networks

Adam¹,

Karras²,

Magoulas³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

show abstract

“…Before training our ANN models, their weights and biases were initialized according to the Nguyen-Widrow initialization algorithm [32]. The optimization of a network can be accomplished by changing the network parameters (such as the number of neurons, number of hidden layers and number of epochs) one at a time.…”

Section: Artificial Neural Network Modeling and Performance Measuresmentioning

confidence: 99%

Modeling of polygalacturonase enzyme activity and biomass production by Aspergillus sojae ATCC 20235

Tokatlı

Tarı

Ünlütürk

et al. 2009

J Ind Microbiol Biotechnol

View full text Add to dashboard Cite

Aspergillus sojae, which is used in the making of koji, a characteristic Japanese food, is a potential candidate for the production of polygalacturonase (PG) enzyme, which of a major industrial significance. In this study, fermentation data of an A. sojae system were modeled by multiple linear regression (MLR) and artificial neural network (ANN) approaches to estimate PG activity and biomass. Nutrient concentrations, agitation speed, inoculum ratio and final pH of the fermentation medium were used as the inputs of the system. In addition to nutrient conditions, the final pH of the fermentation medium was also shown to be an effective parameter in the estimation of biomass concentration. The ANN parameters, such as number of hidden neurons, epochs and learning rate, were determined using a statistical approach. In the determination of network architecture, a cross-validation technique was used to test the ANN models. Goodness-offit of the regression and ANN models was measured by the R 2 of cross-validated data and squared error of prediction. The PG activity and biomass were modeled with a 5-2-1 and 5-9-1 network topology, respectively. The models predicted enzyme activity with an R 2 of 0.84 and biomass with an R 2 value of 0.83, whereas the regression models predicted enzyme activity with an R 2 of 0.84 and biomass with an R 2 of 0.69.

show abstract

“…The parameters of the network are initialised according to the Nguyen-Widrow method [19] and a 1 and a 2 are initially set to one and zero, respectively. It should be noted that the objective function E is adapted at each iteration since the hyperparameters a i are re-estimated.…”

Section: Methodsmentioning

confidence: 99%

An ischemia detection method based on artificial neural networks

Papaloukas

Fotiadis

Likas

et al. 2002

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

An automated technique was developed for the detection of ischemic episodes in long duration electrocardiographic (ECG) recordings that employs an artificial neural network. In order to train the network for beat classification, a cardiac beat dataset was constructed based on recordings from the European Society of Cardiology (ESC) ST-T database. The network was trained using a Bayesian regularisation method. The raw ECG signal containing the ST segment and the T wave of each beat were the inputs to the beat classification system and the output was the classification of the beat. The input to the network was produced through a principal component analysis (PCA) to achieve dimensionality reduction. The network performance in beat classification was tested on the cardiac beat database providing 90% sensitivity (Se) and 90% specificity (Sp). The neural beat classifier is integrated in a four-stage procedure for ischemic episode detection. The whole system was evaluated on the ESC ST-T database. When aggregate gross statistics was used the Se was 90% and the positive predictive accuracy (PPA) 89%. When aggregate average statistics was used the Se became 86% and the PPA 87%. These results are better than other reported. #

show abstract

Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights

Cited by 1,044 publications

References 2 publications

Solving the linear interval tolerance problem for weight initialization of neural networks

Solving the linear interval tolerance problem for weight initialization of neural networks

Modeling of polygalacturonase enzyme activity and biomass production by Aspergillus sojae ATCC 20235

An ischemia detection method based on artificial neural networks

Contact Info

Product

Resources

About