The effect of initial weights on premature saturation in back-propagation learning

Lee, Y.; Oh, Seon Ho; Kim, M.W.

doi:10.1109/ijcnn.1991.155275

Cited by 75 publications

(38 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such networks are good tools for forecasting issues, however, they have several limitations such as low learning speed and falling into local minima [32][33][34]. As mentioned in literatures [20,23,[35][36][37][38][39], using efficient optimization algorithms (OAs), these limitations can be overcome.…”

Section: Introductionmentioning

confidence: 99%

Three hybrid intelligent models in estimating flyrock distance resulting from blasting

Koopialipoor

Fallah

Armaghani

et al. 2018

Engineering with Computers

155

View full text Add to dashboard Cite

Flyrock is an adverse effect produced by blasting in open-pit mines and tunnelling projects. So, it seems that the precise estimation of flyrock is essential in minimizing environmental effects induced by blasting. In this study, an attempt has been made to evaluate/predict flyrock induced by blasting through applying three hybrid intelligent systems, namely imperialist competitive algorithm (ICA)-artificial neural network (ANN), genetic algorithm (GA)-ANN and particle swarm optimization (PSO)-ANN. In fact, ICA, PSO and GA were used to adjust weights and biases of ANN model. To achieve the aim of this study, a database composed of 262 datasets with six model inputs including burden to spacing ratio, blast-hole diameter, powder factor, stemming length, the maximum charge per delay, and blast-hole depth and one output (flyrock distance) was established. Several parametric investigations were conducted to determine the most effective factors of GA, ICA and PSO algorithms. Then, at the end of modelling process of each hybrid model, eight models were constructed and their results were checked considering two performance indices, i.e., root mean square error (RMSE) and coefficient of determination (R 2 ). The obtained results showed that although all predictive models are able to approximate flyrock, PSO-ANN predictive model can perform better compared to others. Based on R 2 , values of (0.943, 0.958 and 0.930) and (0.958, 0.959 and 0.932) were found for training and testing of ICA-ANN, PSO-ANN and GA-ANN predictive models, respectively. In addition, RMSE values of (0.052, 0.045 and 0.057) and (0.045, 0.044 and 0.058) were achieved for training and testing of ICA-ANN, PSO-ANN and GA-ANN predictive models, respectively. These results show higher efficiency of the PSO-ANN model in predicting flyrock distance resulting from blasting. Moreover, sensitivity analysis shows that hole diameter is more effective than others.

show abstract

Section: Introductionmentioning

confidence: 99%

Three hybrid intelligent models in estimating flyrock distance resulting from blasting

Koopialipoor

Fallah

Armaghani

et al. 2018

Engineering with Computers

155

View full text Add to dashboard Cite

show abstract

“…Although ANN can solve complex engineering problems, it has a number of disadvantages; for example, slow learning rate and getting trapped in local minima (e.g., [43]). In order to dominate the ANN problems, utilizing several optimization algorithms (OA) such as genetic algorithm (GA) for adjusting the weight and bias of ANNs to enhance the performance capacity of them, is of advantage.…”

Section: Introductionmentioning

confidence: 99%

An optimized ANN model based on genetic algorithm for predicting ripping production

Mohamad

Faradonbeh

Armaghani

et al. 2016

Neural Comput & Applic

View full text Add to dashboard Cite

Due to the environmental constraints and the limitations on blasting, ripping as a ground loosening and breaking method has become more popular in both mining and civil engineering applications. As a result, a more applicable rippability model is required to predict ripping production (Q) before conducting such tests. In this research, a hybrid genetic algorithm (GA) optimized by artificial neural network (ANN) was developed to predict ripping production results obtained from three sites in Johor state, Malaysia. It should be noted that the mentioned hybrid model was first time applied in this field. In this regard, 74 ripping tests were investigated in the studied areas and the relevant parameters were also measured. A series of GA-ANN models were conducted in order to propose a hybrid model with a higher accuracy level. To demonstrate the performance capacity of the hybrid GA-ANN model, a pre-developed ANN model was also proposed and results of predictive models were compared using several performance indices. The results revealed higher accuracy of the proposed hybrid GA-ANN model in estimating Q compared to ANN technique. As an example, root-mean-square error values of 0.092 and 0.131 for testing datasets of GA-ANN and ANN techniques, respectively, express the superiority of the newly developed model in predicting ripping production.

show abstract

“…Effective weight initialization is associated to performance characteristics such as the time needed to successfully train the network and the generalization ability of the trained network. Inappropriate weight initialization is very likely to increase the training time or even to cause non convergence of the training algorithm, while another unfortunate result may be to decrease the network's ability to generalize well, especially when training with backpropagation (BP), a procedure suffering from local minima, (Haykin, 1999;Hassoun, 1995;Lee et al, 1991). These are defaults and limitations for having successful practical application of neural networks in real life processes.…”

Section: Introductionmentioning

confidence: 99%

Solving the linear interval tolerance problem for weight initialization of neural networks

Adam¹,

Karras²,

Magoulas³

et al. 2014

Neural Networks

View full text Add to dashboard Cite

Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval Analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed Linear Interval Tolerance Approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.

show abstract

The effect of initial weights on premature saturation in back-propagation learning

Cited by 75 publications

References 4 publications

Three hybrid intelligent models in estimating flyrock distance resulting from blasting

Three hybrid intelligent models in estimating flyrock distance resulting from blasting

An optimized ANN model based on genetic algorithm for predicting ripping production

Solving the linear interval tolerance problem for weight initialization of neural networks

Contact Info

Product

Resources

About