Scaled Conjugate Gradient Algorithm in Neural Network Based Approach for Handwritten Text Recognition

Chel, Haradhan; Majumder, Aurpan; Nandi, Debashis

doi:10.1007/978-3-642-24043-0_21

Cited by 18 publications

(11 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The SCG has been applied successfully in many areas such as in inverse natural convention modeling (Wong and Protas, 2013), handwritten text identification (Chel et al, 2011), fault location in high voltage transmission lines (Gayathri and Kumarappan, 2010), solving unconstrained optimization problems (Al-Bayati and Muhammad, 2010) and estimating radar pulse modulation (Lundén and Koivunen, 2007). The basis of the SCG is the gradient descent and the conjugate gradient methods (Hestenes and Stiefel, 1952).…”

Section: Scaled Conjugate Gradient Methodsmentioning

confidence: 99%

Causal, Correlation and Automatic Relevance Determination Machines for Granger Causality

Marwala¹

2015

Causality, Correlation and Artificial Intelligence for Rational Decision Making

View full text Add to dashboard Cite

In this chapter, the causal, correlation and automatic relevance determination (ARD) machines are built to model non-linear Granger causality and are compared. The multi-layer perceptron (MLP) neural network trained using the scaled gradient method is used to build both the causal, correlation and ARD machines. The causal machine is a feed-forward MLP with past data being used as inputs to predict future values of a chaotic time series. The correlation machine is the auto-associative model which maps the past and present values to predict itself and a Nelder-Mead optimization method is used to predict future values. The ARD machine is an MLP which models the input-output relationship via Bayesian approach by penalizing the prior coefficient associated with each input variable. When these methods were tested, it was found that the correlation machine models Granger causality better than the causal machine while the ARD performs well. The complex nature of the correlation machine (inference using auto-associative neural networks) compared to the MLP causal machine renders the correlation machine more sensitive to Granger causality than the causal machine.

show abstract

Section: Scaled Conjugate Gradient Methodsmentioning

confidence: 99%

Causal, Correlation and Automatic Relevance Determination Machines for Granger Causality

Marwala¹

2015

Causality, Correlation and Artificial Intelligence for Rational Decision Making

View full text Add to dashboard Cite

show abstract

“…The most important problem with training the ANN using known algorithms (Levenberg-Marquardt [28,29], Bayesian Regularization [30,31], Scaled Conjugate Gradient [32,33]) is that even if the ANN is trained resulting with high correlation and low error between outputs and reference data (divided to training, validation, and testing data), the ANN does not show convergence when working in recurrence during hybrid simulation. When outputs from ANN become its inputs in subsequent time steps, the common problem is that the results become inconsistent with reality either by large error or lack of convergence when u I → ±∞.…”

Section: U(t) + (Cmentioning

confidence: 99%

Application of Artificial Neural Networks in Hybrid Simulation

Mucha

2019

Applied Sciences

View full text Add to dashboard Cite

Featured Application: Performing effective and efficient hybrid simulations of mechanical systems in real-time by implementing a model order reduction technique to the computational algorithm. The efficiency of the computations is understood as the ability to test systems of a higher number of degrees of freedom while maintaining high accuracy, without increasing the time step. Effective simulation is a simulation that allows acquiring correct results while maintaining the time regime.Abstract: Hybrid simulation is a technique for testing mechanical systems. It applies to structures with elements hard or impossible to model numerically. These elements are tested experimentally by straining them by means of actuators, while the rest of the system is simulated numerically using a finite element method (FEM). Data is interchanged between experiment and simulation. The simulation is performed in real-time in order to accurately recreate the dynamic behavior in the experiment. FEM is very computationally demanding, and for systems with a great number of degrees of freedom (DOFs), real-time simulation with small-time steps (ensuring high accuracy) may require powerful computing hardware or may even be impossible. The author proposed to swap the finite element (FE) model with an artificial neural network (ANN) to significantly lower the computational cost of the real-time algorithm. The presented examples proved that the computational cost could be reduced by at least one number of magnitude while maintaining high accuracy of the simulation; however, obtaining appropriate ANN was not trivial and might require many attempts. subsystem, which are measured in the form of a vector r P i+1 , where subsequent elements of the vector correspond to degrees of freedom (DOFs) of the discretized FE model [4,5]. Appl. Sci. 2019, 9, x FOR PEER REVIEW 2 of 19 are measured in the form of a vector r P i+1, where subsequent elements of the vector correspond to degrees of freedom (DOFs) of the discretized FE model [4,5].

show abstract

“…Møller (1993) and Chel et al (2011) suggested the solution for avoiding the time-consuming line search in scaled conjugate gradient method (SCGM) by combining the model-trust region approach with the conjugate-gradient approach (Borkar et al, 2016). Memory requirements for SCGM are similar to Fletcher-Reeves variation of CGM.…”

Section: Scaled Conjugate Gradient Descentmentioning

confidence: 99%

“…SCGM depends on the computation of conjugate directions but avoids time-consuming line search in every iteration. In addition, this method avoids the tedious and memory consuming computation of hessian; done in traditional second order methods (Møller, 1993;Chel et al, 2011).…”

Section: Scaled Conjugate Gradient Descentmentioning

confidence: 99%

Performance comparison of second order conjugate algorithms in neural networks for predictive data mining

Sehgal

2017

Int. j. adv. appl. sci.

View full text Add to dashboard Cite

In this paper, a performance comparison of several variations of the nonlinear conjugate gradient method has been investigated. Neural Networkbased prediction models for life insurance sector have been developed and their training has been done with a variety of first and second order algorithms to find an efficient training algorithm, but keeping the focus on conjugate gradient based methods. Traditional second order methods require computation of second order derivatives and need to compute hessian for quadratic termination; which is a tedious and memory consuming task. Here we employ conjugate gradient methods which bypass the computation of hessian, but still achieve quadratic termination and thus prove to be memory efficient.

show abstract

Scaled Conjugate Gradient Algorithm in Neural Network Based Approach for Handwritten Text Recognition

Cited by 18 publications

References 18 publications

Causal, Correlation and Automatic Relevance Determination Machines for Granger Causality

Causal, Correlation and Automatic Relevance Determination Machines for Granger Causality

Application of Artificial Neural Networks in Hybrid Simulation

Performance comparison of second order conjugate algorithms in neural networks for predictive data mining

Contact Info

Product

Resources

About