A memoryless BFGS neural network training algorithm

Apostolopoulou, M.S.; Sotiropoulos, D. G.; Livieris, Ioannis E.; Pintelas, Panayiotis E.

doi:10.1109/indin.2009.5195806

Cited by 12 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Andrei [28] and Yao and Ning [29] proposed two conjugate gradient methods, where the search direction was given by a symmetrical Perry matrix and was associated with a positive parameter determined by minimizing the distance of this matrix and the self-scaling memoryless BFGS matrix in the Frobenius norm. Apostolopoulou et al [30] presented a curvilinear algorithmic model for training neural networks which was based on modifications of the memoryless BFGS method by incorporating a curvilinear search. In summary, the above-mentioned algorithms have global convergence and satisfactory numerical efficiency, but stability of the algorithms cannot be guaranteed.…”

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Single-Parameter Scaling Memoryless Broyden-Fletcher-Goldfarb-Shanno Algorithm for Solving Large Scale Unconstrained Optimization Problems

Deng

Wan

2020

IEEE Access

View full text Add to dashboard Cite

In this paper, a new spectral scaling memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm is developed for solving large scale unconstrained optimization problems, where the scaling parameter is chosen so as to minimize all the eigenvalues of search direction matrices. The search directions in this algorithm are proved to satisfy the approximate Dai-Liao conjugate condition. With this advantage of the search directions, a scaling memoryless BFGS update formula is constructed and an algorithm is developed by incorporating acceleration strategy of line search and restart criterion. Under mild assumptions, global convergence of the algorithm is proved. Numerical tests demonstrate that the developed algorithm is more robust and efficient in solving large scale benchmark test problems than the similar ones in the literature.INDEX TERMS Computational efficiency, convergence of numerical methods, optimization methods, algorithm design and analysis.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Single-Parameter Scaling Memoryless Broyden-Fletcher-Goldfarb-Shanno Algorithm for Solving Large Scale Unconstrained Optimization Problems

Deng

Wan

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, this method requires a lot of memory to converge, especially on a large scale [66][67][68][69], whereas many researchers are interested in how to reduce memory needs [67][68][69][70][71].…”

Section: Comparison Between Model and Anns Outputsmentioning

confidence: 99%

“…where H k is the Hessian matrix of the performance index at the current values of the weights and biases. When H k is large, w k+1 computation is complex and time-consuming [66][67][68]. BFGS does not calculate the inverse Hessian but approximates it as follows:…”

Section: Quasi-newton Bfgs Bp Algorithmmentioning

confidence: 99%

“…This method has several advantages: it has a better convergence rate than using conjugate gradients [58][59][60][61], it is stable because the BFGS Hessian update is symmetric and positive definite [60], and BFGS computes an approximation to the inverse Hessian in only O(n 2 ) operations [60]. However, this method requires a lot of memory to converge, especially on a large scale [66][67][68][69], whereas many researchers are interested in how to reduce memory needs [67][68][69][70][71].…”

Section: Quasi-newton Bfgs Bp Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Estimation of speed, armature temperature, and resistance in brushed DC machines using a CFNN based on BFGS BP

Mellah¹,

Hemsas²,

Taleb³

et al. 2018

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

In this paper, a sensorless speed and armature resistance and temperature estimator for Brushed (B) DC machines is proposed, based on a Cascade-Forward Neural Network (CFNN) andQuasi-Newton BFGS backpropagation (BP). Since we wish to avoid the use of a thermal sensor, a thermal model is needed to estimate the temperature of the BDC machine. Previous studies propose either non-intelligent estimators which depend on the model, such as the Extended Kalman Filter (EKF) and Luenberger's observer, or estimators which do not estimate the speed, temperature and resistance simultaneously. The proposed method has been verified both by simulation and by comparison with the measurements and simulation results available in the literature.

show abstract

“…The advantage of using these methods is that we don't need to solve a linear system to get the search direction, but only perform a matrix/vector multiply. The quasi-Newton method used in this paper is the Broyden, Fletcher, Goldfarb, and Shanno (BFGS) [14]. This algorithm is implemented in MATLAB using the neural network toolbox nprtool.…”

Section: B Learning Algorithmmentioning

confidence: 99%

Efficient and reliable lung nodule detection using a neural network based computer aided diagnosis system

Ashwin¹,

Ramesh²,

Kumar³

et al. 2012

2012 International Conference on Emerging Trends in Electrical Engineering and Energy Management (ICETEEEM)

View full text Add to dashboard Cite

The manual examination of histological images like computed tomography (CT) images by physicians is prone to subjectivity and limited intra and inter-surgeon reproducibility, due to its heavy reliance on human interpretation. As result of which, diagnosis of cancer especially in lungs becomes less accurate and unreliable. So, a computer-aided diagnosis (CAD) system, based on artificial intelligence that efficiently detects nodules of any shape and size, is used for diagnosis without human intervention. In this work, we have developed a two stage CAD system in which the first stage involves pre-processing applied for a better quality image to enable higher success rate on detection following which the cancerous nodule region is segmented. The second stage involves artificial neural network (ANN) architecture which is trained using a modified BFGS algorithm.The proposed system was trained, tested, and evaluated specifically on the problem of detecting lung cancer nodules found on CT images to give a positive detection. A significant comparative analysis was done between the proposed method and several existing CAD systems used for lung nodule diagnosis and the proposed method using training-based neural networks prove to provide accuracy of 96.7% and also better specificity; thus, the overall performance of the CAD scheme was improved substantially.

show abstract

A memoryless BFGS neural network training algorithm

Cited by 12 publications

References 17 publications

An Efficient Single-Parameter Scaling Memoryless Broyden-Fletcher-Goldfarb-Shanno Algorithm for Solving Large Scale Unconstrained Optimization Problems

An Efficient Single-Parameter Scaling Memoryless Broyden-Fletcher-Goldfarb-Shanno Algorithm for Solving Large Scale Unconstrained Optimization Problems

Estimation of speed, armature temperature, and resistance in brushed DC machines using a CFNN based on BFGS BP

Efficient and reliable lung nodule detection using a neural network based computer aided diagnosis system

Contact Info

Product

Resources

About