&lt;title&gt;Application of simulated annealing to the backpropagation model improves convergence&lt;/title&gt;

Owen, Charles B.; Abunawass, Adel M.

doi:10.1117/12.152626

Cited by 15 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the BPNN is prone to be trapped into a local minimum, global and stochastic search technologies, e.g. simulated annealing (Owen and Abunawass 1993), electromagnetism algorithm (Ma and Ji 1998), and GAs (Hong et al 2001, Zhao andHuang 2002), were introduced and proved to be useful in some specific applications.…”

Section: Adaptive Back-propagation Neural Networkmentioning

confidence: 99%

Intelligent information retrieval system using automatic thesaurus construction

Song

et al. 2011

International Journal of General Systems

View full text Add to dashboard Cite

This paper presents an intelligent information retrieval (IR) system based on automatic thesaurus construction for its applications of document clustering and classification. These two applications are the most influential and widely used fields amongst the IR research community. We apply two biologically inspired algorithms, i.e. genetic algorithm (GA) and neural network (NN), to these two fields. A fuzzy logic controller GA and an adaptive back-propagation NN are proposed in our study, which can validly overcome the problems existing in their archetypes, e.g. slow evolution and being prone to trap into a local optimum. Furthermore, a well-constructed thesaurus has been recognised as a valuable tool in the effective operation of clustering and classification. It solves the problem in document representation organised by a bag of words, where some important relationships between words, e.g. synonymy and polysemy, are ignored. To investigate how our IR system could be used effectively, we conduct experiments on four data sets from the benchmark Reuter-21578 document collection and 20-newsgroup corpus. The results reveal that our IR system enhances the performance in comparison with k-means, common GA, and conventional backpropagation NN.

show abstract

Section: Adaptive Back-propagation Neural Networkmentioning

confidence: 99%

Intelligent information retrieval system using automatic thesaurus construction

Song

et al. 2011

International Journal of General Systems

View full text Add to dashboard Cite

show abstract

“…To avoid the local-trap problem, simulated annealing (SA) (Kirkpatrick et al, 1983) has been employed by some authors to train neural networks. Amato et al (1991) and Owen & Abunawass (1993) show that for complex learning tasks, SA has a better chance to converge to a global minimum than have the gradientbased algorithms. Geman & Geman (1984) show that the global minimum can be reached by SA with probability 1 if the temperature decreases at a logarithmic rate of O(1/log t), where t denotes the number of iterations.…”

Section: Introductionmentioning

confidence: 99%

Stochastic Approximation Monte Carlo for MLP Learning

Liang

2009

Encyclopedia of Artificial Intelligence

View full text Add to dashboard Cite

Over the past several decades, multilayer perceptrons (MLPs) have achieved increased popularity among scientists, engineers, and other professionals as tools for knowledge representation. Unfortunately, there is no a universal architecture which is suitable for all problems. Even with the correct architecture, frustrating problems of connection weights training still remain due to the rugged nature of the energy landscape of MLPs. The energy function often refers to the sum-of-square error function for conventional MLPs and the negative logposterior density function for Bayesian MLPs. This article presents a Monte Carlo method that can be used for MLP learning. The main focus is on how to apply the method to train connection weights for MLPs. How to apply the method to choose the optimal architecture and to make predictions for future values will also be discussed, but within the Bayesian framework.

show abstract

“…Amato, Apolloni, Caporali, Madesani, & Zanaboni (1991) and Owen and Abunawass (1993) used simulated annealing to improve convergence. This method could guarantee global minimum convergence if the cooling temperature is slow enough.…”

Section: Introductionmentioning

confidence: 99%

An Algorithm of Supervised Learning for Multilayer Neural Networks

et al. 2003

View full text Add to dashboard Cite

A method of supervised learning for multilayer arti cial neural networks to escape local minima is proposed. The learning model has two phases: a backpropagation phase and a gradient ascent phase. The backpropagation phase performs steepest descent on a surface in weight space whose height at any point in weight space is equal to an error measure, and it nds a set of weights minimizing this error measure. When the backpropagation gets stuck in local minima, the gradient ascent phase attempts to ll up the valley by modifying gain parameters in a gradient ascent direction of the error measure. The two phases are repeated until the network gets out of local minima. The algorithm has been tested on benchmark problems, such as exclusive-or (XOR), parity, alphabetic characters learning, Arabic numerals with a noise recognition problem, and a realistic real-world problem: classi cation of radar returns from the ionosphere. For all of these problems, the systems are shown to be capable of escaping from the backpropagation local minima and converge faster when using the new proposed method than using the simulated annealing techniques.

show abstract

<title>Application of simulated annealing to the backpropagation model improves convergence</title>

Cited by 15 publications

References 0 publications

Intelligent information retrieval system using automatic thesaurus construction

Intelligent information retrieval system using automatic thesaurus construction

Stochastic Approximation Monte Carlo for MLP Learning

An Algorithm of Supervised Learning for Multilayer Neural Networks

Contact Info

Product

Resources

About