Linear Hopfield networks and constrained optimization

Lendaris, G.G.; Mathia, Karl; Saeks, R.

doi:10.1109/3477.740171

Cited by 43 publications

(17 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A Hopfield network with linear transfer functions augmented by an additional feedforward layer can be used to solve a set of linear equations [62] and to compute the pseudoinverse of a matrix [39]. The resultant augmented linear Hopfield network can be used to solve constrained LS optimization problems.…”

Section: Solving Other Optimization Problemsmentioning

confidence: 99%

Hopfield Networks, Simulated Annealing, and Chaotic Neural Networks

Du¹,

Swamy

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

The Hopfield model [27,28] is the most popular dynamic model. It is biologically plausible since it functions like the human retina [36]. It is a fully interconnected recurrent network with J McCulloch-Pitts neurons. The Hopfield model is usually represented by using a J -J layered architecture, as illustrated in Fig. 6.1. The input layer only collects and distributes feedback signals from the output layer. The network has a symmetric architecture with a symmetric zero-diagonal real weight matrix, that is, w i j = w ji and w ii = 0. Each neuron in the second layer sums the weighted inputs from all the other neurons to calculate its current net activation net i , then applies an activation function to net i and broadcasts the result along the connections to all the other neurons. In the figure, w ii = 0 is represented by a dashed line; φ(·) and θ are, respectively, a vector comprising the activation functions for all the neurons and a vector comprising the biases for all the neurons.The Hopfield model operates in an unsupervised manner. The dynamics of the network are described by a system of nonlinear ordinary differential equations. The discrete form of the dynamics is defined by1)where net i is the weighted net input of the ith neuron, x i (t) is the output of the ith neuron, θ i is a bias to the neuron, and φ(·) is the sigmoidal function.

show abstract

Section: Solving Other Optimization Problemsmentioning

confidence: 99%

Hopfield Networks, Simulated Annealing, and Chaotic Neural Networks

Du¹,

Swamy

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

show abstract

“…Nonlinear optimization problems, however, are nonetheless quadratic to second order around the local vicinity of the optimum. Therefore, Quadratic Programming (QP)-which finds the minima/maxima quadratic functions of variables subject to linear constraints [183]-becomes an effective first pass at such problems, and can be applied to a wide array of applications. For example, many machine learning problems, such as support vector machine (SVM) training and least squares regression, can be reformulated in terms of a QP problem.…”

Section: Nonlinear Programmingmentioning

confidence: 99%

Principles of Neuromorphic Photonics

Shastri¹,

Tait²,

Lima³

et al. 2018

Unconventional Computing

View full text Add to dashboard Cite

GLOSSARYBenchmark A standardized task that can be performed by disparate computing approaches, used to assess their relative processing merit in specific cases. Bifurcation A qualitative change in behavior of a dynamical system in response to parameter variation. Examples include cusp (from monostable to bistable), Hopf (from stable to oscillating), transcritical (exchange of stability between two steady states). Brain-inspired computing (a.k.a. neuro-inspired computing) A biologically inspired approach to build processors, devices, and computing models for applications including adaptive control, machine learning, and cognitive radio. Similarities with biological signal processing include architectural, such as distributed, representational, such as analog or spiking, or algorithmic, such as adaptation. Broadcast and Weight A multi-wavelength analog networking protocol in which multiple all photonic neuron outputs are multiplexed and distributed to all neuron inputs. Weights are reconfigured by tunable spectral filters. Excitability A far-from-equilibrium nonlinear dynamical mechanism underlying all-or-none responses to small perturbations. Fan-in The number of inputs to a neuron. Layered network A network topology consisting of a series of sets (i.e., layers) of neurons. The neurons in each set project their outputs only to neurons in the subsequent layer. Most commonly used type of network used for machine learning. Metric A quantity assessing performance of a device in reference to a specific computing approach. Microring weight bank A silicon photonic implementation of a reconfigurable spectral filter capable of independently setting transmission at multiple carrier wavelengths. Modulation The act of representing an abstract variable in a physical quantity, such as photon rate (i.e., optical power), free carrier density (i.e., optical gain), carrier drift (i.e., current). Electrooptic modulators are devices that convert from an electrical signal to the power envelope of an optical signal. Recently, the scientific community has set out to build bridges between the domains of photonic device physics and neural networks, giving rise to the field of neuromorphic photonics (Fig. 1). This article reviews the recent progress in integrated neuromorphic photonics. We provide an overview of neuromorphic computing, discuss the associated technology (microelectronic and photonic) platforms and compare their metric performance. We discuss photonic neural network approaches and challenges for integrated neuromorphic photonic processors while providing an in-depth description of photonic neurons and a candidate interconnection architecture. We conclude with a future outlook of neuro-inspired photonic processing.tion processing, describe the photonic neural-network approaches being developed by our lab and others, and photonic integrated circuit (PIC) platforms, which have recently undergone rapid growth. PICs are becoming a

show abstract

“…In all these situations, numerical methods are usually required. These methods include cyclic coordinate descent methods [11], the Levenberg-Marquardt damped least squares methods [12], [13], quasi-Newton and conjugate gradient methods [14], [11], [15], neural network and artificial intelligence methods [16], [17], [18], [19], [20], [21], genetic algorithms [22], pseudo-inverse methods [23] and Jacobian transpose methods [24], [25].…”

Section: Background and Related Workmentioning

confidence: 99%

A parallel evolutionary solution for the inverse kinematics of generic robotic manipulators

Farzan

DeSouza

2014

2014 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

This paper is an improvement of our previous work [1]. It provides a robust, fast and accurate solution for the inverse kinematics problem of generic serial manipulators -i.e. any number and any combination of revolute and prismatic joints. Here, we propose further enhancements by applying an evolutionary approach on the previous architecture and explore the effects of different parameters on the performance of the algorithm. The algorithm only requires the Denavit-Hartenberg (D-H) representation of the robot as input and no training or robot-dependent optimization function is needed. In order to handle singularities and to overcome the possibility of multiple paths in redundant robots, our approach relies on the computation of multiple (parallel) numerical estimations of the inverse Jacobian while it selects the current best path to the desired configuration of the end-effector using an evolutionary algorithm. But unlike other iterative methods, our method achieves submillimeter accuracy in 20 iterations in average. The algorithm was implemented in C/C++ using POSIX threads, and it can be easily expanded to use more threads and/or many-core GPUs. We demonstrate the high accuracy and real-time performance of our method by testing it with five different robots including a 7-DoF redundant robot. Results show that the evolutionary implementation of the algorithm is able to reduce the number of iterations compared to the previous method significantly, while also finding the solution within the specified margin of error.

show abstract

Linear Hopfield networks and constrained optimization

Cited by 43 publications

References 9 publications

Hopfield Networks, Simulated Annealing, and Chaotic Neural Networks

Hopfield Networks, Simulated Annealing, and Chaotic Neural Networks

Principles of Neuromorphic Photonics

A parallel evolutionary solution for the inverse kinematics of generic robotic manipulators

Contact Info

Product

Resources

About