Nonlinear backpropagation: doing backpropagation without derivatives of the activation function

Hertz, John; Krogh, Anders; Lautrup, B.; Lehmann, Torsten

doi:10.1109/72.641455

Cited by 23 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training process of the NICE model can be accomplished by using the back propagation algorithm which mainly includes two stages: incentive propagation and weight update [33]. As shown in Fig.…”

Section: B the Training Process Of The Nicementioning

confidence: 99%

Modeling Daily Load Profiles of Distribution Network for Scenario Generation Using Flow-Based Generative Network

Liao

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The daily load profiles modeling is of great significance for the economic operation and stability analysis of the distribution network. In this paper, a flow-based generative network is proposed to model daily load profiles of the distribution network. Firstly, the real samples are used to train a series of reversible functions that map the probability distribution of real samples to the prior distribution. Then, the new daily load profiles are generated by taking the random number obeying the Gaussian distribution as the input data of these reversible functions. Compared with existing methods such as explicit density models, the proposed approach does not need to assume the probability distribution of real samples, and can be used to model different loads only by adjusting the structure and parameters. The simulation results show that the proposed approach not only fits the probability distribution of real samples well, but also accurately captures the spatial-temporal correlation of daily load profiles. The daily load profiles with specific characteristics can be obtained by simply classification.INDEX TERMS Daily load profiles, distribution network, generative network. BIRGITTE BAK-JENSEN (Senior Member, IEEE) received the M.Sc. degree in electrical engineering and the Ph.D. degree in modeling of high voltage components from the

show abstract

Section: B the Training Process Of The Nicementioning

confidence: 99%

Modeling Daily Load Profiles of Distribution Network for Scenario Generation Using Flow-Based Generative Network

Liao

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Firstly, the loss function of the discriminator is calculated after the samples are input into the discriminator. Then, the parameters of the discriminator are updated by the back propagation algorithm [27]. The loss function of the discriminator is:…”

Section: A Traditional Ganmentioning

confidence: 99%

Robust Voltage Control Considering Uncertainties of Renewable Energies and Loads via Improved Generative Adversarial Network

Zhao

Liao

Wang

et al. 2020

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

The fluctuation of output power of renewable energies and loads brings challenges to the scheduling and operation of the distribution network. In this paper, a robust voltage control model is proposed to cope with the uncertainties of renewable energies and loads based on an improved generative adversarial network (IGAN). Firstly, both real and predicted data are used to train the IGAN consisting of a discriminator and a generator. The noises sampled from the Gaussian distribution are fed to the generator to generate a large number of scenarios that are utilized for robust voltage control after scenario reduction. Then, a new improved wolf pack algorithm (IWPA) is presented to solve the formulated robust voltage control model, since the accuracy of the solutions obtained by traditional methods is limited. The simulation results show that the IGAN can accurately capture the probability distribution characteristics and dynamic nonlinear characteristics of renewable energies and loads, which makes the scenarios generated by IGAN more suitable for robust voltage control than those generated by traditional methods. Furthermore, IWPA has a better performance than traditional methods in terms of convergence speed, accuracy, and stability for robust voltage control.

show abstract

“…By contrast, analog hardware is ideal for implementing continuous-time dynamics such as those of leaky integrator neurons. Previous work have proposed such implementations [Hertz et al, 1997].…”

Section: Possible Implementation On Analog Hardwarementioning

confidence: 99%

“…Other alternatives to recurrent back-propagation in the framework of fixed point recurrent networks were proposed by O'Reilly [1996] and Hertz et al [1997]. Their algorithms are called 'Generalized Recirculation algorithm' (or 'GeneRec' for short) and 'Non-Linear Back-propagation', respectively.…”

Section: Related Workmentioning

confidence: 99%

Generalization of Equilibrium Propagation to Vector Field Dynamics

Scellier¹,

Goyal²,

Binas³

et al. 2018

Preprint

View full text Add to dashboard Cite

The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists. Two major reasons are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections. We present a simple two-phase learning procedure for fixed point recurrent networks that addresses both these issues. In our model, neurons perform leaky integration and synaptic weights are updated through a local mechanism. Our learning method generalizes Equilibrium Propagation to vector field dynamics, relaxing the requirement of an energy function. As a consequence of this generalization, the algorithm does not compute the true gradient of the objective function, but rather approximates it at a precision which is proven to be directly related to the degree of symmetry of the feedforward and feedback weights. We show experimentally that our algorithm optimizes the objective function.

show abstract

Nonlinear backpropagation: doing backpropagation without derivatives of the activation function

Cited by 23 publications

References 11 publications

Modeling Daily Load Profiles of Distribution Network for Scenario Generation Using Flow-Based Generative Network

Modeling Daily Load Profiles of Distribution Network for Scenario Generation Using Flow-Based Generative Network

Robust Voltage Control Considering Uncertainties of Renewable Energies and Loads via Improved Generative Adversarial Network

Generalization of Equilibrium Propagation to Vector Field Dynamics

Contact Info

Product

Resources

About