A hybrid linear/nonlinear training algorithm for feedforward neural networks

Abstract

This paper presents a new hybrid optimization strategy for training feedforward neural networks. The algorithm combines gradient-based optimization of nonlinear weights with singular value decomposition (SVD) computation of linear weights in one integrated routine. It is described for the multilayer perceptron (MLP) and radial basis function (RBF) networks and then extended to the local model network (LMN), a new feedforward structure in which a global nonlinear model is constructed from a set of locally valid submodels. Simulation results are presented demonstrating the superiority of the new hybrid training scheme compared to second-order gradient methods. It is particularly effective for the LMN architecture where the linear to nonlinear parameter ratio is large.

Small-Signal Refinement of Power System Static Load Modelling Techniques

“…In all cases active power measurements are recorded for applied voltages in the range -12% to +14% of the nominal voltage of 230 V in increments of 2%. The MLP networks are trained using the BFGS training algorithm [10] with leave-one-out cross validation used to optimize the number of neurons, ℎ . The results obtained for the three loads are presented in rows (a), (b) and (c) of Fig.…”

Direct and Indirect Classification of High-Frequency LNA Performance using Machine Learning Techniques

“…There are various configurations and structures of NNs, but all contain an array of neurons that are linked together, usually in multiple layers. In this application a single hidden layer Multilayer Perceptron (MLP) topology is chosen because of its universal function approximation capabilities, good generalisation properties and the availability of robust efficient training algorithms [12]. The output of a single hidden layer MLP can be written as a linear combination of sigmoid functions (i.e.…”

“…MLP training was performed using the hybrid BFGS training algorithm [12] with stopped minimisation used to prevent over-fitting [13]. The optimum number of neurons (M ) was determined for each model by systematically evaluating different network sizes and selecting the network with the minimum MSE on the test data set.…”

Research of Feedback Correction Method Based on Neural Network

“…The inevitable model errors will definitely lower the control accuracy and robust performance of control system, may even lead to system instability [1]. Therefore, it is not only need to improve the accuracy of prediction model but also need to take additional necessary measures to supplement the lack of model predictions.…”