Simultaneous recurrent neural network trained with non-recurrent backpropagation algorithm for static optimisation

Abstract

This paper explores feasibility of employing the non-recurrent backpropagation training algorithm for a recurrent neural network, Simultaneous Recurrent Neural network, for static optimisation. A simplifying observation that maps the recurrent network dynamics, which is configured to operate in relaxation mode as a static optimizer, to feedforward network dynamics is leveraged to facilitate application of a non-recurrent training algorithm such as the standard backpropagation and its variants. A simulation study that aims to assess feasibility, optimizing potential, and computational efficiency of training the Simultaneous Recurrent Neural network with non-recurrent backpropagation is conducted. A comparative computational complexity analysis between the Simultaneous Recurrent Neural network trained with non-recurrent backpropagation algorithm and the same network trained with the recurrent backpropagation algorithm is performed. Simulation results demonstrate that it is feasible to apply the non-recurrent backpropagation to train the Simultaneous Recurrent Neural network. The optimality and computational complexity analysis fails to demonstrate any advantage on behalf of the non-recurrent backpropagation versus the recurrent backpropagation for the optimisation problem considered. However, considerable future potential that is yet to be explored exists given that computationally efficient versions of the backpropagation training algorithm, namely quasiNewton and conjugate gradient descent among others, are also applicable for the neural network proposed for static optimisation in this paper.

Training Simultaneous Recurrent Neural Network With Resilient Propagation for Static Optimization

“…13 Therefore, the weight parameters were initialized to the values shown in Table 1 for the SRN trained using the resilient propagation training algorithm as well. More specifically, constraint weight parameters needed to be increased following a set schedule, which turned out to be critical for the SRN to locate a solution, which concurs with the findings of previous studies 13,15 In order to encourage the SRN algorithm to locate higher quality solutions, the increment value of the constraint weight parameter for the distance constraint was specified as large as possible compared to values for the remaining parameters while ensuring that the network converges to valid solutions. More specifically, constraint weight parameters needed to be increased following a set schedule, which turned out to be critical for the SRN to locate a solution, which concurs with the findings of previous studies 13,15 In order to encourage the SRN algorithm to locate higher quality solutions, the increment value of the constraint weight parameter for the distance constraint was specified as large as possible compared to values for the remaining parameters while ensuring that the network converges to valid solutions.…”

Optimization of composition of as-cast chromium white cast iron based on wear-resistant performance

“…It is well-known that artificial neural network (ANN) can extract usable information from large quantity of discrete data with noise and is usable for resolving the highly non-linear and uncertain problems [14][15][16][17][18][19]. In general, there are two kinds of ANN, i.e., back-propagation artificial neural network (BPANN) and radial basis function artificial neural network (RBFANN), for being used to establish the model of the prediction.…”

Optimization of composition and technology for phosphate graphite mold

“…In addition, the Fuzzy evaluation can obtain better multi-objective evaluation, but it cannot be used to predict the performances of other tests. However, it is well-known that artificial neural network ANN can extract usable information from large quantity of discrete data with noise and is usable for resolving the highly non-linear and uncertain problems [20,21]. And then, we established three ANN models with four inputs (phosphoric acid, Al 2 O 3 , drying temperature and drying time) and one (either compression strength or tension strength) or two outputs (compression strength and tension strength) in Table 2.…”