Constraining the topology of neural networks to ensure dynamics with symmetry properties

Abstract

This paper addresses the training of network models from data produced by systems with symmetry properties. It is argued that although general networks are global approximators, in practice some properties such as symmetry are very hard to learn from data. In order to guarantee that the final network will be symmetrical, constraints are developed for two types of models, namely, the multilayer perceptron (MLP) network and the radial basis function (RBF) network. In global modeling problems it becomes crucial to impose conditions for symmetry in order to stand a chance of reproducing symmetry-related phenomena. Sufficient conditions are given for MLP and RBF networks to have a set of fixed points that are symmetrical with respect to the origin of the phase space. In the case of MLP networks such conditions reduce to the absence of bias parameters and the requirement of odd activation functions. This turns out to be important from a dynamical point of view since some phenomena are only observed in the context of symmetry, which is not a structurally stable property. The results are illustrated using bench systems that display symmetry, such as the Duffing-Ueda oscillator and the Lorenz system.

Symmetric RBF Classifier for Nonlinear Detection in Multiple-Antenna-Aided Systems

“…The use of prior knowledge in data modeling often leads to an improved performance. In regression-type applications, the symmetric properties of the underlying system have been exploited by imposing symmetry in both RBF networks and least squares support vector machines (SVMs) [23], [24]. An important message from these two studies is worth revisiting.…”

Symmetric Kernel Detector for Multiple-Antenna Aided Beamforming Systems

“…However, by imposing symmetry on the model's structure, exploiting the symmetry properties becomes easier and this often leads to substantial improvements in the achievable modelling performance. In regression-type applications, the symmetric properties of the underlying system have been exploited by imposing symmetry in both RBF networks and least squares support vector machines (SVMs) [17], [18].…”

Model selection approaches for non-linear system identification: a review

“…While both linear and non-linear ARX models gave comparable one step ahead, short-term prediction performances, only the grey-box model was capable of open-loop predictions of NO x emissions spanning several weeks. It has also been reported that the inherent symmetry properties of the system can be incorporated into the linear-in-the-parameter models to improve the performance (Aguirre, Lopes, Amaral, and Letellier 2004;Espinoza, Suykens, and De Moor 2005a;Chen, Wolfgang, Harris, and Hanzon 2007), system eigenvalues can be used to choose the model types (Aguirre, Coelho, and Correa 2005), and simple system a priori information like steady-state relations of variables has been used in the identification of nonlinear models for a Buck Converter (Aguirre, DonosoGauia, and Santos-Filho 2000). System specific neural network models have also been studied ) with application to power plants and chemical processes, and the activation functions in the network are system specific, aiming to improve the model interpretability and generalisation performance (Connally et al 2005(Connally et al , 2007.…”