A Numerical Implementation of Kolmogorov's Superpositions

Abstract

Hecht-Nielsen proposed a feedforward neural network based on Kolmogorov's superpositionsf(x(i), em leader,x(n))= summation operator q=02piPhi(q)(y(q)) that apply to all real valued continuous functions f(x(1), em leader, x(n)) defined on a Euclidean unit cube of dimension n >/= 2. This network has a hidden layer that is independent of f and that transforms the n-tuples (x(1), em leader, x(n)) into the 2n + 1 variables y(q), and an output layer in which f is computed. Kůrková has shown that such a network has an approximate implementation with arbitrary activation functions of sigmoidal type. Actual implementation is, however, impeded by the lack of numerical algorithms for the hidden layer which contain continuous functions of the formy(q)=Sigma(n)(p=1)alpha(p)psi(x(p)+qa) with constants a and alpha(p). This paper gives an explicit numerical implementation of the hidden layer that also enables the implementation of the output layer. Copyright 1996 Elsevier Science Ltd

Self-Organizing Architecture for Information Fusion in Distributed Sensor Networks

“…Besides, the MLP agent was trained taking into account the inputs for each of the sensors and with a value of −1 for the rest of the inputs. The hidden layer of the MLP is designed with 2 + 1 [41] neurons, where is the number of neurons in the input layer. The output layer is composed of three neurons that correspond with the coordinates ( , , ).…”

Transform Regression and the Kolmogorov Superposition Theorem

“…Decision-tree methods and greedy search heuristics are used to construct g i and h ij functions based on training data in an attempt to make the superposition equation a good predictor. The approach contrasts with previous work on direct application of the superposition theorem (Neruda, Štědrý, & Drkošová, 2000;Sprecher 1996Sprecher , 1997Sprecher , 2002. One difficulty with direct application is that the g i and h ij functions that need to be constructed are extremely complex and entail very large computational overheads to implement, even when the target function is known (Neruda, Štědrý, & Drkošová, 2000).…”

“…In order to avoid local minima and saddle points entirely, additional research is needed to further improve the transform regression algorithm. Several authors (Kůrková, 1991(Kůrková, , 1992Neruda, Štědrý, & Drkošová, 2000;Sprecher 1996Sprecher , 1997Sprecher , 2002 have been investigating ways of overcoming the computational problems of directly applying Kolmogorov's theorem. Given the strength of the results obtain here using the form of the superposition equation alone, research aimed at creating a combined approach could potentially be quite fruitful.…”

Some Theorems for Feed Forward Neural Networks

“…We will now describe such a process. We start with a certain number of clusters in an initial set (say) N = N 0 , belonging to the actual configuration of N = N f clusters and then choose an initial set of planes q = q 0 , to separate these N 0 , as a start 8 we will assume N 0 << 2 q 0 . We will then include more clusters into this set and at the same time choosing an additional plane (or planes) to separate the new arrivals of clusters from one another and from those which are already in this set.…”