Towards a mathematical framework to inform neural network modelling via polynomial regression

NN2Poly: A polynomial representation for deep feed-forward artificial neural networks

“…The initial step (Section 3.1) of NN2Poly involves considering several single hidden NNs in a regression setting, i.e., with linear output, situation solved in Morala et al (2021). Each of these NNs is a subnet with their output unit being each one of the units in the second hidden layer of the original NN, and therefore a polynomial is obtained at each unit.…”

“…1). This situation is solved in Morala et al (2021) and the obtained solution is a PR up to degree Q 1 = q 1 , where q 1 is the degree at which the Taylor expansion is truncated when approximating the activation function at the first layer (l = 1). Therefore, the polynomial regression is:…”

“…More recently, this relation between neural networks and polynomial regression was also proposed in Cheng et al (2019), were the authors conjecture that both models are equivalent. In this context, a method to obtain a polynomial regression from a given neural network with a single hidden layer and linear output was previously proposed in Morala et al (2021). This is the initial step where this paper relies to propose NN2Poly, a new method which is an extension of the previous one to arbitrarily deep fully connected feed-forward neural networks, i.e., multilayered perceptrons (MLP).…”

“…This is the initial step where this paper relies to propose NN2Poly, a new method which is an extension of the previous one to arbitrarily deep fully connected feed-forward neural networks, i.e., multilayered perceptrons (MLP). In brief, the previous method from Morala et al (2021) is used to obtain a polynomial regression at each unit in the first layer of an arbitrarily deep multilayered perceptron, and from this point, another polynomial is built, layer by layer until the final output of the neural network, which is modelled as well by a final polynomial, or several ones in the classification case. Therefore, NN2Poly uses a direct combination of the weights of a given deep neural network, combined with a polynomial approximation of the activation function, and obtains an explicit expression for the polynomial regression's coefficients, differing here from previous proposals.…”

“…The rest of the paper is divided as follows: Section 2 introduces the notation that will be used in Section 3, which contains the proposed method NN2Poly that extends the idea from Morala et al (2021) to an arbitrarily deep neural network, starting from the single layer regression case that was already solved and extending it iteratively layer by layer. Then, Section 4 shows the implementation methodology and its main computational limitations.…”