Monotone Approximation by Quadratic Neural Network of Functions in Lp Spaces for p<1

Abstract

Some researchers are interested in using the flexible and applicable properties of quadratic functions as activation functions for FNNs. We study the essential approximation rate of any Lebesgue-integrable monotone function by a neural network of quadratic activation functions. The simultaneous degree of essential approximation is also studied. Both estimates are proved to be within the second order of modulus of smoothness.

Generalized Jacobian Weighted Modulus of Smoothness

“…64, No. 1, pp: 294-303 Many versions of moduli of smoothness were defined later, for more details see [5], [6], [7] , [8], [9], [10] , [11] , and [12] . In 2007, Jianjun [13] defined the weighted modulus in terms of the classical Jacobi weights 𝜔 𝛼,𝛽 (𝑥) = (1 − 𝑥) 𝛼 (1 + 𝑥) 𝛽 , (1.1) where ∈ [−1,1] , 𝛼 and 𝛽 ∈ 𝐽 𝑝 , 𝐽 𝑝 is given by…”