We demonstrate the emergence of weight matrix singular value decomposition (SVD) in interpreting neural networks (NNs) for parameter estimation from noisy signals. The SVD appears naturally as a consequence of initial application of a descrambling transform -a recently-developed technique for addressing interpretability in NNs [1]. We find that within the class of noisy parameter estimation problems, the SVD may be the means by which networks memorize the signal model. We substantiate our theoretical findings with empirical evidence from both linear and non-linear settings. Our results also illuminate the connections between a mathematical theory of semantic development [2] and neural network interpretability.

No abstract

We develop a theory of Sobolev orthogonal polynomials on the Sierpiński gasket (SG). These orthogonal polynomials arise through the Gram-Schmidt orthogonalisation process applied on the set of monomials on SG using several notions of a Sobolev inner products. After establishing some recurrence relations for these orthogonal polynomials, we give estimates for their L 2 , L ∞ and Sobolev norms, and study their asymptotic behaviour.Finally, we study the properties of zero sets of polynomials and develop fast computational tools to explore applications to quadrature and interpolation. Contents 1. Introduction 2. Polynomials on SG 2.1. Analysis on SG 2.2. Polynomials on SG 2.3. Zeroes of Polynomials 3. Sobolev-Legendre Orthogonal Polynomials on SG 3.1. General properties 3.2. Sobolev Orthogonal Polynomials with respect to k = 2, 3 3.3. Sobolev Orthogonal Polynomials with respect to k = 1 3.4. Orthogonal polynomials with respect to the higher order Sobolev-type inner products 3.5. Numerical results 3.6. Implementation and Code Design