This article concerns the expressive power of depth in neural nets with ReLU activations and bounded width. We are particularly interested in the following questions: what is the minimal width wmin(d) so that ReLU nets of width wmin(d) (and arbitrary depth) can approximate any continuous function on the unit cube [0, 1] d aribitrarily well? For ReLU nets near this minimal width, what can one say about the depth necessary to approximate a given function? We obtain an essentially complete answer to these questions for convex functions. Our approach is based on the observation that, due to the convexity of the ReLU activation, ReLU nets are particularly well-suited for representing convex functions. In particular, we prove that ReLU nets with width d + 1 can approximate any continuous convex function of d variables arbitrarily well. These results then give quantitative depth estimates for the rate of approximation of any continuous scalar function on the d-dimensional cube [0, 1] d by ReLU nets with width d + 3.

Abstract. Let (M, g) be a compact smooth Riemannian manifold. We obtain new off-diagonal estimates as λ → ∞ for the remainder in the pointwise Weyl Law for the kernel of the spectral projector of the Laplacian onto functions with frequency at most λ. A corollary is that, when rescaled around a non self-focal point, the kernel of the spectral projector onto the frequency interval (λ, λ + 1] has a universal scaling limit as λ → ∞ (depending only on the dimension of M ). Our results also imply that if M has no conjuage points, then immersions of M into Euclidean space by an orthonormal basis of eigenfunctions with frequencies in (λ, λ + 1] are embeddings for all λ sufficiently large.

No abstract