Malliavin–Stein method: a survey of some recent developments

Abstract

Initiated around the year 2007, the Malliavin-Stein approach to probabilistic approximations combines Stein's method with infinite-dimensional integration by parts formulae based on the use of Malliavin-type operators. In the last decade, Malliavin-Stein techniques have allowed researchers to establish new quantitative limit theorems in a variety of domains of theoretical and applied stochastic analysis. The aim of this survey is to illustrate some of the latest developments of the Malliavin-Stein method, with specific emphasis on extensions and generalizations in the framework of Markov semigroups and of random point measures.

A quantitative functional central limit theorem for shallow neural networks

“…The statement of Theorem 1 is given in order to cover the most general activation functions, allowing for possibly nondifferentiable choices such as the ReLu. Under stronger conditions, the result can be improved; in particular, assuming the activation function has a Malliavin derivative with bounded fourth moment (i.e., it belongs to the class D 1,4 , see [25,8]), we obtain the following extension.…”

“…We consider in this work functional quantitative central limit theorems under general activations and for coefficients that are Gaussian for both layers, which seems the most relevant case for applications; our approach is largely based upon very recent results by [8] on the Stein-Malliavin techniques for random elements taking values in Hilbert spaces (we refer to [24,25] for the general foundations of this approach, together with [20,7,1,12] for some more recent references). Our main results are collected in Section 2, whereas their proofs with a few technical lemmas are given in Section 4.…”

“…V j σ (W j x), (1) where V j ∈ R, W j ∈ R 1×d are, respectively, random variables and vectors, whose entries are independent Gaussian with zero mean and variance E[V 2 j ] = E[W 2 j ] = 1, j = 1, . .…”

“…Neural networks are usually optimized by (variants of) gradient descent with a random initial condition, hence, at initialization, they can be seen as random fields. In many applications, the Gaussian is in fact the distribution of choice, which leads (in the shallow case) to the model considered in (1). On the other hand, taking large, over-parametrized architectures has become an established practice, achieving impressive empirical performance in spite of classical statistical knowledge that would warn from the risks of overfitting [30,4].…”

Optimal Gamma Approximation on Wiener Space

“…Unlike for the normal or the Gamma distribution, in the context of the Malliavin-Stein method this results in the appearance of the higher-order iterated Gamma operator Γ alt,2 , see Definition 20 below. Due to the involved nature of the Γ alt,2 operator, up to this date only the case of the second Wiener chaos has been treated successfully, since in this case the general Malliavin-Stein bound could be translated into the language of finitely many cumulants, see for example [AP17,Conjecture 6.8]. Let Y ∼ V G c (r, θ, σ) and assume that for each n 1, F n ∈ H 2 is an element in the second Wiener chaos.…”