On Stein’s identity and its applications

Kattumannil, Sudheesh K.

doi:10.1016/j.spl.2009.03.021

Cited by 36 publications

(15 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Theorem 2 (Generalized Stein's Identity [3]): Let Y be an absolutely continuous random variable. If the probability density function f Y (y) satisfies the following equations,…”

Section: Preliminary Resultsmentioning

confidence: 99%

See 1 more Smart Citation

On the equivalence between Stein and de Bruijn identities

Park¹,

Serpedin²,

Qaraqe³

2012

Preprint

View full text Add to dashboard Cite

This paper focuses on illustrating 1) the equivalence between Stein's identity and De Bruijn's identity, and 2) two extensions of De Bruijn's identity. First, it is shown that Stein's identity is equivalent to De Bruijn's identity under additive noise channels with specific conditions. Second, for arbitrary but fixed input and noise distributions under additive noise channels, the first derivative of the differential entropy is expressed by a function of the posterior mean, and the second derivative of the differential entropy is expressed in terms of a function of Fisher information. Several applications over a number of fields such as signal processing and information theory, are presented to support the usefulness of the developed results in this paper.

show abstract

“…Theorem 2 (Generalized Stein's Identity [3]): Let Y be an absolutely continuous random variable. If the probability density function f Y (y) satisfies the following equations,…”

Section: Preliminary Resultsmentioning

confidence: 99%

“…S TEIN'S identity (or lemma) was first established in 1956 [1], and since then it has been widely used by many researchers (e.g., [2], [3], [4]). Due to its applications in the James-Stein estimation technique, empirical Bayes methods, and numerous other fields, Stein's identity has attracted a lot of interest (see e.g., [5], [6], [7]).…”

Section: Introductionmentioning

confidence: 99%

On the equivalence between Stein and de Bruijn identities

Park¹,

Serpedin²,

Qaraqe³

2012

Preprint

View full text Add to dashboard Cite

show abstract

“…However, Price (1958); Opper & Archambeau (2009) use the characteristic function of Gaussian to prove Price's theorem, which is not easy to extend to the Gaussian mixture case. Hudson et al (1978); Brown (1986); Arnold et al (2001); Landsman (2006); Landsman & Nešlehová (2008); Kattumannil (2009); Kattumannil & Dewan (2016); Adcock (2007); Adcock & Shutes (2012) further extend Stein's lemma to exponential family and beyond. Unfortunately, these works neither show the connection between the gradient identity and the implicit reparameterization trick nor give any second-order identity.…”

Section: Related Workmentioning

confidence: 97%

Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures

Lin,

Khan,

Schmidt

2019

Preprint

View full text Add to dashboard Cite

Stein's method (Stein, 1973;1981) is a powerful tool for statistical applications, and has had a significant impact in machine learning. Stein's lemma plays an essential role in Stein's method. Previous applications of Stein's lemma either required strong technical assumptions or were limited to Gaussian distributions with restricted covariance structures.In this work, we extend Stein's lemma to exponential-family mixture distributions including Gaussian distributions with full covariance structures. Our generalization enables us to establish a connection between Stein's lemma and the reparamterization trick to derive gradients of expectations of a large class of functions under weak assumptions. Using this connection, we can derive many new reparameterizable gradientidentities that goes beyond the reach of existing works. For example, we give gradient identities when expectation is taken with respect to Student's t-distribution, skew Gaussian, exponentially modified Gaussian, and normal inverse Gaussian.

show abstract

“…The connection between Stein identities and information theory has already been noted in the literature (although only in dimension 1). For instance, explicit applications are known in the context of Poisson and compound Poisson approximations [7,49], and recently several promising identities have been discovered for some discrete [28,48] as well as continuous distributions [26,29,45]. However, with the exception of [29], the existing literature seems to be silent about any connection between entropic CLTs and Stein's identities for normal approximations.…”

Section: Overview and Motivationmentioning

confidence: 99%

Entropy and the fourth moment phenomenon

Nourdin¹,

Peccati²,

Swan³

2014

Journal of Functional Analysis

View full text Add to dashboard Cite

We develop a new method for bounding the relative entropy of a random vector in terms of its Stein factors. Our approach is based on a novel representation for the score function of smoothly perturbed random variables, as well as on the de Bruijn's formula of information theory. When applied to sequences of functionals of a general Gaussian field, our results can be combined with the Carbery-Wright inequality in order to yield multidimensional entropic rates of convergence that coincide, up to a logarithmic factor, with those achievable in smooth distances (such as the 1-Wasserstein distance). In particular, our findings settle the open problem of proving a quantitative version of the multidimensional fourth moment theorem for random vectors having chaotic components, with explicit rates of convergence in total variation that are independent of the order of the associated Wiener chaoses. The results proved in the present paper are outside the scope of other existing techniques, such as for instance the multidimensional Stein's method for normal approximations.

show abstract

On Stein’s identity and its applications

Cited by 36 publications

References 13 publications

On the equivalence between Stein and de Bruijn identities

On the equivalence between Stein and de Bruijn identities

Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures

Entropy and the fourth moment phenomenon

Contact Info

Product

Resources

About