Sub‐Weibull distributions: Generalizing sub‐Gaussian and sub‐Exponential properties to heavier tailed distributions

Vladimirova, Mariia; Girard, Sabine; Nguyen, Hien D.; Arbel, Julyan

doi:10.1002/sta4.318

Cited by 45 publications

(61 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature [31,33], a random variable is often called sub-Gaussian, sub-exponential, or sub-Weibull with tail parameter (1/α), if X ψ2 ≤ C, X ψ1 ≤ C, and X ψα ≤ C, respectively. The matrix spectral norm is defined as X = sup u,v u Xv u 2 v 2 .…”

Section: Resultsmentioning

confidence: 99%

On the non-asymptotic concentration of heteroskedastic Wishart-type matrix

Cai

Han

Zhang

2022

Electron. J. Probab.

View full text Add to dashboard Cite

This paper focuses on the non-asymptotic concentration of the heteroskedastic Wisharttype matrices. Suppose Z is a p1-by-p2 random matrix and Zij ∼ N (0, σ 2 ij ) independently, we prove the expected spectral norm of Wishart matrix deviations (i.e., E ZZ − EZZ ) is upper bounded bywhereA minimax lower bound is developed that matches this upper bound. Then, we derive the concentration inequalities, moments, and tail bounds for the heteroskedastic Wisharttype matrix under more general distributions, such as sub-Gaussian and heavy-tailed distributions. Next, we consider the cases where Z has homoskedastic columns or rows (i.e., σij ≈ σi or σij ≈ σj) and derive the rate-optimal Wishart-type concentration bounds. Finally, we apply the developed tools to identify the sharp signal-to-noise ratio threshold for consistent clustering in the heteroskedastic clustering problem.

show abstract

Section: Resultsmentioning

confidence: 99%

On the non-asymptotic concentration of heteroskedastic Wishart-type matrix

Cai

Han

Zhang

2022

Electron. J. Probab.

View full text Add to dashboard Cite

show abstract

“…Let g(x)=e x θ −1 and E[g(|X|/η)]≤1 implies E[exp(|X| θ /η θ )]≤2, which is the definition of sub-Weibull norm. Similar to sub-exponential,[85,89,96] attained the following.…”

mentioning

confidence: 77%

“…The proof can be found in [85,89] by mimicking the proof of [84, Proposition 2.5.2]. It follows from Corollary 6.1(4) that X is sub-Weibull with tail parameter θ if and only if |X| 1/θ is sub-exponential.…”

Section: Corollary 61 (Characterizations Of Sub-weibull Condition)mentioning

confidence: 92%

Concentration Inequalities for Statistical Inference

Chen¹

2021

CMR

View full text Add to dashboard Cite

This paper gives a review of concentration inequalities which are widely employed in non-asymptotical analyses of mathematical statistics in a wide range of settings, from distribution-free to distribution-dependent, from sub-Gaussian to sub-exponential, sub-Gamma, and sub-Weibull random variables, and from the mean to the maximum concentration. This review provides results in these settings with some fresh new results. Given the increasing popularity of high-dimensional data and inference, results in the context of high-dimensional linear and Poisson regressions are also provided. We aim to illustrate the concentration inequalities with known constants and to improve existing bounds with sharper constants.

show abstract

“…In our analysis, we associate the gradient error with a class of potentially heavy tailed probability distributions. As in [52], we consider class of sub-Weibull random variables as defined next.…”

Section: Stochastic Projected Primal-dual Methodsmentioning

confidence: 99%

“…Empirical and theoretical results have demonstrated that heavier tailed distributions arise naturally in deep learning. The class of sub-Weibull random variables subsumes the sub-Guassian and sub-Exponential classes of distributions, while also including heavier tailed distributions [30,52,55]. It also includes random variables whose distribution has a bounded support.…”

Section: Related Workmentioning

confidence: 99%

Stochastic Saddle Point Problems with Decision-Dependent Distributions

Wood¹,

Dall’Anese²

2022

Preprint

View full text Add to dashboard Cite

This paper focuses on stochastic saddle point problems with decision-dependent distributions in both the static and time-varying settings. These are problems whose objective is the expected value of a stochastic payoff function, where random variables are drawn from a distribution induced by a distributional map. For general distributional maps, the problem of finding saddle points is in general computationally burdensome, even if the distribution is known. To enable a tractable solution approach, we introduce the notion of equilibrium points -which are saddle points for the stationary stochastic minimax problem that they induce -and provide conditions for their existence and uniqueness. We demonstrate that the distance between the two classes of solutions is bounded provided that the objective has a strongly-convex-strongly-concave payoff and Lipschitz continuous distributional map. We develop deterministic and stochastic primal-dual algorithms and demonstrate their convergence to the equilibrium point. In particular, by modeling errors emerging from a stochastic gradient estimator as sub-Weibull random variables, we provide error bounds in expectation and in high probability that hold for each iteration; moreover, we show convergence to a neighborhood in expectation and almost surely. Finally, we investigate a condition on the distributional map-which we call opposing mixture dominance-that ensures the objective is strongly-convex-strongly-concave. Under this assumption, we show that primal-dual algorithms converge to the saddle points in a similar fashion.

show abstract

Sub‐Weibull distributions: Generalizing sub‐Gaussian and sub‐Exponential properties to heavier tailed distributions

Cited by 45 publications

References 12 publications

On the non-asymptotic concentration of heteroskedastic Wishart-type matrix

On the non-asymptotic concentration of heteroskedastic Wishart-type matrix

Concentration Inequalities for Statistical Inference

Stochastic Saddle Point Problems with Decision-Dependent Distributions

Contact Info

Product

Resources

About