Linear Regression for Heavy Tails

Balkema, Guus; Embrechts, Paul

doi:10.3390/risks6030093

Cited by 8 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An empirical study conducted by Balkema and Embrechts [15] aimed to assess the performance of various estimators in a simple linear regression problem with deterministic and stochastic components, both modelled as potentially heavy tailed random variables. Balkema and Embrechts showed that under the condition that the deterministic component is of finite variance itself, the least absolute deviation (i.e.…”

Section: On the Validity Of Inferring From The Resultsmentioning

confidence: 99%

“…Balkema and Embrechts showed that under the condition that the deterministic component is of finite variance itself, the least absolute deviation (i.e. the minimisation of the mean absolute error (MAE)) is typically a good estimator for the linear regression parameters, even if the stochastic component has infinite variance [15]. In the context of stochastic gradient descent, the minimisation of MAE is equivalent to the least absolute deviation.…”

Section: On the Validity Of Inferring From The Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Cauchy Loss Function: Robustness Under Gaussian and Cauchy Noise

Thamsanqa¹,

Deventer²,

Bosman³

2023

Preprint

View full text Add to dashboard Cite

In supervised machine learning, the choice of loss function implicitly assumes a particular noise distribution over the data. For example, the frequently used mean squared error (MSE) loss assumes a Gaussian noise distribution. The choice of loss function during training and testing affects the performance of artificial neural networks (ANNs). It is known that MSE may yield substandard performance in the presence of outliers. The Cauchy loss function (CLF) assumes a Cauchy noise distribution, and is therefore potentially better suited for data with outliers. This papers aims to determine the extent of robustness and generalisability of the CLF as compared to MSE. CLF and MSE are assessed on a few handcrafted regression problems, and a real-world regression problem with artificially simulated outliers, in the context of ANN training. CLF yielded results that were either comparable to or better than the results yielded by MSE, with a few notable exceptions.

show abstract

Section: On the Validity Of Inferring From The Resultsmentioning

confidence: 99%

Section: On the Validity Of Inferring From The Resultsmentioning

confidence: 99%

Cauchy Loss Function: Robustness Under Gaussian and Cauchy Noise

Thamsanqa¹,

Deventer²,

Bosman³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Ninth, more broadly, Balkema and Embrechts (2018) provides a review of a substantial literature on robust estimation and heavy tailed data, as well as comparing procedures using Monte Carlo methods. Related work is Kurz- Kim and Loretan (2014).…”

Section: Relating β To Other Estimatorsmentioning

confidence: 99%

An estimator for predictive regression: reliable inference for financial economics

Shephard

2020

Preprint

View full text Add to dashboard Cite

Estimating linear regression using least squares and reporting robust standard errors is very common in financial economics, and indeed, much of the social sciences and elsewhere. For thick tailed predictors under heteroskedasticity this recipe for inference performs poorly, sometimes dramatically so. Here, we develop an alternative approach which delivers an unbiased, consistent and asymptotically normal estimator so long as the means of the outcome and predictors are finite. The new method has standard errors under heteroskedasticity which are easy to reliably estimate and tests which are close to their nominal size. The procedure works well in simulations and in an empirical exercise. An extension is given to quantile regression.

show abstract

“…However, for distributions with infinite mean and/or variance a more powerful approach is needed. We are grateful to an anonymous reviewer for suggesting a comparison with the Theil-Sen estimator, used exclusively to determine linear trends [47]. Its potential limitation is computation time for large data sets.…”

Section: Q Performs Well In Heavy-tailed Noisementioning

confidence: 99%

Universal Rank-Order Transform to Extract Signals from Noisy Data

Ierley

Kostinski

2019

Phys. Rev. X

View full text Add to dashboard Cite

We introduce an ordinate method for noisy data analysis, based solely on rank information and thus insensitive to outliers. The method is nonparametric, objective, and the required data processing is parsimonious. Main ingredients are a rank-order data matrix and its transform to a stable form, which provide linear trends in excellent agreement with least squares regression, despite the loss of magnitude information. A group symmetry orthogonal decomposition of the 2D rank-order transform for iid (white) noise is further ordered by principal component analysis. This two-step procedure provides a noise "etalon" used to characterize arbitrary stationary stochastic processes. The method readily distinguishes both the Ornstein-Uhlenbeck process and chaos generated by the logistic map from white noise. Ranking within randomness differs fundamentally from that in deterministic chaos and signals, thus forming the basis for signal detection. To further illustrate the breadth of applications, we apply this ordinate method to the canonical nonlinear parameter estimation problem of two-species radioactive decay, outperforming special-purpose least square software. It is demonstrated that the method excels when extracting trends in heavy-tailed noise and, unlike the Thiele-Sen estimator, is not limited to linear regression. Lastly, a simple expression is given that yields a close approximation for signal extraction of an underlying generally nonlinear signal. arXiv:1906.08729v1 [physics.data-an]

show abstract

Linear Regression for Heavy Tails

Cited by 8 publications

References 25 publications

Cauchy Loss Function: Robustness Under Gaussian and Cauchy Noise

Cauchy Loss Function: Robustness Under Gaussian and Cauchy Noise

An estimator for predictive regression: reliable inference for financial economics

Universal Rank-Order Transform to Extract Signals from Noisy Data

Contact Info

Product

Resources

About