Robust estimation of the Pareto tail index: a Monte Carlo analysis

Abstract

The Pareto distribution is often used in many areas of economics to model the right tail of heavy-tailed distributions. However, the standard method of estimating the shape parameter (the Pareto tail index) of this distribution-the maximum likelihood estimator (MLE), also known as the Hill estimator-is non-robust, in the sense that it is very sensitive to extreme observations, data contamination or model deviation. In recent years, a number of robust estimators for the Pareto tail index have been proposed, which correct the deficiency of the MLE. However, little is known about the performance of these estimators in small-sample setting, which often occurs in practice. This paper investigates the small-sample properties of the most popular robust estimators for the Pareto tail index, including the optimal B-robust estimator (Victoria-Feser and Ronchetti in Can J Stat 22:247-258, 1994), the weighted maximum likelihood estimator (Dupuis and Victoria-Feser in Can J Stat 34:639-658, 2006), the generalized median estimator (Brazauskas and Serfling in Extremes 3:231-249, 2001), the partial density component estimator (Vandewalle et al. in Comput Stat Data Anal 51:6252-6268, 2007), and the probability integral transform statistic estimator (PITSE) (Finkelstein et al. in N Am Actuar J 10:1-10, 2006). Monte Carlo simulations show that the PITSE offers the desired compromise between ease of use and power to protect against outliers in the small-sample setting.

Data-adaptive trimming of the Hill estimator and detection of outliers in the extremes of heavy-tailed data

“…In fact, the resulting estimator ξ k 0 ,k (n), where k 0 is automatically selected, has an excellent finite sample performance and it is adaptively robust. This novel adaptive robustness property is not present in other robust estimators of [20,23,28,13,31,14], which involve hard to select tuning parameters. Also none of these estimators is able to identify outliers in the extremes, a property inherent to the adaptive trimmed Hill estimator.…”

Conditional Income Disparity between Farm and Non‐farm Households in the European Union: A Longitudinal Analysis

“…Moreover, to overcome the problem of outliers and influential observations, we recalibrate sample weights following the approach proposed by Alfons et al . (2013) and generally adopted by those working with income variables (Alfons and Templ, 2013; Brzesinki, 2016; Jenkins, 2017; Safari et al ., 2018, 2019; Templ et al ., 2019). This procedure consists of detecting outlier observations against a fitted Pareto distribution of the variable of interest, applying Van Kerm’s rule of thumb to determine the threshold (Van Kerm, 2007).…”

Mind the Income Gap: Behavior of Inequality Estimators from Complex Survey Small Samples

“…However, those data have been treated in order to circumvent non-robustness problems. The issue of robust estimation of economic indicators based on a semi-parametric Pareto upper tail model is well-established in literature see [Brzezinski(2016)] for a review and [Alfons et al(2013)Alfons, Templ, and Filzmoser] for a specification suitable for survey data. On the contrary, the issue of robust treatment of outlier in the lower tail of income distribution appears less established, see [Van Kerm(2007)], [Masseran et al(2019)Masseran, Safari, and Ibrahim].…”

“…On the contrary, the issue of robust treatment of outlier in the lower tail of income distribution appears less established, see [Van Kerm(2007)], [Masseran et al(2019)Masseran, Safari, and Ibrahim]. As regards the upper tail we operated a semiparametric Pareto-tail modeling procedure using the Probability Integral Transform Statistic Estimator (PITSE) proposed by [Finkelstein et al(2006)Finkelstein, Tucker, and Alan Veeh], which blends very good performances in small samples and a fast computational implementation, as suggested by [Brzezinski(2016)]. As regards the lower tail extreme value treatment, we used an inverse Pareto modification of PITSE estimator, suggested by [Masseran et al(2019)Masseran, Safari, and Ibrahim].…”