Heavy-Tailed Distributions: Data, Diagnostics, and New Developments

Abstract

This monograph is written for the numerate nonspecialist, and hopes to serve three purposes. First it gathers mathematical material from diverse but related fields of order statistics, records, extreme value theory, majorization, regular variation and subexponentiality. All of these are relevant for understanding fat tails, but they are not, to our knowledge, brought together in a single source for the target readership. Proofs that give insight are included, but for fussy calculations the reader is referred to the excellent sources referenced in the text. Multivariate extremes are not treated. This allows us to present material spread over hundreds of pages in specialist texts in twenty pages. Chapter 5 develops new material on heavy tail diagnostics and gives more mathematical detail.Second, it presents a new measure of obesity. The most popular definitions in terms of regular variation and subexponentiality invoke putative properties that hold at infinity, and this complicates any empirical estimate. Each definition captures some but not all of the intuitions associated with tail heaviness. Chapter 5 studies two candidate indices of tail heaviness based on the tendency of the mean excess plot to collapse as data are aggregated. The probability that the largest value is more than twice the second largest has intuitive appeal but its estimator has very poor accuracy. The Obesity index is defined for a positive random variable X as:For empirical distributions, obesity is defined by bootstrapping. This index reasonably captures intuitions of tail heaviness. Among its properties, if α > 1 then Ob(X) < Ob(X α ). However, it does not completely mimic the tail index of regularly varying distributions, or the extreme value index. A Weibull distribution with shape 1/4 is more obese than a Pareto distribution with tail index 1, even though this Pareto has infinite mean and the Weibull's moments are all finite. Chapter 5 explores properties of the Obesity index.Third and most important, we hope to convince the reader that fat tail phenomena pose real problems; they are really out there and they seriously challenge our usual ways of thinking about historical averages, outliers, trends, regression coefficients and confidence bounds among many other things. Data on flood insurance claims, crop loss claims, hospital discharge bills, precipitation and damages and fatalities from natural catastrophes drive this point home.

Connected we stand: A network perspective on trade and global food security

“…6 The extent to which the tails of the distribution have become heavier over time is also investigated. We calculate two indexes of tail heaviness commonly used in the literature (Table 1): the first is the percentage of observations that stay out of the interval "mean  double standard deviation", while the second is the Obesity Index proposed by Cooke et al (2014). 7 The latter is computed as ( ) = ( 1 + 4 > 2 + 3 | 1 ≤ 2 ≤ 3 ≤ 4 ), and is based on the heuristic that, in the case of heavy-tailed distributions, larger observations lie further apart than smaller observations.…”

“…Yet the likelihood of extreme events diminishes slightly over the period analyzed. A further method to look for fat tails in a distribution is to compare the plot of the mean excess function (MEF) of the data with the MEF plot obtained through aggregation of the original data set by m (as suggested by Cooke et al 2014). The mean excess function of a random variable X gives the expected excess of the random variable over a certain threshold u, given that the random variable is larger than the threshold.…”

Critical values of driver response time and its impact on reducing reliability and safety in road traffic

“…An example of applying this distribution to describe a parameter in a model of fatigue crack growth can be found in Doliński [12]. Log-normal distribution applications associated with the problems of the reliability theory are offered in the works [23,10,28,33], and for the extreme value theory applied to warning forecasts in hydrology they are found in the work [26]. Basic characteristics of the LN distribution are provided in Table 1 [29].T …”

Virtual Water Trade and Country Vulnerability: A Network Perspective

“…Alongside some ordinary descriptive statistics (i.e. kurtosis and skewness), we compute the percentage of observations that stay out of the interval "mean±double standard deviation" and the Obesity Index -ObIn (Cooke & Nieboer, 2011). 14 The latter is based on the heuristic that, in the case of heavy-tailed distributions, larger observations lie further apart than smaller observations, and it is computed as follows:…”

“…The skewness is positive and small, indicating rather symmetric distributions, with a slightly longer right tail. The kurtosis is larger than the reference value for a normal distribution (3) A further method to look for fat tails in a distribution is to compare the mean excess plot of the data sample with the mean excess plot of a data set obtained through aggregating the original data set by k, as suggested by Cooke & Nieboer (2011). The mean excess function -MEF -of a random variable X gives the expected excess of the random variable over a certain threshold u, given that the random variable is larger than the threshold.…”