Features and performance of some outlier detection methods

Abstract

A review of several statistical methods that are currently in use for outlier identification is presented, and their performances are compared theoretically for typical statistical distributions of experimental data, considering values derived from the distribution of extreme order statistics as reference terms. A simple modification of a popular, broadly used method based upon box-plot is introduced, in order to overcome a major limitation concerning sample size. Examples are presented concerning exploitation of methods considered on two data sets: a historical one concerning evaluation of an astronomical constant performed by a number of leading observatories and a substantial database pertaining to an ongoing investigation on absolute measurement of gravity acceleration, exhibiting peculiar aspects concerning outliers. Some problems related to outlier treatment are examined, and the requirement of both statistical analysis and expert opinion for proper outlier management is underlined.exclusion rules, order statistics, outliers, robust statistics, statistical test,

Measurement of elastic modulus by instrumented indentation in the macro-range: Uncertainty evaluation

“…Therefore, the indentation size effect results to be negligible in the investigated macro hardness range [9]. In conclusion, it is possible to estimate the measurement reproducibility of the indentation modulus according the three methods for evaluating S. In case of LEM and authors' methods, once identified and eliminated outliers [31], the standard deviations of the values of E IT are taken as measurement reproducibility; these values are respectively 6.74 kN/mm 2 and 7.14 kN/mm 2 . Instead, in case of PLM, it is considered the standard deviation of residuals with respect to the model in Eq.…”

Building High-Quality Auction Fraud Dataset

“…• Based on the sample mean and standard variance, IQR is less vulnerable to extreme values than other filtering methods because it uses quartiles that are resistant to extreme values (Walfish, 2006); (Barbato, Barini, Genta & Levi, 2011).…”

Treatment of Experimental Data with Discordant Observations: Issues in Empirical Identification of Distribution

“…Evidence of rather abusive exclusion of outliers appears for both IQR and Chauvenet's methods [16], suggesting an empirical distribution shape radically deviating from the original data set. On the other hand, by retaining a few borderline experimental values, modified IQR, Grubbs' and the extreme values method manage to preserve some typical features of the original data set, while at the same time effectively shielding estimates of main population parameters -central tendency, and scatter -from outlier induced bias.…”

“…Fig.3 shows normal probability plots of values of coded gravity acceleration; at either tail a number of discordant results may be observed, mainly traceable to such anomalies as referred to above. Outliers are identified according to Chauvenet's, Grubbs', and IQR (interquartile range) methods; another based upon extreme value distribution [15], and a modification of IQR method were also considered [16], see also [2], [17].…”