A critical overview of outlier detection methods

Smiti, Abir

doi:10.1016/j.cosrev.2020.100306

Cited by 207 publications

(79 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The order of the tests was randomized to reduce the chance of bias in the results that could have occurred due to differences in materials or experimental conditions. However, to improve the accuracy of the models and based on the residual plots, a total of 10 runs were identified as outliers and were eliminated from the model [ 28 ]. Consequently, the second attempt improved the overall accuracy of the models up to 20%.…”

Section: Resultsmentioning

confidence: 99%

Preliminary Evaluation of Cement Mortars Containing Waste Silt Optimized with the Design of Experiments Method

Solouki

Viscomi²,

Tataranni

et al. 2021

Materials

View full text Add to dashboard Cite

Every year, up to 3 billion tons of non-renewable natural aggregates are demanded by the construction sector and approximately 623 million tons of waste (mining and quarrying) was produced in 2018. Global efforts have been made to reduce the number of virgin aggregates used for construction and infrastructure sectors. According to the revised waste framework directive in Europe, recycling at least 70% of construction and demolition waste materials by 2020 was obligatory for all member states. Nonetheless, quarries must work at full capacity to keep up with the demands, which has made quarry/mining waste management an important aspect during the past decades. Amongst the various recycling methods, quarry waste can be included in cement mortar mixtures. Thus, the current research focuses on producing cement mortars by partially substituting natural sand with the waste silt obtained from the limestone aggregate production in S.A.P.A.B.A. s.r.l. (Italy). A Design of Experiments (DOE) method is proposed to define the optimum mix design, aiming to include waste silt in cement mortar mixtures without affecting the final performance. Three cement mortar beams were produced and tested for each of the 49 randomized mixtures defined by the DOE method. The obtained results validate the design approach and suggest the possibility of substituting up to 20% of natural sand with waste silt in cement mortar mixtures.

show abstract

Section: Resultsmentioning

confidence: 99%

Preliminary Evaluation of Cement Mortars Containing Waste Silt Optimized with the Design of Experiments Method

Solouki

Viscomi²,

Tataranni

et al. 2021

Materials

View full text Add to dashboard Cite

show abstract

“…Non-parametric techniques usually allow fast computations and are adopted where such speed is of primary importance. Among them, we find the histogram-based approaches, which assume feature independence and determine the outliers based on the histogram distribution [35][36][37][38]. Other non-parametric approaches are bitmap time series anomaly detectors, which compute the relative frequency of its features to create a bitmap and identify anomalous time series [39,40], and statistical methods that allow estimating outliers based on a kernel density estimation by using a kernel function.…”

Section: Time Series Anomaly Detectionmentioning

confidence: 99%

Explaining Bad Forecasts in Global Time Series Models

et al. 2021

View full text Add to dashboard Cite

While increasing empirical evidence suggests that global time series forecasting models can achieve better forecasting performance than local ones, there is a research void regarding when and why the global models fail to provide a good forecast. This paper uses anomaly detection algorithms and explainable artificial intelligence (XAI) to answer when and why a forecast should not be trusted. To address this issue, a dashboard was built to inform the user regarding (i) the relevance of the features for that particular forecast, (ii) which training samples most likely influenced the forecast outcome, (iii) why the forecast is considered an outlier, and (iv) provide a range of counterfactual examples to understand value changes, in the feature vector or the predicted value, can lead to a different outcome. Moreover, a modular architecture and a methodology were developed to iteratively remove noisy data instances from the train set, to enhance the overall global time series forecasting model performance. Finally, to test the effectiveness of the proposed approach, it was validated on two publicly available real-world datasets.

show abstract

“…After the previous ltering and reclassi cation step, some non-compliant analyses on the GEOROC dataset remained. We interpreted these samples as residual outliers and, to avoid confusion on the machine learning training stage (Smiti, 2020), we removed these outliers from the prepared database.…”

Section: Outlier Removalmentioning

confidence: 99%

Qmin: A machine learning-based application for mineral chemistry data processing and analysis

Silva

Ferreira

Costa

et al. 2021

Preprint

View full text Add to dashboard Cite

Mineral chemistry analysis is a valuable tool in several phases of mineralogy and mineral prospecting studies. This type of analysis can point out relevant information, such as concentration of the chemical element of interest in the analyzed phase and, thus, the predisposition of an area for a given commodity. Due to this, considerable amount of data has been generated, especially with the use of electron probe micro-analyzers (EPMA), either in research for academic purposes or in a typical prospecting campaign in the mineral industry. We have identified an efficiency gap when manually processing and analyzing mineral chemistry data, and thus, we envisage this research niche could benefit from the versatility brought by machine learning algorithms. In this paper, we present Qmin, an application that assists in increasing the efficiency of mineral chemistry data processing and analysis stages through automated routines. Our code benefits from a hierarchical structure of classifiers and regressors trained by a Random Forest algorithm developed on a filtered training database extracted from the GEOROC (Geochemistry of Rocks of the Oceans and Continents) repository, maintained by the Max Planck Institute for Chemistry. To test the robustness of our application, we applied a blind test with more than 11,000 mineral chemistry analyses compiled for diamond prospecting within the scope of the Diamante Brasil Project of the Geological Survey of Brazil. The blind test yielded a balanced classifier accuracy of ca. 99% for the minerals known by Qmin. Therefore, we highlight the potential of machine learning techniques in assisting the processing and analysis of mineral chemistry data.

show abstract

A critical overview of outlier detection methods

Cited by 207 publications

References 5 publications

Preliminary Evaluation of Cement Mortars Containing Waste Silt Optimized with the Design of Experiments Method

Preliminary Evaluation of Cement Mortars Containing Waste Silt Optimized with the Design of Experiments Method

Explaining Bad Forecasts in Global Time Series Models

Qmin: A machine learning-based application for mineral chemistry data processing and analysis

Contact Info

Product

Resources

About