Distribution Forest: An Anomaly Detection Method Based on Isolation Forest

Yao, Chengfei; Ma, Xiaojun; Chen, Biao; Zhao, Xiaodan; Bai, Gang

doi:10.1007/978-3-030-29611-7_11

Cited by 5 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method is very useful because it fundamentally introduces the use of isolation trees as an effective way of detecting anomalies from datasets. In addition, this method can work with low linear time complexity and low memory requirements so it can perform well regardless of data size [Yao et al, 2019]. The main idea of the iForest is that the number of data points is abnormal is usually small, and there is a significant difference between normal and these abnormal attributes.…”

Section: Anomaly Detectionmentioning

confidence: 99%

On the statistical learning analysis of rain gauge data over the Natuna Islands

Fajary¹,

Irawan²

2021

Preprint

View full text Add to dashboard Cite

This article presents state-of-the-art statistical learning methods for analyzing rain gauge data over the Natuna Islands. By using shape preserving piecewise cubic interpolation, we managed to interpolate 671 null values from the daily precipitation data. Dominant periodicity analysis of daily precipitation signals using Lomb-Scargle Power Spectral Density shows annual, intraseasonal, and interannual precipitation patterns over the Natuna Islands. Unsupervised anomaly analysis using the Isolation Forest algorithm shows there are 146 anomaly daily precipitation data points. We also conducted an experiment to predict the accumulation of monthly precipitation over the Natuna Islands using the Bayesian structural time series algorithm. The results show that the local linear trend with seasonality model is able to model the value of accumulated monthly precipitation for a twelve-month prediction horizon. The work presented here has profound implications for rainfall observations in this area.

show abstract

Section: Anomaly Detectionmentioning

confidence: 99%

On the statistical learning analysis of rain gauge data over the Natuna Islands

Fajary¹,

Irawan²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This method is fundamentally useful because it introduces the use of isolation as an effective and efficient way of detecting anomalies. Besides, this method is an algorithm with low linear time complexity and small memory requirements that can build a model that performs well by using a small subsample of fixed size, regardless of the size of the data set [26]. The main idea of the isolation forest algorithm is that the number of abnormal points is usually small, and there is a significant difference between normal points and attributes.…”

Section: Isolationforest Algorithmmentioning

confidence: 99%

“…The main idea of the isolation forest algorithm is that the number of abnormal points is usually small, and there is a significant difference between normal points and attributes. This algorithm has a basic pattern on the decision tree model that can break down complex decision-making processes to be simple so that the decisionmaking process will better interpret the solution of the problem [26].…”

Section: Isolationforest Algorithmmentioning

confidence: 99%

Author Matching Classification with Anomaly Detection Approach for Bibliomethric Repository Data

Yamani

Nurmaini²,

Rini³

2020

ComEngApp

View full text Add to dashboard Cite

Authors name disambiguation (AND) is a complex problem in the process of identifying an author in a digital library (DL). The AND data classification process is very much determined by the grouping process and data processing techniques before entering the classifier algorithm. In general, the data pre-processing technique used is pairwise and similarity to do author matching. In a large enough data set scale, the pairwise technique used in this study is to do a combination of each attribute in the AND dataset and by defining a binary class for each author matching combination, where the unequal author is given a value of 0 and the same author is given a value of 1. The technique produces very high imbalance data where class 0 becomes 98.9% of the amount of data compared to 1.1% of class 1. The results bring up an analysis in which class 1 can be considered and processed as data anomaly of the whole data. Therefore, anomaly detection is the method chosen in this study using the Isolation Forest algorithm as its classifier. The results obtained are very satisfying in terms of accuracy which can reach 99.5%.

show abstract

“…Finally, it is worth noting algorithms altering the main concept of isolating trees formation. Yao et al [43] proposed to introduce Mahalanobis distance to determine the membership of an element to a subnode. One can see there are many different solutions improving the underlying IF algorithm.…”

Section: Introductionmentioning

confidence: 99%

Isolation Forest Based on Minimal Spanning Tree

2022

View full text Add to dashboard Cite

Detecting anomalies in data sets has been one of the most studied issues in modern data analysis. Therefore, there is a plethora of applications in a very wide range of fields of science and technology. One of the most frequently used anomaly detection methods is Isolation Forest. In this study, we propose a novel efficient approach based on this technique. In order to improve the classification accuracy of the base method, we make two-fold modifications. First, we propose a change of the technique of building isolation trees to merge nodes by minimal spanning tree algorithm. The second change is based on a modification of the function assessing the anomaly of the analyzed element (data record) to sum of factors correlated with tree height and nearest point distance. In the series of comprehensive computational experiments, the proposed method has proven to produce better results than other compared state-of-the-art methods available in popular data mining programming libraries. It is worth stressing that the final version of the new method in comparison to original Isolation Forest is 2.9% better in terms of AUC measure.

show abstract

Distribution Forest: An Anomaly Detection Method Based on Isolation Forest

Cited by 5 publications

References 12 publications

On the statistical learning analysis of rain gauge data over the Natuna Islands

On the statistical learning analysis of rain gauge data over the Natuna Islands

Author Matching Classification with Anomaly Detection Approach for Bibliomethric Repository Data

Isolation Forest Based on Minimal Spanning Tree

Contact Info

Product

Resources

About