Generalized isolation forest for anomaly detection

Lesouple, Julien; Baudoin, Cédric; Spigai, Marc; Tourneret, Jean-Yves

doi:10.1016/j.patrec.2021.05.022

Cited by 86 publications

(31 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is accomplished by randomly selecting a feature and then selecting a split value between the minimum and maximum values of that feature. Anomalous data points will have shorter paths in the resulting trees due to this random partitioning, making them stand out from the rest of the data (e.g., refs − ). Unlike typical anomaly detection methods, which start by defining what is considered normal and then flagging anything that falls outside of that definition, isolation forest does not have a predefined notion of normal behavior.…”

Section: Resultsmentioning

confidence: 99%

“…Isolation forest is an unsupervised machine learning algorithm that uses a decision tree algorithm 34,35 to identify anomalies by separating them from the rest of the data. 36 This is accomplished by randomly selecting a feature and then selecting a split value between the minimum and maximum values of that feature. Anomalous data points will have shorter Isolation forest can be used for anomaly detection in the mine water data.…”

Section: Isolation Forest For Anomaly Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Exploring Advanced Statistical Data Analysis Techniques for Interpolating Missing Observations and Detecting Anomalies in Mining Influenced Water Data

Wolkersdorfer

2023

ACS EST Water

View full text Add to dashboard Cite

Collecting mining influenced water (MIW) quality data can result in incomplete data sets with missing values and anomalies, making it challenging to use the data for optimizing mine water management. This work explores advanced statistical data analysis approaches for addressing missing data interpolation and anomaly detection in MIW data sets. The study compares the performance of five different interpolation techniques and four different anomaly detection techniques using supervised and unsupervised machine learning algorithms developed using Python 3.8.16. The results of the study demonstrate that the radial basis function, spline, and k-nearest-neighbors interpolation techniques, along with the predictive confidence interval level anomaly approach based on gradient boosting regression trees, perform best for missing data interpolation and anomaly detection, respectively. Thorough application of these advanced techniques can improve the accuracy and reliability of mine water quality data, which is crucial for making conclusions on the safety of the environment, public health, and effective MIW management. This paper highlights the importance of developing effective methods for addressing missing data and anomalies in MIW data sets, which can ultimately lead to improved treatment plant optimization.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Isolation Forest For Anomaly Detectionmentioning

confidence: 99%

Exploring Advanced Statistical Data Analysis Techniques for Interpolating Missing Observations and Detecting Anomalies in Mining Influenced Water Data

Wolkersdorfer

2023

ACS EST Water

View full text Add to dashboard Cite

show abstract

“…The input feature space was split into “eligible” and “non-eligible” feature sub-spaces, where the “eligible” features were those having less than 30% missing values without inconsistent fields and anomalies after experimentations with the percentage of information loss. The Isolation Forests were trained on non-missing records to identify outliers [19] . The covariance matrix was estimated between each pair of input features to remove duplicated features along with the Levenshtein distance to remove lexically identical features [20] .…”

Section: Methodsmentioning

confidence: 99%

Bayesian Inference-Based Gaussian Mixture Models With Optimal Components Estimation Towards Large-Scale Synthetic Data Generation for In Silico Clinical Trials

Pezoulas

Tachos

Gkois

et al. 2022

IEEE Open J. Eng. Med. Biol.

View full text Add to dashboard Cite

To develop a computationally efficient and unbiased synthetic data generator for large-scale in silico clinical trials (CTs). Methods: We propose the BGMM-OCE, an extension of the conventional BGMM (Bayesian Gaussian Mixture Models) algorithm to provide unbiased estimations regarding the optimal number of Gaussian components and yield high-quality, largescale synthetic data at reduced computational complexity. Spectral clustering with efficient eigenvalue decomposition is applied to estimate the hyperparameters of the generator. A case study is conducted to compare the performance of BGMM-OCE against four straightforward synthetic data generators for in silico CTs in hypertrophic cardiomyopathy (HCM). Results: The BGMM-OCE generated 30000 virtual patient profiles having the lowest coefficient-of-variation (0.046), inter-and intra-correlation differences (0.017, and 0.016, respectively) with the real ones in reduced execution time. Conclusions: BGMM-OCE overcomes the lack of population size in HCM which obscures the development of targeted therapies and robust risk stratification models.

show abstract

“…The basic principle of isolation forest (IF) is to divide the data space into two subspaces with a random hyperplane. The subspace will further divide until there is only one data node in each subspace, forming the IF (Lesouple et al, 2021;Zolfaghari and Golabi, 2021).…”

Section: Ifmentioning

confidence: 99%

Quality control algorithm of wind speed monitoring data along high-speed railway

Xiong

Cui³

et al. 2023

Front. Energy Res.

View full text Add to dashboard Cite

For studying the traffic safety of the high-speed railway, this research considers high-quality second-level wind speed data as its basis. However, the quality of second-level wind speed data can be greatly lowered by disturbances during data collection and storage. Therefore, it is crucial to control the data quality during collection and storage. Wind speed data along the high-speed railway are unstable and non-linear. In order to adapt to this characteristic, this study combines a convolutional neural network (CNN), long short-term memory (LSTM), and isolated forest from the time dimension to form a quality control (QC) algorithm for wind speed monitoring data. First, CNN is used to extract the original data features, which are then transferred to the LSTM network for one-step prediction. The prediction residual of the model is obtained and sent to the isolated forest, where the abnormal value position in the original wind speed data is calibrated by detecting the abnormal value position in the prediction residual. Comparative experiments have been conducted to test the performances of the three different QC methods. The results show that the error detection rate of CNN–LSTM–IF in this research method is approximately 0.95. For different terrains and seasons, the method has certain robustness and generalization.

show abstract

Generalized isolation forest for anomaly detection

Cited by 86 publications

References 14 publications

Exploring Advanced Statistical Data Analysis Techniques for Interpolating Missing Observations and Detecting Anomalies in Mining Influenced Water Data

Exploring Advanced Statistical Data Analysis Techniques for Interpolating Missing Observations and Detecting Anomalies in Mining Influenced Water Data

Bayesian Inference-Based Gaussian Mixture Models With Optimal Components Estimation Towards Large-Scale Synthetic Data Generation for In Silico Clinical Trials

Quality control algorithm of wind speed monitoring data along high-speed railway

Contact Info

Product

Resources

About