K-Means-based isolation forest

Karczmarek, Paweł; Kiersztyn, Adam; Pedrycz, Witold; Al, Ebru

doi:10.1016/j.knosys.2020.105659

Cited by 111 publications

(30 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we discriminate the anomaly by using the EDynGE model. To detect the anomaly in the embedding space, we applied the existing outlier detection methods, which are local outlier factor (LOF) 8 , isolation forest (IF) 9 , and box-plot (BP) 10 . IF method mentions that the distribution of outliers is sparse, and these outliers are far away from the normal observations with high density.…”

Section: Resultsmentioning

confidence: 99%

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Jung

2021

Sci Rep

View full text Add to dashboard Cite

Abnormal climate event is that some meteorological conditions are extreme in a certain time interval. The existing methods for detecting abnormal climate events utilize supervised learning models to learn the abnormal patterns, but they cannot detect the untrained patterns. To overcome this problem, we construct a dynamic graph by discovering the correlation among the climate time series and propose a novel dynamic graph embedding model based on graph entropy called EDynGE to discriminate anomalies. The graph entropy measurement quantifies the information of the graphs and constructs the embedding space. We conducted experiments on synthetic datasets and real-world meteorological datasets. The results showed that EdynGE model achieved a better F1-score than the baselines by 43.2%, and the number of days of abnormal climate events has increased by 304.5 days in the past 30 years.

show abstract

Section: Resultsmentioning

confidence: 99%

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Jung

2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In this section, we apply the EDynGE model to real meteorological data and use local outlier factor (LOF) 16 , isolation forest (IF) 17 , and box-plot (BP) 18 methods to detect the abnormal climate events in an embedding space. IF shows that the distribution of outliers is sparse, and these outliers are far away from the normal observations with high density.…”

Section: Resultsmentioning

confidence: 99%

Entropy-Based Dynamic Graph Embedding for Climate Change Detection

Jung

2020

Preprint

View full text Add to dashboard Cite

Climate change is a severe problem caused by abnormal climate events. The existing methods for detecting climate changes utilize statistical models to analyze the atmospheric temperature, but a climate event commonly comprises multiple meteorological data. To detect climate changes using meteorological data, we propose a novel dynamic graph embedding model based on graph entropy called EDynGE. A climate event is denoted as a graph, in which the nodes indicate meteorological data and edges indicate the correlation between nodes. Graph entropy measures the information of the climate event, and the EDynGE model clusters graphs based on graph entropy. We conducted experiments on real meteorological data. The results showed that the number of days of abnormal climate events has increased by 304.5 days in the past 30 years.

show abstract

“…Their algorithm for identifying sites with fabricated data achieved slightly lower results—except for 1 study, sensitivity and specificity were greater than 70%. In another research work [ 22 ], the authors combined k-means and isolation forest techniques, because the isolation forest–based methods are capable of finding anomalous patients that are not situated on the edge of a feature space. They, however, did not use ROC curves to define thresholds, but instead [ 23 ] split their data set into 2 subsets—first one consisted of only categorical variables and the second one of only continuous variables.…”

Section: Discussionmentioning

confidence: 99%

Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study

Churová¹,

Vyškovský²,

Maršálová³

et al. 2021

JMIR Med Inform

View full text Add to dashboard Cite

Background Statistical analysis, which has become an integral part of evidence-based medicine, relies heavily on data quality that is of critical importance in modern clinical research. Input data are not only at risk of being falsified or fabricated, but also at risk of being mishandled by investigators. Objective The urgent need to assure the highest data quality possible has led to the implementation of various auditing strategies designed to monitor clinical trials and detect errors of different origin that frequently occur in the field. The objective of this study was to describe a machine learning–based algorithm to detect anomalous patterns in data created as a consequence of carelessness, systematic error, or intentionally by entering fabricated values. Methods A particular electronic data capture (EDC) system, which is used for data management in clinical registries, is presented including its architecture and data structure. This EDC system features an algorithm based on machine learning designed to detect anomalous patterns in quantitative data. The detection algorithm combines clustering with a series of 7 distance metrics that serve to determine the strength of an anomaly. For the detection process, the thresholds and combinations of the metrics were used and the detection performance was evaluated and validated in the experiments involving simulated anomalous data and real-world data. Results Five different clinical registries related to neuroscience were presented—all of them running in the given EDC system. Two of the registries were selected for the evaluation experiments and served also to validate the detection performance on an independent data set. The best performing combination of the distance metrics was that of Canberra, Manhattan, and Mahalanobis, whereas Cosine and Chebyshev metrics had been excluded from further analysis due to the lowest performance when used as single distance metric–based classifiers. Conclusions The experimental results demonstrate that the algorithm is universal in nature, and as such may be implemented in other EDC systems, and is capable of anomalous data detection with a sensitivity exceeding 85%.

show abstract

K-Means-based isolation forest

Cited by 111 publications

References 42 publications

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Entropy-based dynamic graph embedding for anomaly detection on multiple climate time series

Entropy-Based Dynamic Graph Embedding for Climate Change Detection

Anomaly Detection Algorithm for Real-World Data and Evidence in Clinical Research: Implementation, Evaluation, and Validation Study

Contact Info

Product

Resources

About