TADILOF: Time Aware Density-Based Incremental Local Outlier Detection in Data Streams

Huang, Jen-Wei; Zhong, Meng Xun; Jaysawal, Bijay Prasad

doi:10.3390/s20205829

Cited by 17 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 7 of these data sets (Hepatitis, Biomed, Heart Disease, Boston_HP, Breast_O, SpamBase, and Satimage), the ROC-AUC score rises with the increase of k. On the other hand, according to the experimental results in [3], LOF values stabilize after k = 10. Therefore, the value of k should be chosen to be at least 10 in order to be less affected by fluctuations in the LOF value caused by k. In addition, the process time should be taken into account, since choosing the value of k too large will require high process time (cost) [45], [46]. In view of these facts, it would be a good option to choose the smallest k value greater than 10 according to M T = 0.50 to obtain the highest (or near) ROC-AUC score.…”

Section: E Detailed Analyses Of Kmentioning

confidence: 99%

Robust Incremental Outlier Detection Approach Based on a New Metric in Data Streams

Değırmencı¹,

Karal²

2021

IEEE Access

View full text Add to dashboard Cite

Detecting outliers in real time from multivariate streaming data is a vital and challenging research topic in many areas. Recently introduced the incremental Local Outlier Factor (iLOF) approach and its variants have received considerable attention as they achieve high detection performance in data streams with varying distributions. However, these iLOF-based approaches still have some major limitations: i) Poor detection in high-dimensional data; ii) The difficulty of determining the proper nearest neighbor number k; iii) Instead of labeling the outlier, assigning a score to each sample that indicates the probability to be an outlier; iv) Inability to detect a long sequence (small cluster) of outliers. This article proposes a new robust outlier detection method (RiLOF) based on iLOF that can effectively overcome these limitations. In the RiLOF method, a novel metric called Median of Nearest Neighborhood Absolute Deviation (MoNNAD) has been developed that uses the median of the local absolute deviation of the samples LOF values. Unlike the previously reported LOF-based approaches, RiLOF is capable of achieving outlier detection in different data stream applications using the same hyperparameters. Extensive experiments performed on 15 different realworld data sets demonstrate that RiLOF remarkably outperforms 12 different state-of-the-art competitors.INDEX TERMS Incremental learning, local outlier factor (LOF), new metric, outlier detection, robustness.

show abstract

Section: E Detailed Analyses Of Kmentioning

confidence: 99%

Robust Incremental Outlier Detection Approach Based on a New Metric in Data Streams

Değırmencı¹,

Karal²

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Then we compute LOF score of p i according to equations 1, 2, and 3 and add p i to the set of outliers, O, if LOF score of p i is greater than LOF threshold, T (lines 7-10). At the same time, we update LOF score of each data point p j in the reverse neighbour set of p i and if the data point p j transformed from outlier to inlier, we remove it from O (lines [11][12][13][14][15][16]…”

Section: B Summarization Stepmentioning

confidence: 99%

“…The communication cost of these networks is also an essential factor. There are many approaches of outlier detection over data stream such as clustering based outlier detection [4],statistical based outlier detection [5], [6], distance based outlier detection [7], [8], [9], [10] [11], and density based outlier detection [12], [13], [14] [15], [16]. In this paper,we are interested in the density-based outlier detection over data streams.…”

Section: Introductionmentioning

confidence: 99%

A Fast and Efficient Algorithm for Outlier Detection Over Data Streams

Hassaan¹,

Maher²,

Gouda³

2021

IJACSA

View full text Add to dashboard Cite

Outlier detection over data streams is an important task in data mining. It has various applications such as fraud detection, public health, and computer network security. Many approaches have been proposed for outlier detection over data streams such as distance-,clustering-, density-, and learning-based approaches. In this paper, we are interested in the densitybased outlier detection over data streams. Specifically, we propose an improvement of DILOF, a recent density-based algorithm. We observed that the main disadvantage of DILOF is that its summarization method has many drawbacks such as it takes a lot of time and the algorithm accuracy is significant degradation. Our new algorithm is called DILOF C that utilizing an efficient summarization method. Our performance study shows that DILOF C outperforms DILOF in terms of total response time and outlier detection accuracy.

show abstract

“…In [2], a new algorithmic rule for the purpose of streaming data, referred to as "time-aware density-based incremental local outlier detection", was proposed to conquer variations in data that change as time goes on. The results show that the proposed "time-aware density-based incremental local outlier detection" performs better than that of the existing candidates in the sense of the AUC in most of the cases on different kinds of datasets.…”

Section: Summary Of the Special Issuementioning

confidence: 99%