Local and Global Outlier Detection Algorithms in Unsupervised Approach: A Review

Jabbar, Ayad Mohammed

doi:10.37917/ijeee.17.1.9

Cited by 11 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the research in the searched range, i.e., the impact of outliers on the quality of clustering, was not included. The paper [ 8 ] presents, in turn, a comparison of dozens of different approaches based on clustering and outlier detection but without any research details. Although, it is impossible to find papers that combine these issues into one study.…”

Section: State Of the Artmentioning

confidence: 99%

How the Outliers Influence the Quality of Clustering?

Nowak-Brzezińska

Gaibei

2022

Entropy

View full text Add to dashboard Cite

In this article, we evaluate the efficiency and performance of two clustering algorithms: AHC (Agglomerative Hierarchical Clustering) and K−Means. We are aware that there are various linkage options and distance measures that influence the clustering results. We assess the quality of clustering using the Davies–Bouldin and Dunn cluster validity indexes. The main contribution of this research is to verify whether the quality of clusters without outliers is higher than those with outliers in the data. To do this, we compare and analyze outlier detection algorithms depending on the applied clustering algorithm. In our research, we use and compare the LOF (Local Outlier Factor) and COF (Connectivity-based Outlier Factor) algorithms for detecting outliers before and after removing 1%, 5%, and 10% of outliers. Next, we analyze how the quality of clustering has improved. In the experiments, three real data sets were used with a different number of instances.

show abstract

Section: State Of the Artmentioning

confidence: 99%

How the Outliers Influence the Quality of Clustering?

Nowak-Brzezińska

Gaibei

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…An effective anomaly detection algorithm should be able to detect both global and local anomalies. In most cases, if the chosen algorithm is effective in finding global anomalies, then they fail to determine local anomalies and vice versa [36].…”

Section: Connectivity-based Outlier Factor (Cof)mentioning

confidence: 99%

Unsupervised Machine Learning for Missing Clamp Detection from an In-Service Train Using Differential Eddy Current Sensor

et al. 2022

View full text Add to dashboard Cite

The rail fastening system plays a crucial role in railway tracks as it ensures operational safety by fixing the rail on to the sleeper. Early detection of rail fastener system defects is crucial to ensure track safety and to enable maintenance optimization. Fastener inspections are normally conducted either manually by trained maintenance personnel or by using automated 2-D visual inspection methods. Such methods have drawbacks when visibility is limited, and they are also found to be expensive in terms of system maintenance cost and track possession time. In a previous study, the authors proposed a train-based differential eddy current sensor system based on the principle of electromagnetic induction for fastener inspection that could overcome the challenges mentioned above. The detection in the previous study was carried out with the aid of a supervised machine learning algorithm. This study reports the finding of a case study, along a heavy haul line in the north of Sweden, using the same eddy current sensor system mounted on an in-service freight train. In this study, unsupervised machine learning models for detecting and analyzing missing clamps in a fastener system were developed. The differential eddy current measurement system was set to use a driving field frequency of 27 kHz. An anomaly detection model combining isolation forest (IF) and connectivity-based outlier factor (COF) was implemented to detect anomalies from fastener inspection measurements. To group the anomalies into meaningful clusters and to detect missing clamps within the fastening system, an unsupervised clustering based on the DBSCAN algorithm was also implemented. The models were verified by measuring a section of the track for which the track conditions were known. The proposed anomaly detection model had a detection accuracy of 96.79% and also exhibited a high score of sensitivity and specificity. The DBSCAN model was successful in clustering missing clamps, both one and two missing clamps, from a fastening system separately.

show abstract

“…It is the sum of distance squares between all samples and their centers of clustering. − Hybrid scale 1: In this scale, the DBI measure is combined with homogeneity and completeness measures using (5).…”

Section: Objective Functionmentioning

confidence: 99%

“…𝐻𝑦𝑏𝑟𝑖𝑑 𝑠𝑐𝑎𝑙𝑒 1 = 0.5 * 𝐷𝐵𝐼 𝑠𝑐𝑜𝑟𝑒 + 0.25 * homogeneity score + 0.25 * completeness score (5) − Hybrid scale 2: In this scale, the DBI measure is combined with Silhouette coefficient using (6).…”

Section: Objective Functionmentioning

confidence: 99%

“…It is one of the most often used strategies for unsupervised classification. Several exist a variety of unsupervised clustering algorithms available including k-means [1], cobweb [2], farthest-first [3], expectationmaximization (EM) [4], density-based [5], and hierarchical clustering [6]. On the whole, though, the most widely used is the k-means algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal k-means clustering using artificial bee colony algorithm with variable food sources length

Raheem

Alabbas²

2022

IJECE

View full text Add to dashboard Cite

<span>Clustering is a robust machine learning task that involves dividing data points into a set of groups with similar traits. One of the widely used methods in this regard is the k-means clustering algorithm due to its simplicity and effectiveness. However, this algorithm suffers from the problem of predicting the number and coordinates of the initial clustering centers. In this paper, a method based on the first artificial bee colony algorithm with variable-length individuals is proposed to overcome the limitations of the k-means algorithm. Therefore, the proposed technique will automatically predict the clusters number (the value of k) and determine the most suitable coordinates for the initial centers of clustering instead of manually presetting them. The results were encouraging compared with the traditional k-means algorithm on three real-life clustering datasets. The proposed algorithm outperforms the traditional k-means algorithm for all tested real-life datasets.</span>

show abstract

Local and Global Outlier Detection Algorithms in Unsupervised Approach: A Review

Cited by 11 publications

References 0 publications

How the Outliers Influence the Quality of Clustering?

How the Outliers Influence the Quality of Clustering?

Unsupervised Machine Learning for Missing Clamp Detection from an In-Service Train Using Differential Eddy Current Sensor

Optimal k-means clustering using artificial bee colony algorithm with variable food sources length

Contact Info

Product

Resources

About