A New Procedure of Clustering Based on Multivariate Outlier Detection

Abstract

Clustering is an extremely important task in a wide variety of application domains especially in management and social science research. In this paper, an iterative procedure of clustering method based on multivariate outlier detection was proposed by using the famous Mahalanobis distance. At first, Mahalanobis distance should be calculated for the entire sample, then using T 2-statistic fix a UCL. Above the UCL are treated as outliers which are grouped as outlier cluster and repeat the same procedure for the remaining inliers, until the variance-covariance matrix for the variables in the last cluster achieved singularity. At each iteration, multivariate test of mean used to check the discrimination between the outlier clusters and the inliers. Moreover, multivariate control charts also used to graphically visualizes the iterations and outlier clustering process. Finally multivariate test of means helps to firmly establish the cluster discrimination and validity. This paper employed this procedure for clustering 275 customers of a famous twowheeler in India based on 19 different attributes of the two wheeler and its company. The result of the proposed technique confirms there exist 5 and 7 outlier clusters of customers in the entire sample at 5% and 1% significance level respectively.

Two novel outlier detection approaches based on unsupervised possibilistic and fuzzy clustering

“…On the other hand, as mentioned in many studies, K-means is sensitive to noise and outliers, and may not give accurate results ( Hodge & Austin, 2004 ; Gan & Ng, 2017 ). Alternatively, K-medoids or partition around medoids (PAM) is less sensitive to local minima problem and, therefore, some studies targeted to use these hard clustering algorithms in outlier detection ( Jayakumar & Thomas, 2013 ; Kumar, Kumar & Singh, 2013 ). However, the hard clustering algorithms such as K-means and PAM force each data point to belong to the nearest cluster.…”

A Review on Outliers-Detection Methods for Multivariate Data

“…Clustering is one of the informal ways to identifying outliers (Jayakumar & Thomas, 2013;Johnson & Wichern, 2002). The aim of clustering is to group a set of observations into clusters based on similarities or distances (dissimilarities) (Irani et al, 2016;Johnson & Wichern, 2002).…”

“…The most popular and easiest way to compute similarity is Euclidean distance but it does not take into account the covariance structure and is not appropriate for multivariate data (Almeida et al, 2007). Studies such as Hardin and Rocke (2004) and Jayakumar and Thomas (2013) used Mahalanobis distance as a similarity measure. In clustering, outliers are defined as observations that is far from any clusters or have large distance from the centre of each cluster (Hardin & Rocke, 2004;Zhang, 2013).…”

“…Additionally, the discarded observations (outliers) in the first step may optionally assign to these clusters (Almeida et al, 2007). Jayakumar and Thomas (2013) proposed a new method of outlier based clustering based on Mahalanobis distance and found that their method is easier to implement compared to other clustering algorithms. The Mahalanobis distance computed for each observation and upper control limit (UCL) is used as a cutoff value to determine outliers.…”

Using Mahalanobis distance and decision tree to analyze abnormal patterns of behavior in a maintenance outsourcing process-a case study

“…Anomaly detection is an unsupervised target detection technique where no prior knowledge about the target or the background is available, focusing on distinguishing unusual material from a typical background (Shaw and Manolakis, 2002). Mahalanobis Distance (MD) is based on correlations between variables and can be used to identify and analyze different patterns (Jayakumar and Thomas, 2013). MD has been used for many different purposes, including detection of outliers (De Maesschalck, Jouan-Rimbaud and Massart, 2000).…”