Self-Organised direction aware data partitioning algorithm

Abstract

In this paper, a novel fully data-driven algorithm, named Self-Organised Direction Aware (SODA) data partitioning and forming data clouds is proposed. The proposed SODA algorithm employs an extra cosine similarity-based directional component to work together with a traditional distance metric, thus, takes the advantages of both the spatial and angular divergences. Using the nonparametric Empirical Data Analytics (EDA) operators, the proposed algorithm automatically identifies the main modes of the data pattern from the empirically observed data samples and uses them as focal points to form data clouds. A streaming data processing extension of the SODA algorithm is also proposed. This extension of the SODA algorithm is able to self-adjust the data clouds structure and parameters to follow the possibly changing data patterns and processes.Numerical examples provided as a proof of the concept illustrate the proposed algorithm as an autonomous algorithm and demonstrate its high clustering performance and computational efficiency.

Self-organising fuzzy logic classifier

“…In this paper, we give an example of recursive calculation expressions using Mahalanobis distance. The recursive calculation forms of the EDA quantities with other types of distance metric can be found in the previous works [4], [18].…”

“…is checked by the following condition to evaluate its potential to be a new prototype [2], [8]: (18) where equation (10) is used for calculating   (16)) and the meta-parameters of the SOF classifier are updated as follows:…”

A method for autonomous data partitioning

“…The best way to deal with such datasets is to partition/cluster the data samples into smaller data clouds/clusters, and combine them later. Therefore, in this paper, we only require the numbers of data clouds/clusters in the clustering results to be close to the ground truth but larger than the number of classes, but not excessively large [28]. Therefore, we consider the clustering result with C meeting the following condition as a valid one:…”

“…The ADP algorithm-offline version; b The ADP algorithm-evolving version. In the experiments, for the high dimensional datasets (N>20, N=N A ), we normalize the data via the following equation, which converts the Euclidean distance between data samples into a cosine dissimilarity [28]:…”

A distance-type-insensitive clustering approach

“…There are two data partitioning algorithms introduced recently that deserve special attentions. One of them is the self-organized direction-aware data partitioning (SODA) algorithm [7]. Unlike other clustering algorithms, the SODA algorithm involves a (linear) combination of a distance metric and a cosine dissimilarity-based component to estimate both, the spatial and angular divergences of the data.…”