Autonomous data-driven clustering for live data stream

Gu, Xiaowei; Angelov, Plamen

doi:10.1109/smc.2016.7844394

Cited by 8 publications

(10 citation statements)

References 15 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ii. A comparative analysis in a streaming data scenario among LAMDA family (LAMDA-RD, LAMDA-TP and original LAMDA) and the algorithm called "Autonomous Data-driven Clustering for Live DataStream (ADDclustering) [46]. This algorithm has been selected since it allows online clustering, a characteristic to be considered in order to make a fair comparison with LAMDA.…”

Section: General Procedures Of Lamda-rd With Automatic Merge Algorithmmentioning

confidence: 99%

“…In this context, a successful algorithm must consider the following restrictions [56]: -Individuals continually arrive; -There is no control in the order in which the individuals are generated; -The size of a stream is (potentially) unbounded; -Data objects are discarded after they have been processed. All these restrictions are considered in the test, for LAMDA family (LAMDA-RD, LAMDA-TP, original LAMDA), and another online clustering method called ADDclustering, for live data stream [46]. A maximum exigency parameter is set ( = 1), because it is desired a strict behavior for the algorithms in the assignment process.…”

Section: B Performance Comparison Of Lamda-rd and Other Online Clustmentioning

confidence: 99%

“…As detailed in Section III, LAMDA-RD initially has a descriptor normalization stage, which is essential to know how the algorithm works with the samples, allowing us to analyze the behavior, performance and sensitivity of LAMDA-RD in the presence of outliers. Normalization is essential in LAMDA-RD since the algorithm does not eliminate outliers, which is not recommended because valuable information can be obtained from this data, especially in the context of a stream mining scenario [46]. For the normalization stage, the maximum and minimum values of each descriptor/feature are required, as shown in Eq.…”

Section: Lamda-rd and Outliersmentioning

confidence: 99%

See 2 more Smart Citations

An Automatic Merge Technique to Improve the Clustering Quality Performed by LAMDA

Morales

Aguilar

2020

IEEE Access

View full text Add to dashboard Cite

Clustering is a research challenge focused on discovering knowledge from data samples whose goal is to build good quality partitions. In this paper is proposed an approach based on LAMDA (Learning Algorithm for Multivariable Data Analysis), whose most important features are: a) it is a non-iterative fuzzy algorithm that can work with online data streams, b) it does not require the number of clusters, c) it can generate new partitions with objects that do not have enough similarity with the preexisting clusters (incremental-learning). However, in some applications, the number of created partitions does not correspond with the number of desired clusters, which can be excessive or impractical for the expert. Therefore, our contribution is the formalization of an automatic merge technique to update the cluster partition performed by LAMDA to improve the quality of the clusters, and a new methodology to compute the Marginal Adequacy Degree that enhances the individual-cluster assignment. The proposal, called LAMDA-RD, is applied to several benchmarks, comparing the results against the original LAMDA and other clustering algorithms, to evaluate the performance based on different metrics. Finally, LAMDA-RD is validated in a real case study related to the identification of production states in a gas-lift well, with data stream. The results have shown that LAMDA-RD achieves a competitive performance with respect to the other well-known algorithms, especially in unbalanced benchmarks and benchmarks with an overlapping of around 9%. In these cases, our algorithm is the best, reaching a Rand Index (RI) > 98%. Besides, it is consistently among the best for all metrics considered (Silhouette coefficient, modification of the Silhouette coefficient, WB-index, Performance Coefficient, among others) in all case studies analyzed in this paper. Finally, in the real case study, it is better in all the metrics.

show abstract

Section: General Procedures Of Lamda-rd With Automatic Merge Algorithmmentioning

confidence: 99%

Section: B Performance Comparison Of Lamda-rd and Other Online Clustmentioning

confidence: 99%

Section: Lamda-rd and Outliersmentioning

confidence: 99%

See 1 more Smart Citation

An Automatic Merge Technique to Improve the Clustering Quality Performed by LAMDA

Morales

Aguilar

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Studies have been carried out using the technique where papers stand on parallel programming [32,39], autonomous clustering techniques [31], clustering techniques for regression problems and date classification stream [67], prediction of terrorist activities [37], and clustering methods for probabilistic graphs, [36]. An example of a graphical representation of how the technique identifies centers is given in figure 3.…”

Section: First Layer-data Density Fuzzificationmentioning

confidence: 99%

Autonomous Data Density pruning fuzzy neural network for Optical Interconnection Network

et al. 2020

View full text Add to dashboard Cite

Traditionally, fuzzy neural networks have parametric clustering methods based on equally spaced membership functions to fuzzify inputs of the model. In this sense, it produces an excessive number calculations for the parameters' definition of the network architecture, which may be a problem especially for realtime large-scale tasks. Therefore, this paper proposes a new model that uses a nonparametric technique for the fuzzification process. The proposed model uses an autonomous data density approach in a pruned fuzzy neural network, wich favours the compactness of the model. The performance of the proposed approach is evaluated through the usage of databases related to the Optical Interconnection Network. Finally, binary patterns classification tests for the identification of Temporal Distribution (Asynchronous or Client-Server) were performed and compared with stateof-the-art fuzzy neural-based and traditional machine learning approaches. Results demonstrated that the proposed model is an efficient tool for these challenging classification tasks.

show abstract

“…In order to solve these challenges, Angelov proposed a Recursive Density Estimation (RDE) [42] formula to model the density distribution of the data stream over time without any user-specified parameters. Based on RDE, several onepass stream clustering approaches are proposed in [42]- [44] to automatically extract the cluster structure from streaming data, and it can effectively overcome the above challenges. However, these methods are developed to handle data streams where the individual sample is arriving continuously while most of the real-world data streams arrive as a sequence of data chunks, which may not be appropriate in the analysis of real-world data streams.…”

Section: B Challenges Motivations and Contributionsmentioning

confidence: 99%

A Novel Streaming Data Clustering Algorithm Based on Fitness Proportionate Sharing

Yan

Razeghi-Jahromi²,

Homaifar

et al. 2019

IEEE Access

View full text Add to dashboard Cite

As an unsupervised learning technique, clustering can effectively capture the patterns in a data stream based on similarities among the data. Traditional data stream clustering algorithms either heavily depend on some prior knowledge or predefined parameters while the characteristics of real-time data are considered unknown. Besides, the user-specified threshold is used to overcome the effect of outliers and noises, which significantly affects the clustering performance. The overlap among clusters is another major challenge for the existing stream clustering methods. These constraints strongly limit their real-time applications. In this paper, a two-phase stream clustering algorithm based on fitness proportionate sharing is proposed. It handles streaming data when prior knowledge is not available and maps the clustering problem into a multimodal optimization problem. It introduces a density-based objective function and adopts the fitness proportionate sharing strategy to perform a more effective search for the cluster centers. To capture the dynamic characteristics of streaming data, a recursive formula for the lower bound of the density function is derived, and a summary of historical data is established for the proposed algorithm. The proposed algorithm is applied to different data sets, and a comprehensive comparison between the proposed algorithm and five well-known data stream clustering algorithms in the literature is provided. Results show comparable or better performance relative to five popular data stream clustering algorithms. A scalability analysis of the proposed streaming clustering method is presented in this paper as well.INDEX TERMS Data streams, clustering, unsupervised learning, data mining. I. INTRODUCTION

show abstract

Autonomous data-driven clustering for live data stream

Cited by 8 publications

References 15 publications

An Automatic Merge Technique to Improve the Clustering Quality Performed by LAMDA

An Automatic Merge Technique to Improve the Clustering Quality Performed by LAMDA

Autonomous Data Density pruning fuzzy neural network for Optical Interconnection Network

A Novel Streaming Data Clustering Algorithm Based on Fitness Proportionate Sharing

Contact Info

Product

Resources

About