New Quality Indexes for Optimal Clustering Model Identification with High Dimensional Data

Lamirel, Jean–Charles; Cuxac, Pascal

doi:10.1109/icdmw.2015.220

Cited by 2 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) Xie-Beni (XB) [22]: the XB index was originally designed to detect compact and separated clusters in fuzzy c-partitions. A hard partition version is given by the following ratio of compactness to separation [23], [24]:…”

Section: A Cluster Validity Indices (Cvis)mentioning

confidence: 99%

Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

2020

View full text Add to dashboard Cite

Validation is one of the most important aspects of clustering, but most approaches have been batch methods. Recently, interest has grown in providing incremental alternatives. This paper extends the incremental cluster validity index (iCVI) family to include incremental versions of Calinski-Harabasz (iCH), I index and Pakhira-Bandyopadhyay-Maulik (iI and iPBM), Silhouette (iSIL), Negentropy Increment (iNI), Representative Cross Information Potential (irCIP) and Representative Cross Entropy (irH), and Conn Index (iConn Index). Additionally, the effect of under-and over-partitioning on the behavior of these six iCVIs, the Partition Separation (PS) index, as well as two other recently developed iCVIs (incremental Xie-Beni (iXB) and incremental Davies-Bouldin (iDB)) was examined through a comparative study. Experimental results using fuzzy adaptive resonance theory (ART)-based clustering methods showed that while evidence of most under-partitioning cases could be inferred from the behaviors of all these iCVIs, over-partitioning was found to be a more challenging scenario indicated only by the iConn Index. The expansion of incremental validity indices provides significant novel opportunities for assessing and interpreting the results of unsupervised learning. PREPRINT SUBMITTED TO ARXIV.ORG 2 I. INTRODUCTION Cluster validation [1] is a critical topic in cluster analysis. It is crucial to assess the quality of the partitions detected by clustering algorithms when there is no class label information. Different clustering solutions may be found by distinct algorithms, or even by the same algorithm subjected to different hyper-parameters or a different input presentation order [2], [3]. Cluster validity indices (CVIs) perform the role of evaluators of such solutions. CVIs typically exhibit a trade-off between measures of compactness (within-cluster scatter) and isolation (between-cluster separation) [2]. Numerous examples of such criteria have been presented in the literature; for comprehensive reviews and experimental studies the interested reader can go to [4]-[11]. Recently, incremental cluster validity indices (iCVIs) have been developed to track the effectiveness of online clustering methods over data streams [12]-[15]. To enable cluster validation in such applications, a recursive formulation of compactness was introduced in [12], [13]. This strategy has been used to develop incremental versions of four CVIs so far [15]: viz., incremental Davies-Bouldin (iDB) [12], [13], incremental Xie-Beni (iXB) [12], [13] and modified Dunn's indices [16]. Particularly, the behavior of iXB and iDB are analyzed in both accurately and poorly partitioned data sets in [12], [13], whereas the studies in [14], [15] only investigate the iDB's behavior in cases where online clustering algorithms accurately detect data structures, i.e., when they yield high performing experimental results. Therefore, the contributions of this work are three-fold: (1) presenting incremental versions of six additional CVIs (thereby extending the family of iC...

show abstract

Section: A Cluster Validity Indices (Cvis)mentioning

confidence: 99%

Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

2020

View full text Add to dashboard Cite

show abstract

“…A neural clustering algorithm [9], more stable and more efficient than the usual clustering algorithms on the textual data, is applied multiple times, with standard parameters, on the data of each time period by varying the desired number of clusters. Clustering quality criteria that are reliable for textual and multidimensional data are exploited in a further step [18] to isolate an optimal model (ideal number of clusters) for each of the periods.…”

Section: The Full Text Of Each Obtained Document Is Treated With a Comentioning

confidence: 99%

Performing and Visualizing Temporal Analysis of Large Text Data Issued for Open Sources: Past and Future Methods

Lamirel¹,

Dugué²,

Cuxac

2016

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

In this paper we first propose a state of the art on the methods for the visualization and the interpretation of textual data, in particular of scientific data. We then shortly present our contributions to this field in the form of original methods for the automatic classification of documents and easy interpretation of their content through characteristic keywords and classes created by our algorithms. In a second step, we focus our analysis on the data evolving over time. We detail our diachronic approach, especially suitable for the detection and visualization of topic changes. This allows us to conclude with Diachronic'Explorer, our upcoming tool for visual exploration of evolutionary data.

show abstract

New Quality Indexes for Optimal Clustering Model Identification with High Dimensional Data

Cited by 2 publications

References 19 publications

Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

Performing and Visualizing Temporal Analysis of Large Text Data Issued for Open Sources: Past and Future Methods

Contact Info

Product

Resources

About