Information State: A Representation for Dynamic Processes Using Information Theory

Validation is one of the most important aspects of clustering, but most approaches have been batch methods. Recently, interest has grown in providing incremental alternatives. This paper extends the incremental cluster validity index (iCVI) family to include incremental versions of Calinski-Harabasz (iCH), I index and Pakhira-Bandyopadhyay-Maulik (iI and iPBM), Silhouette (iSIL), Negentropy Increment (iNI), Representative Cross Information Potential (irCIP) and Representative Cross Entropy (irH), and Conn Index (iConn Index). Additionally, the effect of under-and over-partitioning on the behavior of these six iCVIs, the Partition Separation (PS) index, as well as two other recently developed iCVIs (incremental Xie-Beni (iXB) and incremental Davies-Bouldin (iDB)) was examined through a comparative study. Experimental results using fuzzy adaptive resonance theory (ART)-based clustering methods showed that while evidence of most under-partitioning cases could be inferred from the behaviors of all these iCVIs, over-partitioning was found to be a more challenging scenario indicated only by the iConn Index. The expansion of incremental validity indices provides significant novel opportunities for assessing and interpreting the results of unsupervised learning. PREPRINT SUBMITTED TO ARXIV.ORG 2 I. INTRODUCTION Cluster validation [1] is a critical topic in cluster analysis. It is crucial to assess the quality of the partitions detected by clustering algorithms when there is no class label information. Different clustering solutions may be found by distinct algorithms, or even by the same algorithm subjected to different hyper-parameters or a different input presentation order [2], [3]. Cluster validity indices (CVIs) perform the role of evaluators of such solutions. CVIs typically exhibit a trade-off between measures of compactness (within-cluster scatter) and isolation (between-cluster separation) [2]. Numerous examples of such criteria have been presented in the literature; for comprehensive reviews and experimental studies the interested reader can go to [4]-[11]. Recently, incremental cluster validity indices (iCVIs) have been developed to track the effectiveness of online clustering methods over data streams [12]-[15]. To enable cluster validation in such applications, a recursive formulation of compactness was introduced in [12], [13]. This strategy has been used to develop incremental versions of four CVIs so far [15]: viz., incremental Davies-Bouldin (iDB) [12], [13], incremental Xie-Beni (iXB) [12], [13] and modified Dunn's indices [16]. Particularly, the behavior of iXB and iDB are analyzed in both accurately and poorly partitioned data sets in [12], [13], whereas the studies in [14], [15] only investigate the iDB's behavior in cases where online clustering algorithms accurately detect data structures, i.e., when they yield high performing experimental results. Therefore, the contributions of this work are three-fold: (1) presenting incremental versions of six additional CVIs (thereby extending the family of iC...

show abstract

“…where [49] measure has been used to define a system's state when modeling and analyzing dynamic processes [50], [51].…”

Section: ) Representativementioning

confidence: 99%

“…Smaller values of CEF indicate better clustering solutions (minimization). Recently, the information potential (IP) [49] measure has been used to define a system's state when modeling and analyzing dynamic processes [50], [51]. 9) Conn Index [52], [53]: the Conn Index was also developed for prototype-based clustering.…”

Section: )mentioning

confidence: 99%