On the Effects of Constraints in Semi-supervised Hierarchical Clustering

Abstract

Abstract.We explore the use of constraints with divisive hierarchical clustering. We mention some considerations on the effects of the inclusion of constraints into the hierarchical clustering process. Furthermore, we introduce an implementation of a semi-supervised divisive hierarchical clustering algorithm and show the influence of including constraints into the divisive hierarchical clustering process. In this task our main interest lies in building stable dendrograms when clustering with different subsets of data. BackgroundThe aim of cluster analysis is to explore a collection of data items and to group similar objects together. Similarity can be measured in many different ways, for example by the Euclidean distance. Every information used in an unsupervised learning technique comes from the data themselve, which means no external teaching signal guides the algorithm. During the past years several modifications were proposed to incorporate additional background knowledge into clustering algorithms [1][2][3][4][5][6][7][8][9]. The main idea is that in some tasks prior information about a small amount of data is known and should support the clustering process. This background information is usually provided as a set of constraints between pairs of data items [2]. Real class labels seem not feasible in clustering because of the inherent discrepancy between labels and clusters. Pairwise constraints are typically encoded as must-links and cannot-links [2] either indicating two points belong to the same cluster or to different clusters.Up to now different clustering algorithms were adapted to make use of semisupervised clustering. Constraint-based approaches modify the cluster search, either by including constraints into the objective function [1] or by initalizing and constraining during the clustering process [10, 3,9]. Metric-based approaches first train the cluster metric to satisfy the constraints using shortest-path algorithm Many real-world unsupervised learning tasks may profit from an inclusion of background knowledge, e.g. image processing, text mining or even lane finding using GPS data. Another important field for data mining using semi-supervised clustering is the functional grouping of genes. For this problem we are not only interested in one particular clustering as built by partitioning clustering algorithms, such as K-means or SOM, but in getting a hierarchy of partitions (dendrogram).Our main emphasis lies on building robust hierarchical clusterings for DNA microarray data. Here we are interested in the resulting dendrogram and focus on the question of creating stable dendrograms. In this setting one of the problems that frequently arise is the change of the branching structure when computing several runs with different data subsets. Since the inclusion of background knowledge seems to enhance the accuracy of clustering algorithms, we studied its effect on the dendrogram stability. In Section 2 we explain some basic annotations on the effects of constraints in semi-supervised hierarchical cluste...

Incorporating SAT solvers into hierarchical clustering algorithms

“…Davidson and Ravi use must-link (ML) and cannot-link (CL) constraints previously used for non-hierarchical clustering and demonstrated their use for hierarchical clustering [6,7]. Kestler and Kraus showed the limitations of using ML and CL constraints with hierarchical clustering and proposed a method to limit the scope of the constraints [12]. They used must-link and cannot-link constraints for their problem, but because of their application, needed different sets of constraints for the top half of their dendrogram than for the bottom half.…”

“…All of the constraint types described in previous work (instance constraints [7], level specific instance constraints [12], must-link-before [3]) can be formulated using only Horn clauses of our core relation (see Definition 1). We also show how the expressiveness of logic allows us to easily extend the must-link-before constraints to encode a notion of distance.…”

$\mathcal{SHACUN}$ : Semi-supervised Hierarchical Active Clustering Based on Ranking Constraints

“…The new distance between x k and x j is calculated through a triangle inequality. Kestler et al use pairwise constraints on the first level of hierarchical clustering algorithm when generating the initial clusters [11]. Such constraints are not propagated to the posterior levels.…”

Clustering with Attribute-Level Constraints

“…The first clustering constraints in the literature were socalled must-link and cannot-Link constraints, which were incorporated into several algorithms [1], [2]. One main goal was to specify the types of constraints and to show how heuristics can be used to find a near-optimal clustering solution that satisfies all constraints.…”