The complexity of non-hierarchical clustering with instance and cluster level constraints

Abstract

Recent work has looked at extending clustering algorithms with instance level must-link (ML) and cannot-link (CL) background information. Our work introduces δ and ǫ cluster level constraints that influence inter-cluster distances and cluster composition. The addition of background information, though useful at providing better clustering results, raises the important feasibility question: Given a collection of constraints and a set of data, does there exist at least one partition of the data set satisfying all the constraints? We study the complexity of the feasibility problem for each of the above constraints separately and also for combinations of constraints. Our results clearly delineate combinations of constraints for which the feasibility problem is computationally intractable (i.e., NP-complete) from those for which the problem is efficiently solvable (i.e., in the computational class P).We also consider the ML and CL constraints in conjunctive and disjunctive normal forms (CNF and DNF respectively). We show that for ML constraints, the feasibility problem is intractable for CNF but efficiently solvable for DNF. Unfortunately, for CL constraints, the feasibility problem is intractable for both CNF and DNF. This effectively means that CLconstraints in a non-trivial form cannot be efficiently incorporated into clustering algorithms. To overcome this, we introduce the notion of a choice-set of constraints and prove that the feasibility problem for choice-sets is efficiently solvable for both ML and CL constraints. We also present empirical results which indicate that the feasibility problem occurs extensively in real world problems.

Statistical Relational Learning with Formal Ontologies

“…The use of hard constraints for clustering tasks in purely statistical approaches to learning, as opposed to the ubiquitous use of "soft" prior knowledge, has been approached in, e.g., [15]. A common characteristic of these approaches is that they work with a relatively narrow, semi-formal notion of constraints and do not relate constraints to relational learning.…”

“…In contrast to these efforts, our approach allows for rich constraints which take the form of a OWL DL knowledge base (with much higher expressivity). The notion of forbidden pairings of data points (cannot-link constraints [15]) is replaced with the more general notion of logical (un-)satisfiability w.r.t. formal background knowledge.…”

Efficient incremental constrained clustering

“…To provide a proper perspective for our contributions, we mention the assumptions used in this work. Firstly, our earlier work on the feasibility problem [4,5,7] addressed the question of determining whether there is a partition that satisfies a given set of constraints. In this paper, we assume that a clustering satisfying all the initial constraints (C) is given and that the addition of new constraints (C ′ ) may give rise to a combined set of constraints for which no feasible clustering exists for the given value of k (the number of clusters).…”

“…It is not that the graph is not 3-colorable, rather the ordering of the nodes presents a problem. This is of pragmatic importance [7] since the ordering of the points as processed by clustering algorithms is typically fixed apriori and does not change.…”

“…Firstly, when compared to algorithms that attempt to satisfy all constraints (as it does) incremental constrained clustering compares favorably and indeed performs better. This is to be expected as by incrementally specifying the constraints (rather than all at once) the incremental algorithm does not get over-constrained as the non-incremental variant can [7].…”