Explainable $k$-Means and $k$-Medians Clustering

Abstract

Clustering is a popular form of unsupervised learning for geometric data. Unfortunately, many clustering algorithms lead to cluster assignments that are hard to explain, partially because they depend on all the features of the data in a complicated way. To improve interpretability, we consider using a small decision tree to partition a data set into clusters, so that clusters can be characterized in a straightforward manner. We study this problem from a theoretical viewpoint, measuring cluster quality by the k-means and k-medians objectives: Must there exist a tree-induced clustering whose cost is comparable to that of the best unconstrained clustering, and if so, how can it be found? In terms of negative results, we show, first, that popular top-down decision tree algorithms may lead to clusterings with arbitrarily large cost, and second, that any tree-induced clustering must in general incur an Ω(log k) approximation factor compared to the optimal clustering. On the positive side, we design an efficient algorithm that produces explainable clusters using a tree with k leaves. For two means/medians, we show that a single threshold cut suffices to achieve a constant factor approximation, and we give nearly-matching lower bounds. For general k ≥ 2, our algorithm is an O(k) approximation to the optimal k-medians and an O(k 2 ) approximation to the optimal k-means. Prior to our work, no algorithms were known with provable guarantees independent of dimension and input size.

ExKMC: Expanding Explainable $k$-Means Clustering

“…We address the challenge of obtaining a low cost k-means clustering using a small decision tree. Our approach has roots in previous works on clustering with unsupervised decision trees [7,11,16,22,23,28,43,61] and in prior literature on extending decision trees for tasks beyond classification [29,30,35,45,60,54,55].…”

“…Recent work on explainable clustering goes one step further by enforcing that the clustering be derived from a binary threshold tree [11,18,22,28,32,43]. Each node is associated with a feature-threshold pair that recursively splits the dataset, and labels on the leaves correspond to clusters.…”

“…The goal is to find k centers that approximately minimize the sum of the squared distances between n data points in R d and their nearest center [1,2,4,21,37,52]. In this context, Dasgupta, Frost, Moshkovitz, and Rashtchian have studied the use of a small threshold tree to specify the cluster assignments, exhibiting the first explainable k-means clustering algorithm with provable guarantees [22]. They propose the Iterative Mistake Minimization (IMM) algorithm and prove that it achieves a worst-case O(k 2 ) approximation to the optimal k-means clustering cost.…”

“…As our main contribution, we propose a novel extension of the previous explainable k-means algorithm. Our method, ExKMC, takes as input two parameters k, k and a set X ⊆ R d with |X | = n. It first builds a threshold tree with k leaves using the IMM algorithm [22]. Then, given a budget of k > k leaves, it greedily expands the tree to reduce the clustering cost.…”

“…To efficiently determine the assignment of leaves to clusters, we design and analyze a surrogate cost. Recall that the IMM algorithm first runs a standard k-means algorithm, producing a set of k centers that are given as an additional input [22]. As ExKMC expands the tree to k > k leaves, it minimizes the cost of the current clustering compared to these k reference centers.…”