A local search approximation algorithm for k-means clustering

Abstract

In k-means clustering we are given a set of n data points in d-dimensional space d and an integer k, and the problem is to determine a set of k points in d , called centers, to minimize the mean squared distance from each data point to its nearest center. No exact polynomial-time algorithms are known for this problem. Although asymptotically efficient approximation algorithms exist, these algorithms are not practical due to the very high constant factors involved. There are many heuristics that are used in practice, but we know of no bounds on their performance. We consider the question of whether there exists a simple and practical approximation algorithm for k-means clustering. We present a local improvement heuristic based on swapping centers in and out. We prove that this yields a (9 + ε)-approximation algorithm. We present an example showing that any approach based on performing a fixed number of swaps achieves an approximation factor of at least (9 − ε) in all sufficiently high dimensions. Thus, our approximation factor is almost tight for algorithms based on performing a fixed number of swaps. To establish the practical value of the heuristic, we present an empirical study that shows that, when combined with Lloyd's algorithm, this heuristic performs quite well in practice.

k-Means Walk: Unveiling Operational Mechanism of a Popular Clustering Approach for Microarray Data

“…In k-means clustering, we are given a set of n data points in d-dimensional space, R d and an integer k. The problem is to determine a set of k points m j , j=1,2,3,….k , in R d , called centers, to minimize the mean squared distance from each data point to its nearest center [30]. The objective function is:…”

The Planar k-Means Problem is NP-Hard

“…There are polynomial time algorithms that compute a constant factor approximation to the optimal solution; see for instance the local search algorithm analyzed by Kanungo et al [13]. If k, the number of centers, is a fixed constant, then the problem admits polynomial-time approximation schemes [8,14].…”

An approximation ratio for biclustering

“…This formulation is the objective of the popular K-means algorithm (see, for example, [9]), The optimal biclustering of X consists of {R * 1 , R * 2 } = {{1}, {2, 3, 4}} row clusters and {C * 1 , C * 2 , C * 3 } = {{b, f }, {a, d, e}, {c}} column clusters when using L 1 -norm. (c) Biclusters of the data matrix returned by our scheme, that is, using twice an optimal one-way clustering algorithm, once on the 4 row vectors and another on the 6 column vectors, with L 1 -norm.…”