The Effectiveness of Lloyd-Type Methods for the k-Means Problem

Abstract

We investigate variants of Lloyd's heuristic for clustering high dimensional data in an attempt to explain its popularity (a half century after its introduction) among practitioners, and in order to suggest improvements in its application. We propose and justify a clusterability criterion for data sets. We present variants of Lloyd's heuristic that quickly lead to provably near-optimal clustering solutions when applied to well-clusterable instances. This is the first performance guarantee for a variant of Lloyd's heuristic. The provision of a guarantee on output quality does not come at the expense of speed: some of our algorithms are candidates for being faster in practice than currently used variants of Lloyd's method. In addition, our other algorithms are faster on well-clusterable instances than recently proposed approximation algorithms, while maintaining similar guarantees on clustering quality. Our main algorithmic contribution is a novel probabilistic seeding process for the starting configuration of a Lloyd-type iteration.

Fast indefinite multi-point (IMP) clustering

“…This considerably decreases the flop count of algorithms that try to minimize the above expression, as there are many fewer terms involved if K M. There are many algorithms that directly or indirectly try to minimize the above expression over the K columns Y j . However it is difficult to the find the global minimum and the quality of the local minimum may not be good, though there does not necessarily seem to be agreement over this in the literature, as the precise local minima at which the algorithm stops depends on the starting point [13,14,18].…”

Two density-based k-means initialization algorithms for non-metric data clustering

“…Different techniques have been proposed in the literature for choosing those initial representatives (also called seeds or seeding prototypes) [1,16,30]. In the following, we present some initialization techniques that are considered in the experimental part of this paper as reference systems for performance evaluations.…”

“…It is worth stressing that GD is originally conceived to operate in R n [30]. Therefore, for the purpose of this paper, in the experiment we will adapt the algorithm by idealizing the Voronoi regions by means of the MinSOD cluster representation (4).…”

“…Greedy deletion [30]. The greedy deletion (GD) strategy initially defines the representatives asĈ ¼ S, where S is the input dataset.…”

“…Sample seeding [30]. In the sample seeding (SS) technique, the two most dissimilar patterns in the dataset are selected as the first two temporary representatives.…”