Cluster-Aware Compression with Provable K-means Preservation

Abstract

This work rigorously explores the design of clusterpreserving compression schemes for high-dimensional data. We focus on the K-means algorithm and identify conditions under which running the algorithm on the compressed data yields the same clustering outcome as on the original. The compression is performed using single and multi-bit minimum mean square error quantization schemes as well as a given clustering assignment of the original data. We provide theoretical guarantees on post-quantization cluster preservation under certain conditions on the cluster structure, and propose an additional data transformation that can ensure cluster preservation unconditionally; this transformation is invertible and thus induces virtually no distortion on the compressed data. In addition, we provide an efficient scheme for multi-bit allocation, per cluster and data dimension, which enables a trade-off between high compression efficiency and low data distortion. Our experimental studies highlight that the suggested scheme accurately preserved the clusters formed in all cases, while incurring minimal distortion on the data shapes.Our results can find many applications, e.g., in a) clustering, analysis and distribution of massive datasets, where the proposed data compression can boost performance while providing provable guarantees on the clustering result, as well as, in b) cloud computing services, as the optional transformation provides a data-hiding functionality in addition to preserving the K-means clustering outcome.

Compressive mining: fast and optimal data mining in the compressed domain

“…Similar, in spirit, to our approach is the work of [42]. There, the authors propose 1-bit Minimum Mean…”

“…k-Clustering Problem: Given a DB containing V compressed representations of x (i) , ∀i, and a target number of clusters k, group the compressed data into k clusters in an accurate way through their compressed representations. This is an assignment problem and is in fact NPhard [42]. Many approximations to this problem exist, one of the most widely-used algorithms being the k-Means clustering algorith [43].…”

“…This is an assignment problem and is in fact NPhard [42]. Many approximations to this problem exist, one of the most widely-used algorithms being the k-Means clustering algorith [43].…”

“…For centroid initialization, one can choose C (t) to be (i) completely random points in the compressed domain; (ii) set randomly to one of the compressed representations of X (i) ∈ DB, or (iii) use the better performing k-Means++ initialization algorithm[47,42].…”