Initializing k-Means Efficiently: Benefits for Exploratory Cluster Analysis

Fritz, Manuel; Schwarz, Holger

doi:10.1007/978-3-030-33246-4_9

Cited by 3 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The contributions of this paper are partially based on our previous works to address C1 [23] and C2 [24,25] independent from each other. We significantly extend our prior work by presenting additional contributions related to C3, i.e., exploratory clustering analyses in large-scale exploration processes.…”

Section: Our Contributionsmentioning

confidence: 99%

“…Our approach Delta Initialization efficiently initializes centroid-based clustering algorithms in exploration processes [24]. The underlying idea is straightforward: Instead of performing a time-consuming initialization via k-Means++ or k-Means for each execution of a centroid-based clustering algorithm throughout an exploration process, Delta Initialization re-uses previous clustering results.…”

Section: Delta Initialization (Delta)mentioning

confidence: 99%

“…Therefore, Delta Initialization tremendously reduces the runtime for the repetitive initialization in exploration processes on large datasets. Furthermore, we showed in our previous work that Delta Initialization achieves even more valuable clustering results than initializing via k-Means [24], since it exploits previous clustering results, where the position of the centroids are (local) optima, which only emerge after the execution of the clustering algorithm.…”

Section: Delta Initialization (Delta)mentioning

confidence: 99%

“…The most accurate results for each estimation method are achieved with different k-Means methods, yet in most cases for DELTA+GQA99. The reason for these improvements in accuracy are twofold: a) Regarding the benefits of DELTA, we unveiled in our previous paper that it leads to better separated clusters [24], which may be favorable for specific estimation methods. Furthermore, DELTA draws on less randomness compared to an initialization from scratch for each k ∈ R, since DELTA re-uses previous clustering results and adds Δk centroids in a deterministic manner.…”

Section: Accuracymentioning

confidence: 99%

See 3 more Smart Citations

Efficient exploratory clustering analyses in large-scale exploration processes

et al. 2021

Self Cite

View full text Add to dashboard Cite

Clustering is a fundamental primitive in manifold applications. In order to achieve valuable results in exploratory clustering analyses, parameters of the clustering algorithm have to be set appropriately, which is a tremendous pitfall. We observe multiple challenges for large-scale exploration processes. On the one hand, they require specific methods to efficiently explore large parameter search spaces. On the other hand, they often exhibit large runtimes, in particular when large datasets are analyzed using clustering algorithms with super-polynomial runtimes, which repeatedly need to be executed within exploratory clustering analyses. We address these challenges as follows: First, we present LOG-Means and show that it provides estimates for the number of clusters in sublinear time regarding the defined search space, i.e., provably requiring less executions of a clustering algorithm than existing methods. Second, we demonstrate how to exploit fundamental characteristics of exploratory clustering analyses in order to significantly accelerate the (repetitive) execution of clustering algorithms on large datasets. Third, we show how these challenges can be tackled at the same time. To the best of our knowledge, this is the first work which simultaneously addresses the above-mentioned challenges. In our comprehensive evaluation, we unveil that our proposed methods significantly outperform state-of-the-art methods, thus especially supporting novice analysts for exploratory clustering analyses in large-scale exploration processes.

show abstract

Section: Our Contributionsmentioning

confidence: 99%

Section: Delta Initialization (Delta)mentioning

confidence: 99%

Section: Delta Initialization (Delta)mentioning

confidence: 99%

Section: Accuracymentioning

confidence: 99%

See 2 more Smart Citations

Efficient exploratory clustering analyses in large-scale exploration processes

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Learning from Past Observations: Meta-Learning for Efficient Clustering Analyses

Fritz

Tschechlov

Schwarz

2020

Big Data Analytics and Knowledge Discovery

View full text Add to dashboard Cite

LOG-Means

2020

View full text Add to dashboard Cite

Clustering is a fundamental primitive in manifold applications. In order to achieve valuable results, parameters of the clustering algorithm, e.g., the number of clusters, have to be set appropriately, which is a tremendous pitfall. To this end, analysts rely on their domain knowledge in order to define parameter search spaces. While experienced analysts may be able to define a small search space, especially novice analysts often define rather large search spaces due to the lack of in-depth domain knowledge. These search spaces can be explored in different ways by estimation methods for the number of clusters. In the worst case, estimation methods perform an exhaustive search in the given search space, which leads to infeasible runtimes for large datasets and large search spaces. We propose LOG-Means, which is able to overcome these issues of existing methods. We show that LOG-Means provides estimates in sublinear time regarding the defined search space, thus being a strong fit for large datasets and large search spaces. In our comprehensive evaluation on an Apache Spark cluster, we compare LOG-Means to 13 existing estimation methods. The evaluation shows that LOG-Means significantly outperforms these methods in terms of runtime and accuracy. To the best of our knowledge, this is the most systematic comparison on large datasets and search spaces as of today.

show abstract

Initializing k-Means Efficiently: Benefits for Exploratory Cluster Analysis

Cited by 3 publications

References 16 publications

Efficient exploratory clustering analyses in large-scale exploration processes

Efficient exploratory clustering analyses in large-scale exploration processes

Learning from Past Observations: Meta-Learning for Efficient Clustering Analyses

LOG-Means

Contact Info

Product

Resources

About