A Split-Merge DP-means Algorithm to Avoid Local Minima

Abstract

We present an extension of the DP-means algorithm, a hardclustering approximation of nonparametric Bayesian models. Although a recent work [6] reports that the DP-means can converge to a local minimum, the condition for the DP-means to converge to a local minimum is still unknown. This paper demonstrates one reason the DP-means converges to a local minimum: the DP-means cannot assign the optimal number of clusters when many data points exist within small distances. As a first attempt to avoid the local minimum, we propose an extension of the DP-means by the split-merge technique. The proposed algorithm splits clusters when a cluster has many data points to assign the number of clusters near to optimal. The experimental results with multiple datasets show the robustness of the proposed algorithm.

Generalized Dirichlet-Process-Means for Robust and Maximum Distortion Criteria

“…Proof. We show that the objective function (4) monotonically decreases when the k-th cluster center θ k is newly updated toθ k by (9). More specifi-…”

“…which is the weighted mean of x i weighted by f with the Bregman divergence as its argument. However, the objective function monotonically decreases by this update rule (9) only when the function f is concave (or linear). If function f is convex, the cluster center is updated by gradient descent optimization such as the steepest gradient or Newton's method and so on.…”

“…. , n) 6 update θ 1 by (9) until the decrease ofL f (θ 1 ) becomes smaller than the threshold δ repeat calculate w ik by (10) (i = 1, . .…”

“…. , n) 17 update θ k by (9) until the decrease ofL f (θ k ) becomes smaller than the threshold δ until (4) convergences the monotonic decreasing property of the objective function in the cluster center updating step is considered. In the following, we explain the two cases, concave and convex.…”

“…DP-means specialized for application to large scale genetic data has been devised, and shown to be superior to existing methods from the aspect both of accuracy and efficiency [8]. To improve the accuracy of clustering, studies have been made to avoid local minimum solutions [9]. An extension using Bregman divergence was also given, which introduces an appropriate distance measure when data has a special type such as binary or non-negative integer value [10,11].…”