SOS-SDP: an Exact Solver for Minimum Sum-of-Squares Clustering

Abstract

The minimum sum-of-squares clustering problem (MSSC) consists in partitioning n observations into k clusters in order to minimize the sum of squared distances from the points to the centroid of their cluster. In this paper, we propose an exact algorithm for the MSSC problem based on the branch-andbound technique. The lower bound is computed by using a cutting-plane procedure where valid inequalities are iteratively added to the Peng-Wei SDP relaxation. The upper bound is computed with the constrained version of k-means where the initial centroids are extracted from the solution of the SDP relaxation. In the branch-and-bound procedure, we incorporate instance-level must-link and cannot-link constraints to express knowledge about which instances should or should not be grouped together. We manage to reduce the size of the problem at each level preserving the structure of the SDP problem itself. The obtained results show that the approach allows to successfully solve for the first time real-world instances up to 4000 data points.

An Exact Algorithm for Semi-supervised Minimum Sum-of-Squares Clustering

“…All these approaches use general optimization tools, such as integer programming or constraint programming, and they search for a global optimum that satisfies all the constraints but can only solve instances with limited number of data points. Global optimization algorithms proposed in literature for unconstrained MSSC are based on cutting plane (Sherali & Desai, 2005;Peng & Xia, 2005), branch-and-bound (Koontz et al, 1975;Diehr, 1985;Brusco, 2006), branch-and-cut (Aloise & Hansen, 2009Piccialli et al, 2021) and column generation algorithms (Du Merle et al, 1999;Aloise et al, 2012a).…”

“…The proof of equivalence between ( 6) and ( 7) can be easily derived by using Theorem 2 in Piccialli et al (2021). Note that in Piccialli et al (2021), ML and CL constraints are added one at a time when visiting the branch-and-bound tree, since the children are generated either by merging two points thanks to a ML or adding the corresponding CL constraint.…”

“…The proof of equivalence between ( 6) and ( 7) can be easily derived by using Theorem 2 in Piccialli et al (2021). Note that in Piccialli et al (2021), ML and CL constraints are added one at a time when visiting the branch-and-bound tree, since the children are generated either by merging two points thanks to a ML or adding the corresponding CL constraint. Therefore, the size of the matrix is reduced by one at each branching level, and the effect of the size reduction is essentially negligible thanks to the small number of visited nodes.…”

“…As in SOS-SDP, for pair and triangle inequalities, we randomly separate at most 100000 valid cuts, we sort them in decreasing order with respect to the violation, and we add the first 5% of violated ones in the current cuttingplane iteration. For the separation of clique inequalities, we use the heuristic procedure described in Piccialli et al (2021) that returns at most n valid cuts. The tolerance for checking the violation is set to 10 −4 .…”

“…1. We define the first SDP-based branch-and-bound method for solving semi-supervised MSSC, that is a nontrivial extension of SOS-SDP, a large-scale exact algorithm for unconstrained MSSC recently proposed by Piccialli et al (2021).…”