A Faster Interior Point Method for Semidefinite Programming

Jiang, Haotian; Kathuria, Tarun; Lee, Yin Tat; Padmanabhan, Swati; Song, Zhao

doi:10.1109/focs46700.2020.00089

Cited by 49 publications

(34 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…n blocks of size n × n, and using fast matrix multiplication for each block. Next, using the fact that M (a) is positive semi-definite, the approximate solution X (t) can be computed in time O(n 7/2 log(1/ǫ)) using the interior point SDP solver of [14] (see Lemma A.4 for details). Finally, we can group the vectors a…”

Section: Algorithms For the Worst-case Expected Errormentioning

confidence: 99%

“…The authors note that this SDP can be converted to a more standard SDP by using the Schur complement to re-express the matrix inverse. However this conversion increases the number of constraints in the SDP to at least m. The best known interior point solver [14] has runtime O( √ n(mn 2 + m ω + n ω )) for a general SDP with an n dimensional PSD matrix variable and m constraints. The dominant term in our setting is O( √ nm ω ).…”

Section: The Algorithm Of Chen Valiant and Valiantmentioning

confidence: 99%

See 1 more Smart Citation

Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected Error

Brown-Cohen¹

2021

Preprint

View full text Add to dashboard Cite

The study of statistical estimation without distributional assumptions on data values, but with knowledge of data collection methods was recently introduced by Chen, Valiant and Valiant (NeurIPS 2020). In this framework, the goal is to design estimators that minimize the worst-case expected error. Here the expectation is over a known, randomized data collection process from some population, and the data values corresponding to each element of the population are assumed to be worst-case. Chen, Valiant and Valiant show that, when data values are ℓ ∞normalized, there is a polynomial time algorithm to compute an estimator for the mean with worst-case expected error that is within a factor π 2 of the optimum within the natural class of semilinear estimators. However, their algorithm is based on optimizing a somewhat complex concave objective function over a constrained set of positive semidefinite matrices, and thus does not come with explicit runtime guarantees beyond being polynomial time in the input. In this paper we design provably efficient algorithms for approximating the optimal semilinear estimator based on online convex optimization. In the setting where data values are ℓ ∞normalized, our algorithm achieves a π 2 -approximation by iteratively solving a sequence of standard SDPs. When data values are ℓ 2 -normalized, our algorithm iteratively computes the top eigenvector of a sequence of matrices, and does not lose any multiplicative approximation factor. Further, using experiments in settings where sample membership is correlated with data values (e.g. "importance sampling" and "snowball sampling"), we show that our ℓ 2 -normalized algorithm gives a similar advantage over standard estimators as the original ℓ ∞ -normalized algorithm of Chen, Valiant and Valiant, but with much lower computational complexity. We complement these positive results by stating a simple combinatorial condition which, if satisfied by a data collection process, implies that any (not necessarily semilinear) estimator for the mean has constant worst-case expected error.

show abstract

Section: Algorithms For the Worst-case Expected Errormentioning

confidence: 99%

Section: The Algorithm Of Chen Valiant and Valiantmentioning

confidence: 99%

Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected Error

Brown-Cohen¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, semi-definite programming (SDP) relaxations [27,23,18,13,11,29,14,9] have emerged as an important approach for clustering due to its superior empirical performance [27], robustness against outliers and adversarial attack [13], and attainment of the information-theoretic limit [10]. Despite having polynomial time complexity, the SDP relaxed Kmeans has notoriously poor scalability to large (or even moderate) datasets for instance by interior point methods [3,15]. Hence the goal of this paper is to derive a computationally cheap approximation to the SDP relaxed K-means formulation for reducing the time complexity while maintaining statistical optimality.…”

Section: Introductionmentioning

confidence: 99%

“…Thus the proposed SL approach has an overall linear time complexity as long as m = O(n c ) for some constant c ∈ (0, 1), which substantially mitigates the high polynomial runtime complexity of solving the original SDP relaxed K-means. For instance, we can set c = 2/7 if the interior point method is used to solve the SDP [15].…”

Section: Introductionmentioning

confidence: 99%

Sketch-and-Lift: Scalable Subsampled Semidefinite Program for $K$-means Clustering

Zhuang¹,

Chen²,

Yang³

2022

Preprint

View full text Add to dashboard Cite

Semidefinite programming (SDP) is a powerful tool for tackling a wide range of computationally hard problems such as clustering. Despite the high accuracy, semidefinite programs are often too slow in practice with poor scalability on large (or even moderate) datasets. In this paper, we introduce a linear time complexity algorithm for approximating an SDP relaxed K-means clustering. The proposed sketch-and-lift (SL) approach solves an SDP on a subsampled dataset and then propagates the solution to all data points by a nearest-centroid rounding procedure. It is shown that the SL approach enjoys a similar exact recovery threshold as the K-means SDP on the full dataset, which is known to be information-theoretically tight under the Gaussian mixture model. The SL method can be made adaptive with enhanced theoretic properties when the cluster sizes are unbalanced. Our simulation experiments demonstrate that the statistical accuracy of the proposed method outperforms state-of-the-art fast clustering algorithms without sacrificing too much computational efficiency, and is comparable to the original K-means SDP with substantially reduced runtime.

show abstract

“…The interior point methods [116] for solving semidenite programming problem perform very well in practice and have the worst case polynomial complexity. For example, the computational complexity of the faster interior point method given by [117] for solving (P3.2) is O( √ n(mn 2 ) log(1/η)), where n = N i is the size of sensor measurements, m = S + 2 is the number of constraints and η is the relative accuracy.…”

mentioning

confidence: 99%

Privacy preservation for linear and learning based inference systems

Wang¹

View full text Add to dashboard Cite

In the activities of data sharing and decentralized processing, data belonging to a user need to be transmitted to a third-party data processor or aggregator. However, the third-party unit may be curious about some private information hidden in the unprocessed data sent by the user, which poses tremendous threat to the data owner's privacy. Therefore, it becomes imperative for the data owner to sanitize its raw data to limit the amount of private information before disclosing it. In the meantime, the data owner hopes to maximally preserve the utility of the sanitized data in terms of a legitimate task that she wishes to perform on the third-party unit. In this thesis, we investigate the problem of protecting parameter estimation in a decentralized linear system and a linear dynamical system with known system models. Apart from that, we explore data-driven approaches to protect the private information carried by raw data that serve as an input to a learning-based inference system.In the rst place, we consider a multi-agent system where each agent in a network makes a local observation that is linearly related to a set of public and private parameters. The agents send their observations to a fusion center to allow it to estimate the public parameters. To prevent leakage of the private parameters, each agent rst sanitizes its local observation using a local privacy mechanism before transmitting it to the fusion center. We study the utility-privacy tradeo in terms of the Cramér-Rao lower bounds for estimating the public and private parameters, and compare the class of privacy mechanisms given by linear compression and noise perturbation. We derive necessary and sucient conditions for achieving arbitrarily strong privacy without compromising utility, and provide a method to maximize privacy while keeping utility intact. Finally, we propose an algorithm to optimize the utility-privacy tradeo for satisfying a maximum allowable privacy leakage.As a further development of the rst case, we consider a linear dynamical system in which the state vector is composed of public and private parts and progresses xiii xiv according to a known transition model. Multiple sensors make a series of measurements of the state vector and send the data to a fusion center who is authorized to perform state estimation for the public states. To prevent the fusion center from estimating the private states with good accuracy, each sensor's measurements at each time step are linearly compressed into a lower dimensional space before being sent to the fusion center. We take account of the fact that the public states at one time step may provide statistical information about the private states in a future time step, and propose a formulation that can ensure the same level of privacy in all future time steps. We develop online algorithms to nd the optimal compression matrix for the utility-privacy tradeo.Finally, we consider a practical setup where a precise system model cannot be derived analytically due to the complexity of the underlying data. Thus...

show abstract

A Faster Interior Point Method for Semidefinite Programming

Cited by 49 publications

References 33 publications

Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected Error

Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected Error

Sketch-and-Lift: Scalable Subsampled Semidefinite Program for $K$-means Clustering

Privacy preservation for linear and learning based inference systems

Contact Info

Product

Resources

About