Orthogonal Procrustes and norm-dependent optimality

Cape, Joshua

doi:10.13001/ela.2020.5009

Cited by 9 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lemma 3.1 is obtained via Theorem 1.4 (Bernstein inequality) in [45]. For comparison, [36] applies Theorem 5.2 [33] to bound A − Ω (see, for example, Eq (14) of [36]) and obtains a bound as C √ ρn for some C > 0. However, C √ ρn is the bound between a regularization of A and Ω as stated in the proof of Theorem 5.2 [33], where such regularization of A is obtained from A with some constraints in Lemmas 4.1 and 4.2 of the supplement material [33] [33] and Theorem 2 [50] as long as ρ ≥ max i,j Ω(i, j) (here, let Ω = E[A] without considering models, a ρ satisfying ρ ≥ max i,j Ω(i, j) is also the sparsity parameter which controls the overall sparsity of a network).…”

Section: Consistency Under Mmsb Our Main Results Under Mmsb Provides ...mentioning

confidence: 99%

A useful criterion on studying consistent estimation in community detection

Qing

2021

Preprint

View full text Add to dashboard Cite

In network analysis, developing a unified theoretical framework that can compare methods under different models is an interesting problem. This paper proposes a partial solution to this problem. We summarize the idea of using separation condition for a standard network and sharp threshold of Erdös-Rényi random graph to study consistent estimation, compare theoretical error rates and requirements on network sparsity of spectral methods under models that can degenerate to stochastic block model as a four-step criterion SCSTC. Using SCSTC, we find some inconsistent phenomena on separation condition and sharp threshold in community detection. Especially, we find original theoretical results of the SPACL algorithm introduced to estimate network memberships under the mixed membership stochastic blockmodel were suboptimal. To find the formation mechanism of inconsistencies, we re-establish theoretical convergence rates of this algorithm by applying recent techniques on row-wise eigenvector deviation. The results are further extended to the degree corrected mixed membership model. By comparison, our results enjoy smaller error rates, lesser dependence on the number of communities, weaker requirements on network sparsity, and so forth. Furthermore, separation condition and sharp threshold obtained from our theoretical results match classical results, which shows the usefulness of this criterion on studying consistent estimation.

show abstract

Section: Consistency Under Mmsb Our Main Results Under Mmsb Provides ...mentioning

confidence: 99%

A useful criterion on studying consistent estimation in community detection

Qing

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…2) is the classic matrix approximation problem in linear algebra, named as the Procrustes problem (Schönemann, 1966;Cape, 2020). The solution to (3.2) is referred to as Orthogonal Procrustes Transformation (OPT) and has a closed form:…”

Section: Privacy-preserving Distributed Svdmentioning

confidence: 99%

“…It implies that as a metric on linear space, dist(U, Ũ ) is equivalent to U − Ũ O * 2 (or min O∈O k U − Ũ O 2 ) up to some universal constant. The optimization problem involved in is named as the orthogonal procrustes problem and has been well studied (Schönemann, 1966;Cape, 2020). • (U, Ũ ) = ( Ũ , U ) for all U, Ũ ∈ O d×k .…”

Section: E Definitions On Subspace Distancementioning

confidence: 99%

See 1 more Smart Citation

Privacy-Preserving Distributed SVD via Federated Power

Guo¹,

Li²,

Chang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Singular value decomposition (SVD) is one of the most fundamental tools in machine learning and statistics. The modern machine learning community usually assumes that data come from and belong to small-scale device users. The low communication and computation power of such devices, and the possible privacy breaches of users' sensitive data make the computation of SVD challenging. Federated learning (FL) is a paradigm enabling a large number of devices to jointly learn a model in a communication-efficient way without data sharing. In the FL framework, we develop a class of algorithms called Fed-Power for the computation of partial SVD in the modern setting. Based on the well-known power method, the local devices alternate between multiple local power iterations and one global aggregation to improve communication efficiency. In the aggregation, we propose to weight each local eigenvector matrix with Orthogonal Procrustes Transformation (OPT). Considering the practical stragglers' effect, the aggregation can be fully participated or partially participated, where for the latter we propose two sampling and aggregation schemes. Further, to ensure strong privacy protection, we add Gaussian noise whenever the communication happens by adopting the notion of differential privacy (DP). We theoretically show the convergence bound for FedPower. The resulting bound is interpretable with each part corresponding to the effect of Gaussian noise, parallelization, and random sampling of devices, respectively. We also conduct experiments to demonstrate the merits of Fed-Power. In particular, the local iterations not only improve communication efficiency but also reduce the chance of privacy breaches.

show abstract

“…For the sake of concreteness, one may without loss of generality take the first entry in each column of b U to be nonnegative. When A exhibits large negative eigenvalues, it is sometimes useful in practice (and for organizing theory) to index the leading eigenvalues alternatively in the mannerλ 1 ≥ λ 2 ≥ … ≥ λ p > 0 and Àλ p + 1 ≥ Àλ p + 2 ≥ … ≥ Àλ d > 0.4 In low-dimensional examples one could alternatively minimize with respect to the maximum absolute entry norm or the maximum Euclidean row norm, as highlighted inCape (2020). Such an approach, necessarily requiring numerical optimization in the absence of a general closed-form analytic solution, would be particularly suitable when certain parametrized, structured orthogonal transformations are of application-specific interest.…”

mentioning

confidence: 99%

Spectral analysis of networks with latent space dynamics and signs

Cape

2021

Stat

Self Cite

View full text Add to dashboard Cite

We pursue the problem of modelling and analysing latent space dynamics in collections of networks. Towards this end, we pose and study latent space generative models for signed networks that are amenable to inference via spectral methods. Permitting signs, rather than restricting to unsigned networks, enables richer latent space structure and permissible dynamic mechanisms that can be provably inferred via low rank truncations of observed adjacency matrices. Our treatment of and ability to recover latent space dynamics holds across different levels of granularity, namely, at the overall graph level, for communities of nodes, and even at the individual node level. We provide synthetic and real data examples to illustrate the effectiveness of methodologies and to corroborate accompanying theory. The contributions set forth in this paper complement an emerging statistical paradigm for random graph inference encompassing random dot product graphs and generalizations thereof.

show abstract

Orthogonal Procrustes and norm-dependent optimality

Cited by 9 publications

References 10 publications

A useful criterion on studying consistent estimation in community detection

A useful criterion on studying consistent estimation in community detection

Privacy-Preserving Distributed SVD via Federated Power

Spectral analysis of networks with latent space dynamics and signs

Contact Info

Product

Resources

About