Online Variance Minimization

Warmuth, Manfred K.; Kuzmin, Dima

doi:10.1007/11776420_38

Cited by 43 publications

(51 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such linear losses are special because they are the least convex losses. The main case where such linear losses have been investigated is in connection with the Hedge and the Matrix Hedge algorithm (Freund and Schapire 1997;Warmuth and Kuzmin 2011). However for the latter algorithms the parameter space is one-norm or trace norm bounded, respectively.…”

Section: Technical Contributionsmentioning

confidence: 99%

Learning rotations with little regret

2016

Self Cite

View full text Add to dashboard Cite

We describe online algorithms for learning a rotation from pairs of unit vectors in R n . We show that the expected regret of our online algorithm compared to the best fixed rotation chosen offline over T iterations is √ nT . We also give a lower bound that proves that this expected regret bound is optimal within a constant factor. This resolves an open problem posed in COLT 2008. Our online algorithm for choosing a rotation matrix is essentially an incremental gradient descent algorithm over the set of all matrices, with specially tailored projections. We also show that any deterministic algorithm for learning rotations has Ω(T ) regret in the worst case.

show abstract

Section: Technical Contributionsmentioning

confidence: 99%

Learning rotations with little regret

2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…As stated in the text, the proof of QIP = PSPACE presented in this chapter makes use of simplifications due to Wu [171]. The specific formulation of the matrix multiplicative weights update method upon which the proof that QIP = PSPACE relies was discovered independently by Warmuth and Kuzmin [164] and Arora and Kale [16]. Readers interested in learning more about the matrix multiplicative weights update method, as well as a bit of its history, are referred to Kale's PhD thesis [104].…”

Section: Chapter Notesmentioning

confidence: 99%

Quantum Proofs

Vidick

Watrous

2016

FNT in Theoretical Computer Science

View full text Add to dashboard Cite

Quantum information and computation provide a fascinating twist on the notion of proofs in computational complexity theory. For instance, one may consider a quantum computational analogue of the complexity class \class{NP}, known as QMA, in which a quantum state plays the role of a proof (also called a certificate or witness), and is checked by a polynomial-time quantum computation. For some problems, the fact that a quantum proof state could be a superposition over exponentially many classical states appears to offer computational advantages over classical proof strings. In the interactive proof system setting, one may consider a verifier and one or more provers that exchange and process quantum information rather than classical information during an interaction for a given input string, giving rise to quantum complexity classes such as QIP, QSZK, and QMIP* that represent natural quantum analogues of IP, SZK, and MIP. While quantum interactive proof systems inherit some properties from their classical counterparts, they also possess distinct and uniquely quantum features that lead to an interesting landscape of complexity classes based on variants of this model. In this survey we provide an overview of many of the known results concerning quantum proofs, computational models based on this concept, and properties of the complexity classes they define. In particular, we discuss non-interactive proofs and the complexity class QMA, single-prover quantum interactive proof systems and the complexity class QIP, statistical zero-knowledge quantum interactive proof systems and the complexity class \class{QSZK}, and multiprover interactive proof systems and the complexity classes QMIP, QMIP*, and MIP*.Comment: Survey published by NOW publisher

show abstract

“…The Online PCA Algorithm in (Warmuth & Kuzmin, 2006b) uses this idea and has good worst-case loss bounds. It uses the density matrix update based on matrix logs and exponentials like the one used in (Tsuda et al, 2005;Warmuth & Kuzmin, 2006a;Arora & Kale, 2007) but with the additional constraint that the eigenvalues are upper bounded (capped). Density matrices are symmetric, positive definite matrices of trace one.…”

Section: Online Kernel Pca Algorithmmentioning

confidence: 99%

“…More recently the entropic family of updates has been generalized to the case when the instances are symmetric matrices X and the parameter is a density matrix (Tsuda et al, 2005;Warmuth & Kuzmin, 2006a;Warmuth & Kuzmin, 2006b;Arora & Kale, 2007). The regularization is now the quantum relative entropy for density matrices instead of the regular relative entropy for probability vectors, and the matrix logarithm of the density matrix parameter is essentially a linear combination of the instance matrices.…”

Section: Introductionmentioning

confidence: 99%

Online kernel PCA with entropic matrix updates

Kuzmin

Warmuth

2007

Proceedings of the 24th International Conference on Machine Learning

Self Cite

View full text Add to dashboard Cite

A number of updates for density matrices have been developed recently that are motivated by relative entropy minimization problems. The updates involve a softmin calculation based on matrix logs and matrix exponentials. We show that these updates can be kernelized. This is important because the bounds provable for these algorithms are logarithmic in the feature dimension (provided that the 2-norm of feature vectors is bounded by a constant). The main problem we focus on is the kernelization of an online PCA algorithm which belongs to this family of updates.

show abstract

Online Variance Minimization

Cited by 43 publications

References 30 publications

Learning rotations with little regret

Learning rotations with little regret

Quantum Proofs

Online kernel PCA with entropic matrix updates

Contact Info

Product

Resources

About