Learning Graphical Models Using Multiplicative Weights

Klivans, Adam R.; Meka, Raghu

doi:10.1109/focs.2017.39

Cited by 69 publications

(124 citation statements)

References 18 publications

Supporting

Mentioning

120

Contrasting

Order By: Relevance

“…In the case of logistic regression, there has been a lot of work showing that under certain hightemperature conditions on the Ising model (which are similar to the assumptions we make in our paper), one can perform many statistical tasks such as learning, testing and sampling of Ising models efficiently [28,14,13,15,25,17].…”

Section: Modeling Dependencementioning

confidence: 70%

Regression from dependent observations

Daskalakis

Dikkala

Panageas

2019

Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates.The assumption that the response variables are independent is, however, too strong. In many applications, these responses are collected on nodes of a network, or some spatial or temporal domain, and are dependent. Examples abound in financial and meteorological applications, and dependencies naturally arise in social networks through peer effects. Regression with dependent responses has thus received a lot of attention in the Statistics and Economics literature, but there are no strong consistency results unless multiple independent samples of the vectors of dependent responses can be collected from these models. We present computationally and statistically efficient methods for linear and logistic regression models when the response variables are dependent on a network. Given one sample from a networked linear or logistic regression model and under mild assumptions, we prove strong consistency results for recovering the vector of coefficients and the strength of the dependencies, recovering the rates of standard regression under independent observations. We use projected gradient descent on the negative log-likelihood, or negative log-pseudolikelihood, and establish their strong convexity and consistency using concentration of measure for dependent random variables.

show abstract

Section: Modeling Dependencementioning

confidence: 70%

Regression from dependent observations

Daskalakis

Dikkala

Panageas

2019

Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

show abstract

“…In any case, what this means for our lower bound is that without ferromagneticity, even RBMs with a constant number of latent variables of constant degree inherits the hardness results of learning MRFs [7,24], that in turn follow from the popular assumption that learning sparse parities with noise is hard. For comparison, the technique used in [29] seems insufficient for this reduction -their method can only build certain noiseless functions.…”

Section: Our Resultsmentioning

confidence: 99%

“…In order to learn the two-hop structure of an RBM it will be necessary to have lower and upper bounds on the edge weights of the model, so we introduce the following notion of degeneracy. This is a standard assumption in the literature on learning Ising models [6,40,24]. In particular, a lower bound is needed because otherwise it would be impossible to distinguish a non-edge from an edge with an arbitrarily weak interaction.…”

Section: The Learning Problem For Rbmsmentioning

confidence: 99%

“…In this subsection, we discuss some important obstacles to directly using regression-based methods (in particular [24]) for learning the parameters of a ferromagnetic RBM. By Lemma 4.3, we can cast the problem of learning N 2 (i) for each node i as a structure learning problem on the induced MRF.…”

Section: Maximal Coefficients Can Be Arbitrarily Smallmentioning

confidence: 99%

“…By Lemma 4.3, we can cast the problem of learning N 2 (i) for each node i as a structure learning problem on the induced MRF. In order to use the results of Klivans and Meka [24], we need to get bounds on the potential…”

Section: Maximal Coefficients Can Be Arbitrarily Smallmentioning

confidence: 99%

See 2 more Smart Citations

Learning restricted Boltzmann machines via influence maximization

Bresler

Koehler

Moitra

2019

Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

Graphical models are a rich language for describing high-dimensional distributions in terms of their dependence structure. While there are algorithms with provable guarantees for learning undirected graphical models in a variety of settings, there has been much less progress in the important scenario when there are latent variables. Here we study Restricted Boltzmann Machines (or RBMs), which are a popular model with wide-ranging applications in dimensionality reduction, collaborative filtering, topic modeling, feature extraction and deep learning.The main message of our paper is a strong dichotomy in the feasibility of learning RBMs, depending on the nature of the interactions between variables: ferromagnetic models can be learned efficiently, while general models cannot. In particular, we give a simple greedy algorithm based on influence maximization to learn ferromagnetic RBMs with bounded degree. In fact, we learn a description of the distribution on the observed variables as a Markov Random Field. Our analysis is based on tools from mathematical physics that were developed to show the concavity of magnetization. Our algorithm extends straighforwardly to general ferromagnetic Ising models with latent variables.Conversely, we show that even for a contant number of latent variables with constant degree, without ferromagneticity the problem is as hard as sparse parity with noise. This hardness result is based on a sharp and surprising characterization of the representational power of bounded degree RBMs: the distribution on their observed variables can simulate any bounded order MRF. This result is of independent interest since RBMs are the building blocks of deep belief networks.Lemma 6.2. Suppose X i is the spin at vertex i in an (α, β)-nondegenerate Ising model and j is a neighbor of i. Then for any fixing x =i,j of the other spins X i =j of the Ising model, we haveSince tanh (x) = 1 − tanh 2 (x) and tanh is a monotone function, we see that if we let x = −J ij + k:k / ∈{i,j} J ik x k , then since x ∈ [−β, β] we have | tanh(x + 2J ij ) − tanh(x)| ≥ 2|J ij | inf x∈[−β,β](1 − tanh 2 (x)) ≥ 2α(1 − tanh 2 (β)) .

show abstract

A quantum online portfolio optimization algorithm

Lim,

Rebentrost

2024

Quantum Inf Process

View full text Add to dashboard Cite

Portfolio optimization plays a central role in finance to obtain optimal portfolio allocations that aim to achieve certain investment goals. Over the years, many works have investigated different variants of portfolio optimization. Portfolio optimization also provides a rich area to study the application of quantum computers to obtain advantages over classical computers. In this work, we give a sampling version of an existing classical online portfolio optimization algorithm by Helmbold et al., for which we in turn develop a quantum version. The quantum advantage is achieved by using techniques such as quantum state preparation, inner product estimation and multi-sampling. Our quantum algorithm provides a quadratic speedup in the time complexity, in terms of n, where n is the number of assets in the portfolio. The transaction cost of both of our classical and quantum algorithms is independent of n which is especially useful for practical applications with a large number of assets.

show abstract

Learning Graphical Models Using Multiplicative Weights

Cited by 69 publications

References 18 publications

Regression from dependent observations

Regression from dependent observations

Learning restricted Boltzmann machines via influence maximization

A quantum online portfolio optimization algorithm

Contact Info

Product

Resources

About