Learning in Implicit Generative Models

Mohamed, Shakir; Lakshminarayanan, Balaji

doi:10.48550/arxiv.1610.03483

Cited by 139 publications

(191 citation statements)

References 37 publications

Supporting

Mentioning

183

Contrasting

Order By: Relevance

“…As we will show, many generic implicit generative modelling algorithms -i.e. those which are not designed to exploit a particular structure in the target distribution class -are of this type, including those for training quantum circuit Born machines [LW18a,CMDK20,ML17]. As such, hardness results in the SQ model apply to many implicit generative modelling algorithms of practical interest, and in particular to those which are often used for the concept class of interest in this work.…”

Section: Overview Of This Workmentioning

confidence: 99%

Learnability of the output distributions of local quantum circuits

Hinsche¹,

Ioannou²,

Nietner³

et al. 2021

Preprint

View full text Add to dashboard Cite

in the SQ model is often taken as strong evidence for hardness in the sample model.In summary, we study in this work the following problems, which are stated more formally in Section 3:Problems: PAC probabilistic modelling of quantum circuit Born machines (informal). Let C be the set of output distributions corresponding to a class of local quantum circuits. Given either sample-oracle or SQ-oracle access to some unknown distribution P ∈ C, output, with high probability, either generative modelling: an efficient generator, or density modelling: an efficient evaluator for a distribution P which is sufficiently close to P .If there exists either a sample or computationally efficient algorithm which, with respect to either the sample oracle or the SQ oracle, solves the generative (density) modelling problem associated with a given set of distributions C, then we say that C is sample or computationally efficiently generator (evaluator) learnable within the relevant oracle model. We are particularly interested in this work in establishing the existence or non-existence, of efficient quantum or classical learning algorithms, for the output distributions of various classes of local quantum circuits, within both the sample and statistical query model. Main resultsGiven this motivation and context, we provide two main results, which stated informally, are as follows:Result 1 (Informal version of Corollary 1). The concept class consisting of the output distributions of super-logarithmic depth nearest neighbour Clifford circuits is not sample efficiently PAC generator-learnable or evaluator-learnable, in the statistical query model.Result 2 (Informal version of Theorem 2). The concept class consisting of the output distributions of nearest neighbour Clifford circuits is both sample and computationally efficiently classically PAC generator-learnable and evaluator-learnable, in the sample model.

show abstract

Section: Overview Of This Workmentioning

confidence: 99%

Learnability of the output distributions of local quantum circuits

Hinsche¹,

Ioannou²,

Nietner³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Suppose, in addition, that one has access to an exact binary classifier d * (x), which outputs the probability that the sample x originated from q θ (x). Then, assuming uniform prior probabilities for the two classes, it is straightforward to show via Bayes' theorem that (see Section II B in [50])…”

Section: Adversarial Generative Modelling With F -Divergencesmentioning

confidence: 99%

“…However, under the assumption that we can efficiently sample from both distributions, we can train a classifier d φ (x), parameterised by φ, to distinguish between the two distributions. One can use any proper scoring rule to train the classifier [50]. A typical choice is the negative cross entropy, given by…”

Section: Adversarial Generative Modelling With F -Divergencesmentioning

confidence: 99%

See 1 more Smart Citation

F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits

Leadbeater,

Sharrock,

Coyle

et al. 2021

Preprint

View full text Add to dashboard Cite

Generative modelling is an important unsupervised task in machine learning. In this work, we study a hybrid quantum-classical approach to this task, based on the use of a quantum circuit Born machine. In particular, we consider training a quantum circuit Born machine using f -divergences. We first discuss the adversarial framework for generative modelling, which enables the estimation of any f -divergence in the near term. Based on this capability, we introduce two heuristics which demonstrably improve the training of the Born machine. The first is based on f -divergence switching during training. The second introduces locality to the divergence, a strategy which has proved important in similar applications in terms of mitigating barren plateaus. Finally, we discuss the long-term implications of quantum devices for computing f -divergences, including algorithms which provide quadratic speedups to their estimation. In particular, we generalise existing algorithms for estimating the Kullback-Leibler divergence and the total variation distance to obtain a fault-tolerant quantum algorithm for estimating another f -divergence, namely, the Pearson divergence.

show abstract

“…Compared with VI that uses explicit density models, PIS uses an implicit model and has the advantage of free-form network design. The explicit density models have weaker expressive power and flexibility compared with implicit models, both theoretically and empirically (Cornish et al, 2020;Chen et al, 2019;Kingma & Welling, 2013;Mohamed & Lakshminarayanan, 2016). Compared with MCMC, PIS is more efficient and is able to generate high-quality samples with fewer steps.…”

Section: Introductionmentioning

confidence: 99%

Path Integral Sampler: a stochastic control approach for sampling

Zhang

Chen

2021

Preprint

View full text Add to dashboard Cite

We present Path Integral Sampler (PIS), a novel algorithm to draw samples from unnormalized probability density functions. The PIS is built on the Schrödinger bridge problem which aims to recover the most likely evolution of a diffusion process given its initial distribution and terminal distribution. The PIS draws samples from the initial distribution and then propagates the samples through the Schrödinger bridge to reach the terminal distribution. Applying the Girsanov theorem, with a simple prior diffusion, we formulate the PIS as a stochastic optimal control problem whose running cost is the control energy and terminal cost is chosen according to the target distribution. By modeling the control as a neural network, we establish a sampling algorithm that can be trained end-to-end. We provide theoretical justification of the sampling quality of PIS in terms of Wasserstein distance when sub-optimal control is used. Moreover, the path integrals theory is used to compute importance weights of the samples to compensate for the bias induced by the sub-optimality of the controller and time-discretization. We experimentally demonstrate the advantages of PIS compared with other startof-the-art sampling methods on a variety of tasks.

show abstract

Learning in Implicit Generative Models

Cited by 139 publications

References 37 publications

Learnability of the output distributions of local quantum circuits

Learnability of the output distributions of local quantum circuits

F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits

Path Integral Sampler: a stochastic control approach for sampling

Contact Info

Product

Resources

About