Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration

Nguyen, Thanh Huy; Şimşekli, Umut; Richard, Gaël; Cemgil, Ali Taylan

doi:10.1109/lsp.2018.2822550

Cited by 2 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computational complexity of the algorithm scales with the total sum of the elements in the observed tensor to be decomposed and the treewidth of the graph G that defines the mark distribution. Unlike other approaches that are based on standard techniques such as non-convex optimization, variational Bayes, or Markov chain Monte Carlo (Acar and Yener, 2009;S ¸imşekli and Cemgil, 2012;Ermis et al, 2014;Nguyen et al, 2018), the proposed algorithm does not depend on the size of the observed tensor. This is a direct consequence of the junction tree factorization that implies also a factorized representation of the Pólya urn.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns

Cemgil,

Kurutmaz,

Yildirim

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce a dynamic generative model, Bayesian allocation model (BAM), which establishes explicit connections between nonnegative tensor factorization (NTF), graphical models of discrete probability distributions and their Bayesian extensions, and the topic models such as the latent Dirichlet allocation. BAM is based on a Poisson process, whose events are marked by using a Bayesian network, where the conditional probability tables of this network are then integrated out analytically. We show that the resulting marginal process turns out to be a Pólya urn, an integer valued self-reinforcing process. This urn processes, which we name a Pólya-Bayes process, obey certain conditional independence properties that provide further insight about the nature of NTF. These insights also let us develop space efficient simulation algorithms that respect the potential sparsity of data: we propose a class of sequential importance sampling algorithms for computing NTF and approximating their marginal likelihood, which would be useful for model selection. The resulting methods can also be viewed as a model scoring method for topic models and discrete Bayesian networks with hidden variables. The new algorithms have favourable properties in the sparse data regime when contrasted with variational algorithms that become more accurate when the total sum of the elements of the observed tensor goes to infinity. We illustrate the performance on several examples and numerically study the behaviour of the algorithms for various data regimes.

show abstract

Section: Introductionmentioning

confidence: 99%