Phase Transition Structure of Variational Bayesian Nonnegative Matrix Factorization

Kohjima, Masahiro; Watanabe, Sumio

doi:10.1007/978-3-319-68612-7_17

Cited by 3 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VBNMF has been devised [4], and the exact learning coefficient of VBNMF has been derived [13]. Nevertheless, the variational approximation accuracy has not been clarified since the RLCT of NMF has been unknown.…”

Section: Learning Theory Of Bayesian and Variational Inferencementioning

confidence: 99%

“…In such cases, decomposition of a set of matrices results in statistical inference. We consider this situation and it is common to [13] and Theorem 2.1. A statistical model p(X|U, V ) which has parameters (U, V ) are employed for estimation.…”

Section: Main Theoremsmentioning

confidence: 99%

“…On the other hand, for several statistical models, the variational free energy was proved that it asymptotically equals λ vb log n + O p (1),where λ vb is a learning coefficient and it depends on the model. Normal mixture models [17], hidden Markov models [10], and NMF [13] are such examples. In general, the learning coefficient of VB may not be equal to but becomes an upper bound of the RLCT: λ vb ≧ λ, since the variational free energy is larger than the usual free energy even if the sample size diverges infinity.…”

Section: Learning Theory Of Bayesian and Variational Inferencementioning

confidence: 99%

“…From the theoretical point of view, Fn has an asymptotic behavior which is similar to the one of F n . For example, in VBNMF, the following theorem is proved [13].…”

Section: Variational Bayesian Algorithmmentioning

confidence: 99%

See 3 more Smart Citations

Variational approximation error in non-negative matrix factorization

Hayashi

2020

Neural Networks

View full text Add to dashboard Cite

Non-negative matrix factorization (NMF) is a knowledge discovery method that is used for many fields, besides, its variational inference and Gibbs sampling method are also well-known. However, the variational approximation accuracy is not yet clarified, since NMF is not statistically regular and the prior used in the variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference of the negative log evidence (free energy) between VBNMF and Bayesian NMF, and give a lower bound of the approximation accuracy, asymptotically. The results quantitatively show how well the VBNMF algorithm can approximate Bayesian NMF.

show abstract

Section: Learning Theory Of Bayesian and Variational Inferencementioning

confidence: 99%

Section: Main Theoremsmentioning

confidence: 99%

Section: Learning Theory Of Bayesian and Variational Inferencementioning

confidence: 99%

“…From the theoretical point of view, Fn has an asymptotic behavior which is similar to the one of F n . For example, in VBNMF, the following theorem is proved [13].…”

Section: Variational Bayesian Algorithmmentioning

confidence: 99%

See 2 more Smart Citations