Learning multi-linear representations of distributions for efficient inference

Abstract

We examine the class of multi-linear representations (MLR) for expressing probability distributions over discrete variables. Recently, MLR have been considered as intermediate representations that facilitate inference in distributions represented as graphical models.We show that MLR is an expressive representation of discrete distributions and can be used to concisely represent classes of distributions which have exponential size in other commonly used representations, while supporting probabilistic inference in time linear in the size of the representation. Our key contribution is presenting techniques for learning bounded-size distributions represented using MLR, which support efficient probabilistic inference. We demonstrate experimentally that the MLR representations we learn support accurate and very efficient inference.

Sum-product networks: A new deep architecture

“…More interestingly, as the previous example shows, SPNs can compactly represent some classes of distributions in which no conditional independences hold. Multi-linear representations (MLRs) also have this property [24]. Since MLRs are essentially expanded SPNs, an SPN can be exponentially more compact than the corresponding MLR.…”

Guest editors’ introduction: special issue of selected papers from ECML PKDD 2009

“…In "Learning multi-linear representations of distributions for efficient inference" (Roth and Samdani 2009) present a method of speeding up probabilistic inference by representing discrete probability distributions using multilinear forms.…”

Tractable Boolean and Arithmetic Circuits

“…A few additional milestones have contributed to broadening the applications of tractable circuits and their expanded role today. First is the learning of tractable arithmetic circuits from data, starting with [71]; see also [91]. Second is handcrafting the structure of these circuits, which started with [88].…”