Conditional sum-product networks: Modular probabilistic circuits via gate functions

Shao, Xiaoting; Molina, Alejandro; Vergari, Antonio; Stelzner, Karl; Peharz, Robert; Liebig, Thomas; Kersting, Kristian

doi:10.1016/j.ijar.2021.10.011

Cited by 9 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The limited expressive power of sum units was also noted by Shao et al (2022), which led them to introduce conditional sum-product networks. The idea is to condition the weights of the sum units in a probabilistic circuit on the value of a random value (using neural networks).…”

Section: Related Workmentioning

confidence: 99%

Probabilistic Neural Circuits

Zuidberg Dos Martires

2024

AAAI

View full text Add to dashboard Cite

Probabilistic circuits (PCs) have gained prominence in recent years as a versatile framework for discussing probabilistic models that support tractable queries and are yet expressive enough to model complex probability distributions. Nevertheless, tractability comes at a cost: PCs are less expressive than neural networks. In this paper we introduce probabilistic neural circuits (PNCs), which strike a balance between PCs and neural nets in terms of tractability and expressive power. Theoretically, we show that PNCs can be interpreted as deep mixtures of Bayesian networks. Experimentally, we demonstrate that PNCs constitute powerful function approximators.

show abstract

Section: Related Workmentioning

confidence: 99%

Probabilistic Neural Circuits

Zuidberg Dos Martires

2024

AAAI

View full text Add to dashboard Cite

show abstract

“…As proposed by Shao et al [64], any (smooth and decomposable) PC q Θ (Y) encoding a joint distribution over the labels Y can be turned into a (smooth and decomposable) conditional circuit, conditioned by input variables X, by letting its parameters be a function of X. Definition 3.4 (Neural conditional circuits [63]).…”

Section: Definition 33 (Smoothness and Decomposabilitymentioning

confidence: 99%

Semantic Probabilistic Layers for Neuro-Symbolic Learning

Ahmed¹,

Teso²,

Chang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints. Our Semantic Probabilistic Layer (SPL) can model intricate correlations, and hard constraints, over a structured output space all while being amenable to end-to-end learning via maximum likelihood. SPLs combine exact probabilistic inference with logical reasoning in a clean and modular way, learning complex distributions and restricting their support to solutions of the constraint. As such, they can faithfully, and efficiently, model complex SOP tasks beyond the reach of alternative neuro-symbolic approaches. We empirically demonstrate that SPLs outperform these competitors in terms of accuracy on challenging SOP tasks including hierarchical multi-label classification, pathfinding and preference learning, while retaining perfect constraint satisfaction.Preprint. Under review.

show abstract

“…On the basis of score matching, Sasaki and Hyvärinen designed a neural-kernelized conditional density estimator (NKC) to estimate the conditional density. A conditional sum-product networks (CSPNs) is proposed (Shao et al 2020) for multivariate and potentially hybrid domains that allow harnessing the expressive power of neural networks while still maintaining tractability guarantees. A neural noise-regularization and data-normalization scheme for CDE was used for financial applications, addressing problems like overfitting, weight initialization, and hyperparameter sensitivity (Rothfuss et al 2019b).…”

Section: Related Workmentioning

confidence: 99%

Deconvolutional Density Network: Modeling Free-Form Conditional Distributions

Chen

Islam

Gao

et al. 2022

AAAI

View full text Add to dashboard Cite

Conditional density estimation (CDE) is the task of estimating the probability of an event conditioned on some inputs. A neural network (NN) can also be used to compute the output distribution for continuous-domain, which can be viewed as an extension of regression task. Nevertheless, it is difficult to explicitly approximate a distribution without knowing the information of its general form a priori. In order to fit an arbitrary conditional distribution, discretizing the continuous domain into bins is an effective strategy, as long as we have sufficiently narrow bins and very large data. However, collecting enough data is often hard to reach and falls far short of that ideal in many circumstances, especially in multivariate CDE for the curse of dimensionality. In this paper, we demonstrate the benefits of modeling free-form conditional distributions using a deconvolution-based neural net framework, coping with data deficiency problems in discretization. It has the advantage of being flexible but also takes advantage of the hierarchical smoothness offered by the deconvolution layers. We compare our method to a number of other density-estimation approaches and show that our Deconvolutional Density Network (DDN) outperforms the competing methods on many univariate and multivariate tasks. The code of DDN is available at https://github.com/NBICLAB/DDN

show abstract

Conditional sum-product networks: Modular probabilistic circuits via gate functions

Cited by 9 publications

References 8 publications

Probabilistic Neural Circuits

Probabilistic Neural Circuits

Semantic Probabilistic Layers for Neuro-Symbolic Learning

Deconvolutional Density Network: Modeling Free-Form Conditional Distributions

Contact Info

Product

Resources

About