On the relative expressiveness of Bayesian and neural networks

Choi, Arthur; Wang, Ruocheng; Darwiche, Adnan

doi:10.1016/j.ijar.2019.07.008

Cited by 18 publications

(20 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is known that neural networks are universal approximators, which means that they can approximate any functions to an arbitrary small error. However, Choi et al [ 45 ] showed that the functions induced by BN queries are polynomials. To improve their queries to become universal approximators, they proposed a testing BN, which chooses a parameter value depending on a threshold instead of simply having a fixed parameter value.…”

Section: Discussionmentioning

confidence: 99%

Exact Learning Augmented Naive Bayes Classifier

Sugahara

Ueno

2021

Entropy

View full text Add to dashboard Cite

Earlier studies have shown that classification accuracies of Bayesian networks (BNs) obtained by maximizing the conditional log likelihood (CLL) of a class variable, given the feature variables, were higher than those obtained by maximizing the marginal likelihood (ML). However, differences between the performances of the two scores in the earlier studies may be attributed to the fact that they used approximate learning algorithms, not exact ones. This paper compares the classification accuracies of BNs with approximate learning using CLL to those with exact learning using ML. The results demonstrate that the classification accuracies of BNs obtained by maximizing the ML are higher than those obtained by maximizing the CLL for large data. However, the results also demonstrate that the classification accuracies of exact learning BNs using the ML are much worse than those of other methods when the sample size is small and the class variable has numerous parents. To resolve the problem, we propose an exact learning augmented naive Bayes classifier (ANB), which ensures a class variable with no parents. The proposed method is guaranteed to asymptotically estimate the identical class posterior to that of the exactly learned BN. Comparison experiments demonstrated the superior performance of the proposed method.

show abstract

Section: Discussionmentioning

confidence: 99%

Exact Learning Augmented Naive Bayes Classifier

Sugahara

Ueno

2021

Entropy

View full text Add to dashboard Cite

show abstract

“…Methods for XAI have been developed including locally interpretable modelagnostic explanations for Bayesian predictive models (Peltola 2018) and for convolutional neural networks (Mishra, Sturm, and Dixon 2017), visualization techniques (Grad-CAM) for CNNs (Selvaraju et al 2017), or black box explanations through transparent approximations (Lakkaraju et al 2017). Hybrid models use explicit symbolic representations in conjunction with black-box techniques (Choi, Wang, and Darwiche 2019). XAI was also expanded beyond the classical domains.…”

Section: State-of-the-artmentioning

confidence: 99%

AI for Explaining Decisions in Multi-Agent Environments

Kraus

Azaria

Fiošina

et al. 2020

AAAI

View full text Add to dashboard Cite

Explanation is necessary for humans to understand and accept decisions made by an AI system when the system's goal is known. It is even more important when the AI system makes decisions in multi-agent environments where the human does not know the systems' goals since they may depend on other agents' preferences. In such situations, explanations should aim to increase user satisfaction, taking into account the system's decision, the user's and the other agents' preferences, the environment settings and properties such as fairness, envy and privacy. Generating explanations that will increase user satisfaction is very challenging; to this end, we propose a new research direction: Explainable decisions in Multi-Agent Environments (xMASE). We then review the state of the art and discuss research directions towards efficient methodologies and algorithms for generating explanations that will increase users' satisfaction from AI systems' decisions in multi-agent environments.

show abstract

“…In contrast, (deep) density estimators can model uncertainty, but (marginal) inference is in general intractable. Indeed, probabilistic circuits such as sum-product networks (SPNs) [26] provide tractable inference, but unfortunately, they are generally not universal function approximators [4]. Therefore, we recently proposed conditional sum-product networks (CSPNs) [33] that can harness the expressive power of universal function approximators such as neural networks, while still maintaining a wide range of probabilistic inference routines.…”

Section: Introductionmentioning

confidence: 99%

Towards Understanding and Arguing with Classifiers: Recent Progress

et al. 2020

View full text Add to dashboard Cite

Machine learning and argumentation can potentially greatly benefit from each other. Combining deep classifiers with knowledge expressed in the form of rules and constraints allows one to leverage different forms of abstractions within argumentation mining. Argumentation for machine learning can yield argumentation-based learning methods where the machine and the user argue about the learned model with the common goal of providing results of maximum utility to the user. Unfortunately, both directions are currently rather challenging. For instance, combining deep neural models with logic typically only yields deterministic results, while combining probabilistic models with logic often results in intractable inference. Therefore, we review a novel deep but tractable model for conditional probability distributions that can harness the expressive power of universal function approximators such as neural networks while still maintaining a wide range of tractable inference routines. While this new model has shown appealing performance in classification tasks, humans cannot easily understand the reasons for its decision. Therefore, we also review our recent efforts on how to "argue" with deep models. On synthetic and real data we illustrate how "arguing" with a deep model about its explanations can actually help to revise the model, if it is right for the wrong reasons.

show abstract

On the relative expressiveness of Bayesian and neural networks

Cited by 18 publications

References 23 publications

Exact Learning Augmented Naive Bayes Classifier

Exact Learning Augmented Naive Bayes Classifier

AI for Explaining Decisions in Multi-Agent Environments

Towards Understanding and Arguing with Classifiers: Recent Progress

Contact Info

Product

Resources

About