A test for independence via Bayesian nonparametric estimation of mutual information

Al-Labadi, Luai; Asl, Forough Fazeli; Saberi, Zahra

doi:10.1002/cjs.11645

Cited by 5 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, it is recommended to choose a < n/2 based on the definition of H * in F pos (Al-Labadi and Zarepour, 2017). The idea behind using such a value of a is to avoid the excessive effect of the prior H on the test results by considering the chance of sampling from the observed data at least twice the chance of generating samples from H. However, some computational methods in the literature are proposed to elicit a that one may be interested in using (Al-Labadi et al, 2022b;Al-Labadi, 2021). Both expectations of M M D 2 BN P (F pos 1 , F 2 ) and M M D 2 BN P (F pri 1 , F 2 ) tend to 0 as a → ∞, N → ∞, and m → ∞, respectively, according to Corollaries 2 and 5.…”

Section: Constructing a Semi-bnp Two-sample Kernel-based Test Using R...mentioning

confidence: 99%

A Semi-Bayesian Nonparametric Hypothesis Test Using Maximum Mean Discrepancy with Applications in Generative Adversarial Networks

Fazeli-Asl¹,

Zhang²,

Lin³

2023

Preprint

View full text Add to dashboard Cite

A classic inferential problem in statistics is the two-sample hypothesis test, where we test whether two samples of observations are either drawn from the same distribution or two distinct distributions. However, standard methods for performing this test require strong distributional assumptions on the two samples of data. We propose a semi-Bayesian nonparametric (semi-BNP) procedure for the two-sample hypothesis testing problem. First, we will derive a novel BNP maximum mean discrepancy (MMD) measure-based hypothesis test. Next, we will show that our proposed test will outperform frequentist MMD-based methods by yielding a smaller false rejection and acceptance rate of the null. Finally, we will show that we can embed our proposed hypothesis testing procedure within a generative adversarial network (GAN) framework as an application of our method. Using our novel BNP hypothesis test, this new GAN approach can help to mitigate the lack of diversity in the generated samples and produce a more accurate inferential algorithm compared to traditional techniques.

show abstract

Section: Constructing a Semi-bnp Two-sample Kernel-based Test Using R...mentioning

confidence: 99%

A Semi-Bayesian Nonparametric Hypothesis Test Using Maximum Mean Discrepancy with Applications in Generative Adversarial Networks

Fazeli-Asl¹,

Zhang²,

Lin³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Apart from the G-test, there are other methods for testing independence that rely on mutual information. Of particular interest is the work of Al-Labadi et al [21], who propose a test of independence for continuous features that relies on computing mutual information using Bayesian nonparametric estimators for entropy. In theory, this would allow Markov boundary discovery algorithms to operate on data sets with continuous variables.…”

Section: Related Workmentioning

confidence: 99%

Accelerating Causal Inference and Feature Selection Methods through G-Test Computation Reuse

Băncioiu

Brad

2021

Entropy

View full text Add to dashboard Cite

This article presents a novel and remarkably efficient method of computing the statistical G-test made possible by exploiting a connection with the fundamental elements of information theory: by writing the G statistic as a sum of joint entropy terms, its computation is decomposed into easily reusable partial results with no change in the resulting value. This method greatly improves the efficiency of applications that perform a series of G-tests on permutations of the same features, such as feature selection and causal inference applications because this decomposition allows for an intensive reuse of these partial results. The efficiency of this method is demonstrated by implementing it as part of an experiment involving IPC–MB, an efficient Markov blanket discovery algorithm, applicable both as a feature selection algorithm and as a causal inference method. The results show outstanding efficiency gains for IPC–MB when the G-test is computed with the proposed method, compared to the unoptimized G-test, but also when compared to IPC–MB++, a variant of IPC–MB which is enhanced with an AD–tree, both static and dynamic. Even if this proposed method of computing the G-test is presented here in the context of IPC–MB, it is in fact bound neither to IPC–MB in particular, nor to feature selection or causal inference applications in general, because this method targets the information-theoretic concept that underlies the G-test, namely conditional mutual information. This aspect grants it wide applicability in data sciences.

show abstract

“…It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines, pattern recognition, data mining and tests of independence. The main goal of this paper is to provide an efficient estimator of the mutual information based on the approach of Al Labadi et. al.…”

mentioning

confidence: 99%

A Bayesian Nonparametric Estimation of Mutual Information

Al-Labadi¹,

Fazeli-Asl²,

Saberi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines, pattern recognition, data mining and tests of independence. The main goal of this paper is to provide an efficient estimator of the mutual information based on the approach of Al Labadi et. al. (2021). The estimator is explored through various examples and is compared to its frequentist counterpart due to Berrett et al. (2019). The results show the good performance of the procedure by having a smaller mean squared error.

show abstract

A test for independence via Bayesian nonparametric estimation of mutual information

Cited by 5 publications

References 32 publications

A Semi-Bayesian Nonparametric Hypothesis Test Using Maximum Mean Discrepancy with Applications in Generative Adversarial Networks

A Semi-Bayesian Nonparametric Hypothesis Test Using Maximum Mean Discrepancy with Applications in Generative Adversarial Networks

Accelerating Causal Inference and Feature Selection Methods through G-Test Computation Reuse

A Bayesian Nonparametric Estimation of Mutual Information

Contact Info

Product

Resources

About