Testing goodness-of-fit via rate distortion

Harremoës, Peter

doi:10.1109/itwnit.2009.5158533

Cited by 6 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these refinements are often difficult to compute. For instance, it can be shown using the results of [15] that p FA ≈ cn One approach to addressing the implementation issues of the universal test is through clustering (or partitioning) the alphabet as in [16], or smoothing in the space of probability measures as in [17], [18] to extend the Hoeffding test to the case of continuous alphabets. The mismatched test proposed here is a generalization of a partition in the following sense.…”

Section: ) Hypothesis Testingmentioning

confidence: 99%

Universal and Composite Hypothesis Testing via Mismatched Divergence

Unnikrishnan

Huang

Meyn

et al. 2011

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

For the universal hypothesis testing problem, where the goal is to decide between the known null hypothesis distribution and some other unknown distribution, Hoeffding proposed a universal test in the nineteen sixties. Hoeffding's universal test statistic can be written in terms of Kullback-Leibler (K-L) divergence between the empirical distribution of the observations and the null hypothesis distribution. In this paper a modification of Hoeffding's test is considered based on a relaxation of the K-L divergence test statistic, referred to as the mismatched divergence. The resulting mismatched test is shown to be a generalized likelihood-ratio test (GLRT) for the case where the alternate distribution lies in a parametric family of the distributions characterized by a finite dimensional parameter, i.e., it is a solution to the corresponding composite hypothesis testing problem. For certain choices of the alternate distribution, it is shown that both the Hoeffding test and the mismatched test have the same asymptotic performance in terms of error exponents. A consequence of this result is that the GLRT is optimal in differentiating a particular distribution from others in an exponential family. It is also shown that the mismatched test has a significant advantage over the Hoeffding test in terms of finite sample size performance. This advantage is due to the difference in the asymptotic variances of the two test statistics under the null hypothesis. In particular, the variance of the K-L divergence grows linearly with the alphabet size, making the test impractical for applications involving large alphabet distributions. The variance of the mismatched divergence on the other hand grows linearly with the dimension of the parameter space, and can hence be controlled through a prudent choice of the function class defining the mismatched divergence.Comment: Accepted to IEEE Transactions on Information Theory, July 201

show abstract

Section: ) Hypothesis Testingmentioning

confidence: 99%

Universal and Composite Hypothesis Testing via Mismatched Divergence

Unnikrishnan

Huang

Meyn

et al. 2011

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…The tendency is that mutual information is better for large deviations and χ 2 is better if the sample size is large compared with the degrees of freedom. The last problem can be corrected by 'smoothing' by a rate distortion test as suggested in [15], but both continuity corrections and rate distortion tests just tell that there are good alternative to χ 2 and mutual information. Looking at the behavior of such alternatives is not relevant if the question how to choose between χ 2 and mutual information.…”

Section: Qq-plots For Other Contingency Tablesmentioning

confidence: 99%

Mutual information of contingency tables and related inequalities

Harremoës¹

2014

2014 IEEE International Symposium on Information Theory

Self Cite

View full text Add to dashboard Cite

Abstract-For testing independence it is very popular to use either the χ 2 -statistic or G 2 -statistics (mutual information). Asymptotically both are χ 2 -distributed so an obvious question is which of the two statistics that has a distribution that is closest to the χ 2 -distribution. Surprisingly the distribution of mutual information is much better approximated by a χ 2 -distribution than the χ 2 -statistic. For technical reasons we shall focus on the simplest case with one degree of freedom. We introduce the signed log-likelihood and demonstrate that its distribution function can be related to the distribution function of a standard Gaussian by inequalities. For the hypergeometric distribution we formulate a general conjecture about how close the signed loglikelihood is to a standard Gaussian, and this conjecture gives much more accurate estimates of the tail probabilities of this type of distribution than previously published results. The conjecture has been proved numerically in all cases relevant for testing independence and further evidence of its validity is given. I. CHOICE OF STATISTICWe consider the problem of testing independence in a discrete setting. Here we shall follow the classic approach to this problem as developed by Pearson, Neuman and Fisher. The question is whether a sample with observation counts (X ij ) has been generated by the distribution Q = (q ij ) where q ij = s i ·t j for some probability vectors (s i ) and (t j ) reflecting independence. We introduce the marginal counts R i = j X ij and S j = i X ij and the sample size N = ij X ij . The maximum likelihood estimates of probability vectors (s i ) and (t j ) are given by s

show abstract

“…Applications of ideas from rate distortion theory for testing Goodness-of-Fit have been studied in [ 33 , 34 ]. For testing Goodness-of-Fit, we face the problem that data are discrete while the null-hypothesis may claim that the distribution is continuous.…”

Section: Introductionmentioning

confidence: 99%

Rate Distortion Theory for Descriptive Statistics

Harremoës

2023

Entropy

View full text Add to dashboard Cite

Rate distortion theory was developed for optimizing lossy compression of data, but it also has applications in statistics. In this paper, we illustrate how rate distortion theory can be used to analyze various datasets. The analysis involves testing, identification of outliers, choice of compression rate, calculation of optimal reconstruction points, and assigning “descriptive confidence regions” to the reconstruction points. We study four models or datasets of increasing complexity: clustering, Gaussian models, linear regression, and a dataset describing orientations of early Islamic mosques. These examples illustrate how rate distortion analysis may serve as a common framework for handling different statistical problems.

show abstract

Testing goodness-of-fit via rate distortion

Cited by 6 publications

References 11 publications

Universal and Composite Hypothesis Testing via Mismatched Divergence

Universal and Composite Hypothesis Testing via Mismatched Divergence

Mutual information of contingency tables and related inequalities

Rate Distortion Theory for Descriptive Statistics

Contact Info

Product

Resources

About