An information-theoretic approach to statistical dependence: Copula information

Calsaverini, Rafael; Vicente, Renato

doi:10.1209/0295-5075/88/68003

Cited by 58 publications

(58 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the MI ( , ) I X Y becomes the negative copula entropy [24,25] given by In PP12 [12], we have generalized this problem to a less constrained MinMI version by changing marginal RVs into ME prescribed ones-the ME-morphisms (e.g., standard Gaussians)-and imposing a finite set of marginal constraints instead of the full marginal PDFs. Under these conditions, the number of control Lagrange multipliers is finite, leaving the possibility of using nonlinear minimization algorithms for the MinMI estimation, as already tested in [8] We further provide asymptotic analytical N-scaled formulas for the variance and distribution of MinMI estimation errors as functions of statistics of the ME cross constraints estimation errors.…”

Section: The Rationale Of the Papermentioning

confidence: 99%

Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples

Pires

Perdigão

2013

Entropy

View full text Add to dashboard Cite

Abstract:The Minimum Mutual Information (MinMI) Principle provides the least committed, maximum-joint-entropy (ME) inferential law that is compatible with prescribed marginal distributions and empirical cross constraints. Here, we estimate MI bounds (the MinMI values) generated by constraining sets T cr comprehended by m cr linear and/or nonlinear joint expectations, computed from samples of N iid outcomes. Marginals (and their entropy) are imposed by single morphisms of the original random variables.N-asymptotic formulas are given both for the distribution of cross expectation's estimation errors, the MinMI estimation bias, its variance and distribution. A growing T cr leads to an increasing MinMI, converging eventually to the total MI. Under N-sized samples, the MinMI increment relative to two encapsulated sets T cr1  T cr2 (with numbers of constraints As an example, we have set marginals to being normally distributed (Gaussian) and have built a sequence of MI bounds, associated to successive non-linear correlations due to joint non-Gaussianity. Noting that in real-world situations available sample sizes can be rather low, the relationship between MinMI bias, probability density over-fitting and outliers is put in evidence for under-sampled data. OPEN ACCESSEntropy 2013, 15 722

show abstract

Section: The Rationale Of the Papermentioning

confidence: 99%

Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples

Pires

Perdigão

2013

Entropy

View full text Add to dashboard Cite

show abstract

“…, which is uniquely dependent on the cumulated marginal probabilities and equal to the density ratio, independently from the specific forms of marginal PDFs [24]. In particular, the Gaussian correlation is given by:…”

Section: T θmentioning

confidence: 99%

“…Then we get the MI ˆ( , ) ( , ) I X Y I X Y = , which is decomposed into two generic positive quantities, a Gaussian MI I g and a non-Gaussian MI I ng [21] [24], uniquely dependent on the cross dependency between variables. The non-Gaussian MI I ng holds some interesting characteristics.…”

Section: Introductionmentioning

confidence: 99%

Minimum Mutual Information and Non-Gaussianity Through the Maximum Entropy Method: Theory and Properties

Pires

Perdigão

2012

Entropy

View full text Add to dashboard Cite

Abstract:The application of the Maximum Entropy (ME) principle leads to a minimum of the Mutual Information (MI), I(X,Y), between random variables X,Y, which is compatible with prescribed joint expectations and given ME marginal distributions. A sequence of sets of joint constraints leads to a hierarchy of lower MI bounds increasingly approaching the true MI. In particular, using standard bivariate Gaussian marginal distributions, it allows for the MI decomposition into two positive terms: the Gaussian MI (I g ), depending upon the Gaussian correlation or the correlation between 'Gaussianized variables', and a non-Gaussian MI (I ng ), coinciding with joint negentropy and depending upon nonlinear correlations. Joint moments of a prescribed total order p are bounded within a compact set defined by Schwarz-like inequalities, where I ng grows from zero at the 'Gaussian manifold' where moments are those of Gaussian distributions, towards infinity at the set's boundary where a deterministic relationship holds. Sources of joint non-Gaussianity have been systematized by estimating I ng between the input and output from a nonlinear synthetic channel contaminated by multiplicative and non-Gaussian additive noises for a full range of signal-to-noise ratio (snr) variances. We have studied the effect of varying snr on I g and I ng under several signal/noise scenarios.

show abstract

“…The main idea behind copula theory is that the cumulative distribution function (CDF) of a random vector can be represented by uniform marginal cumulative distribution functions, and a copula that connects these marginal cumulative distribution functions [2].…”

Section: Copula Functions Overviewmentioning

confidence: 99%

“…In copula theory, the cumulative distribution function (CDF) of a random vector can be represented by uniform marginal cumulative distribution functions and a copula that connects these marginal cumulative distribution functions [2]. Although, copula functions are used in various applications such as economics and finance, climate research, oceanography, hydrology, geodesy, evolutionary computation, they are used in limited image processing applications such as image change detection, and image registration [3].…”

Section: Introductionmentioning

confidence: 99%

Image quality and forgery detection copula-based algorithms

Hammad¹

View full text Add to dashboard Cite

Copula functions are important tools to investigate dependence structure between random variables. There are many copulas such as: Gaussian, Marshall-Olkin, Clayton, and Frank copulas. Although, copulas have been used in finance, oceanography, and hydrology, they have been applied in limited applications in the image processing field. In this thesis, copulas are applied to calculate the mutual information of two images, which in tum is used to measure image quality of a targeted image and also used to detect copy-move forgery in images. The proposed algorithms introduce new alternatives for existing image quality assessment and forgery detection methods. These algorithms are easy to use and highly accurate. The results for our image quality assessment algorithm are comparable or better than those of established methods in the literature, while the results for our image forgery detection algorithm are accurate even after applying different manipulation and postprocessing techniques on the forged images.

show abstract

An information-theoretic approach to statistical dependence: Copula information

Cited by 58 publications

References 23 publications

Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples

Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples

Minimum Mutual Information and Non-Gaussianity Through the Maximum Entropy Method: Theory and Properties

Image quality and forgery detection copula-based algorithms

Contact Info

Product

Resources

About