A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.)

Ahmad, Ibrahim A.; Lin, Pi-Erh

doi:10.1109/tit.1976.1055550

Cited by 163 publications

(96 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Non-parametric estimation of Shannon entropy has been of interest to many in non-parametric statistics, pattern recognition, model identification, image registration and other areas [11], [12], [13], [14], [15], [1], [16]. Estimation of «-entropy arises as a step towards Shannon entropy estimation, e.g., Mokkadem [17] constructed a nonparametric estimate of the Shannon entropy from a convergent sequence of «-entropy estimates.…”

Section: Introductionmentioning

confidence: 99%

Applications of entropic spanning graphs

Hero¹,

Ma²,

Michel³

et al. 2002

IEEE Signal Process. Mag.

199

192

View full text Add to dashboard Cite

Abstract-This paper presents applications of entropic spanning graphs to imaging and feature clustering applications. Entropic spanning graphs span a set of feature vectors in such a way that the normalized spanning length of the graph converges to the entropy of the feature distribution as the number of random feature vectors increases. This property makes these graphs naturally suited to applications where entropy and information divergence are used as discriminants including: texture classification; feature clustering; image indexing; and image registration. Among other areas, these problems arise in geographical information systems, digital libraries, medical information processing, video indexing, multi-sensor fusion, and content-based retrieval.

show abstract

Section: Introductionmentioning

confidence: 99%

Applications of entropic spanning graphs

Hero¹,

Ma²,

Michel³

et al. 2002

IEEE Signal Process. Mag.

199

192

View full text Add to dashboard Cite

show abstract

“…Certainly, mutual information-based methods similar to what we have outlined in this paper could be developed to test locality assumptions in spatially distributed data assimilation applications. There are also methods for calculating joint entropies using kernel approximations of underlying probability mass function which do not require reprojections [e.g., Ahmad and Lin, 1976]; however, these methods are ultimately subject to the same curse of dimensionality which limits all nonparametric density estimation, and are only practical up to a few dimensions. In both the Theory and Demonstration sections (sections 2 and 3), we estimated the entropy of one-and two-dimensional probability density functions and considered only one observation at a time.…”

Section: Summary and Discussionmentioning

confidence: 99%

An approach to quantifying the efficiency of a Bayesian filter

Nearing

Gupta

Crow

et al. 2013

Water Resources Research

View full text Add to dashboard Cite

[1] Data assimilation is the Bayesian conditioning of uncertain model simulations on observations to reduce uncertainty about model states. In practice, it is common to make simplifying assumptions about the prior and posterior state distributions, and to employ approximations of the likelihood function, which can reduce the efficiency of the filter. We propose metrics that quantify how much of the uncertainty in a Bayesian posterior state distribution is due to (i) the observation operator, (ii) observation error, and (iii) approximations of Bayes' Law. Our approach uses discrete Shannon entropy to quantify uncertainty, and we define the utility of an observation (for reducing uncertainty about a model state) as the ratio of the mutual information between the state and observation to the entropy of the state prior. These metrics make it possible to analyze the efficiency of a proposed observation system and data assimilation strategy, and provide a way to examine the propagation of information through the dynamic system model. We demonstrate the procedure on the problem of estimating profile soil moisture from observations at the surface (top 5 cm). The results show that when synthetic observations of 5 cm soil moisture are assimilated into a three-layer model of soil hydrology, the ensemble Kalman filter does not use all of the information available in observations.

show abstract

“…Assuming we have estimates p 1 , p 2 of the pdfs p 1 , p 2 , we use the Ahmad-Lin entropy estimators [28]:…”

Section: Non-parametric Estimation Of the Similarity Measurementioning

confidence: 99%

Sparse Multiscale Patches for Image Processing

Piro

Anthoine

Debreuve

et al. 2009

Emerging Trends in Visual Computing

View full text Add to dashboard Cite

Abstract. This paper presents a framework to define an objective measure of the similarity (or dissimilarity) between two images for image processing. The problem is twofold: 1) define a set of features that capture the information contained in the image relevant for the given task and 2) define a similarity measure in this feature space. In this paper, we propose a feature space as well as a statistical measure on this space. Our feature space is based on a global descriptor of the image in a multiscale transformed domain. After decomposition into a Laplacian pyramid, the coefficients are arranged in intrascale/interscale/interchannel patches which reflect the dependencies of neighboring coefficients in presence of specific structures or textures. At each scale, the probability density function (pdf) of these patches is used as a descriptor of the relevant information. Because of the sparsity of the multiscale transform, the most significant patches, called Sparse Multiscale Patches (SMP), describe efficiently these pdfs. We propose a statistical measure (the Kullback-Leibler divergence) based on the comparison of these probability density functions. Interestingly, this measure is estimated via the nonparametric, k-th nearest neighbor framework without explicitly building the pdfs. This framework is applied to a query-by-example image retrieval method. Experiments on two publicly available databases showed the potential of our SMP approach for this task. In particular, it performed comparably to a SIFT -based retrieval method and two versions of a fuzzy segmentation-based method (the UFM and CLUE methods), and it exhibited some robustness to different geometric and radiometric deformations of the images.

show abstract

A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.)

Cited by 163 publications

References 4 publications

Applications of entropic spanning graphs

Applications of entropic spanning graphs

An approach to quantifying the efficiency of a Bayesian filter

Sparse Multiscale Patches for Image Processing

Contact Info

Product

Resources

About