An information-theoretic central limit theorem for finitely susceptible FKG systems

Johnson, Oliver

doi:10.4213/tvp110

Cited by 3 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The convergence of the entropy of I s (x, y) to that of a Gaussian distribution has been established by [28]. But the monotone convergence has only been established for the iid case by [4].…”

Section: Transformation H(p(ĵ))−h(p(î)) = H(p(j))−h(p(i))mentioning

confidence: 99%

From information scaling of natural images to regimes of statistical models

Guo

Zhu

2007

Quart. Appl. Math.

View full text Add to dashboard Cite

Abstract. Vision can be considered a highly specialized data collection and analysis problem. We need to understand the special properties of natural image data in order to construct statistical models and develop statistical methods for representing and recognizing the wide variety of natural image patterns. One fundamental property of natural image data that distinguishes vision from other sensory tasks such as speech recognition is that scale plays a profound role in image formation and interpretation. Specifically, visual objects can appear at a wide range of scales in the images due to the change of viewing distance as well as camera resolution. The same objects appearing at different scales produce different image data with different statistical properties. In particular, we show that the entropy rate of the image data changes over scale. Moreover, the inferential uncertainty changes over scale too. We call these changes information scaling. We then examine both empirically and theoretically two prominent and yet largely isolated classes of image models, namely, wavelet sparse coding models and Markov random field models. Our results indicate that the two classes of models are appropriate for two different entropy regimes: sparse coding targets low entropy regimes, whereas Markov random fields are appropriate for high entropy regimes. Because information scaling connects different entropy regimes, both sparse coding and Markov random fields are necessary for representing natural image data, and information scaling triggers transitions between these two regimes. This motivates us to propose a modeling scheme that embraces both regimes of models in a common framework. The contribution of our work is two-fold. First, the study of information scaling provides a unifying perspective for the rich variety of natural image patterns. Second, the modeling scheme that we develop provides a natural integration of different regimes of image models.

show abstract

Section: Transformation H(p(ĵ))−h(p(î)) = H(p(j))−h(p(i))mentioning

confidence: 99%

From information scaling of natural images to regimes of statistical models

Guo

Zhu

2007

Quart. Appl. Math.

View full text Add to dashboard Cite

show abstract

Forward and Reverse Entropy Power Inequalities in Convex Geometry

Madiman

Melbourne

2017

The IMA Volumes in Mathematics and Its Applications

View full text Add to dashboard Cite

The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent developments on forward and reverse entropy power inequalities not just for the Shannon-Boltzmann entropy but also more generally for Rényi entropy. In the process, we discuss connections between the so-called functional (or integral) and probabilistic (or entropic) analogues of some classical inequalities in geometric functional analysis. * All the authors are with the

show abstract