On the Rényi Cross-Entropy

Thierrin, Ferenc Cole; Alajaji, Fady; Linder, Tamás

doi:10.1109/cwit55308.2022.9817662

Cited by 2 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [ 8 ], the cross-entropy between pdfs

and

of the same type from the exponential family was proven to be

where

…”

Section: Rényi and Natural Rényi Differential Cross-entropies For Dis...mentioning

confidence: 99%

“…In [ 8 ], the Rényi differential cross-entropy rate between two stationary zero-mean Gaussian processes is derived. We present its formula alongside the expressions of the Shannon and Natural Rényi differential cross-entropy rates in Table 3 .…”

Section: Rényi and Natural Rényi Differential Cross-entropy Rates For...mentioning

confidence: 99%

“…In [ 8 ], the Rényi cross-entropy rate between finite-state time-invariant Markov sources was established, using, as in [ 16 ], tools from the theory of non-negative matrices and Perron–Frobenius theory (e.g., cf. [ 17 , 18 ]).…”

Section: Rényi and Natural Rényi Cross-entropy Rates For Markov Sourcesmentioning

confidence: 99%

“…In a recent conference paper [ 8 ], we showed how to calculate the Rényi differential cross-entropy

between distributions of the same type from the exponential family. Building upon the results shown there, the purpose of this paper is to derive in closed form the expression of

for thirteen commonly used univariate distributions from the exponential family, as well as for multivariate Gaussians, and tabulate the results for ease of reference.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Thierrin

Alajaji

Linder

2022

Entropy

View full text Add to dashboard Cite

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.

show abstract

“…In [ 8 ], the cross-entropy between pdfs

and

of the same type from the exponential family was proven to be

where

…”

Section: Rényi and Natural Rényi Differential Cross-entropies For Dis...mentioning

confidence: 99%

Section: Rényi and Natural Rényi Differential Cross-entropy Rates For...mentioning

confidence: 99%

Section: Rényi and Natural Rényi Cross-entropy Rates For Markov Sourcesmentioning

confidence: 99%

“…In a recent conference paper [ 8 ], we showed how to calculate the Rényi differential cross-entropy

between distributions of the same type from the exponential family. Building upon the results shown there, the purpose of this paper is to derive in closed form the expression of

for thirteen commonly used univariate distributions from the exponential family, as well as for multivariate Gaussians, and tabulate the results for ease of reference.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations