Entropy-Based Statistics and Their Applications

Zhang, Zhiyi

doi:10.3390/e25060936

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The said porosity permeates across the board, from basic axioms to a general definition of entropy, and from model interpretability to statistical inference, although some recent effort is observed to alleviate it. See, for example, [13] and [14], where a general definition of entropy and several fundamental results are given.…”

Section: Discussionmentioning

confidence: 99%

Khinchin’s Fourth Axiom of Entropy Revisited

Zhang,

Huang,

2023

Stats

Self Cite

View full text Add to dashboard Cite

The Boltzmann–Gibbs–Shannon (BGS) entropy is the only entropy form satisfying four conditions known as Khinchin’s axioms. The uniqueness theorem of the BGS entropy, plus the fact that Shannon’s mutual information completely characterizes independence between the two underlying random elements, puts the BGS entropy in a special place in many fields of study. In this article, the fourth axiom is replaced by a slightly weakened condition: an entropy whose associated mutual information is zero if and only if the two underlying random elements are independent. Under the weaker fourth axiom, other forms of entropy are sought by way of escort transformations. Two main results are reported in this article. First, there are many entropies other than the BGS entropy satisfying the weaker condition, yet retaining all the desirable utilities of the BGS entropy. Second, by way of escort transformations, the newly identified entropies are the only ones satisfying the weaker axioms.

show abstract

Section: Discussionmentioning

confidence: 99%

Khinchin’s Fourth Axiom of Entropy Revisited

Zhang,

Huang,

2023

Stats

Self Cite

View full text Add to dashboard Cite

show abstract

Levels of Confidence and Utility for Binary Classifiers

Zhang

2024

Stats

View full text Add to dashboard Cite

Two performance measures for binary tree classifiers are introduced: the level of confidence and the level of utility. Both measures are probabilities of desirable events in the construction process of a classifier and hence are easily and intuitively interpretable. The statistical estimation of these measures is discussed. The usual maximum likelihood estimators are shown to have upward biases, and an entropy-based bias-reducing methodology is proposed. Along the way, the basic question of appropriate sample sizes at tree nodes is considered.

show abstract

Entropy-Based Statistics and Their Applications

Abstract: During the last few decades, research activity in modeling the properties of random systems via entropies has grown noticeably across a wide spectrum of fields [...]

Cited by 2 publications

References 0 publications

Khinchin’s Fourth Axiom of Entropy Revisited

Khinchin’s Fourth Axiom of Entropy Revisited

Levels of Confidence and Utility for Binary Classifiers

Contact Info

Product

Resources

About