The Statistical Foundations of Entropy

Abstract

During the last few decades, the notion of entropy has become omnipresent in many scientific disciplines, ranging from traditional applications in statistical physics and chemistry, information theory, and statistical estimation to more recent applications in biology, astrophysics, geology, financial markets, or social networks[...]

The Separability Problem in Molecular Quantum Systems: Information‐Theoretic Framework for Atoms in Molecules

“…It is interesting to mention that the two-parameter Sharma-Mittal (S-M) uncertainty measure represents the most general approach to attack the Stockholder partitioning scheme, since it represents a generalization of the Gibbs-Shannon, Renyi, and Tsallis information measures. [77,78] Applications of the Sharma-Mittal entropy in chemistry are still relatively limited [75] compared to other areas, [78,79] its ability to capture nonextensive behavior and long-range interactions makes it a valuable tool for studying complex chemical systems and processes where traditional entropy measures may not be sufficient. Although it is not the goal of this study to perform any numerical calculations to test the S-M uncertainty measure, other than demonstrating its utility to provide a theoretical justification for employing the stockholder partitioning scheme without resource of any arbitrary reference atoms.…”

Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data

“…However, this change in averaging constraints neither prevents the Rényi entropy from violating these two important axioms. Finally, one might try to extend the scope of one of the axioms, for example, the probability independence axiom [41]. However, this does not suffice, since the Rényi entropy would still violate the subset independence axiom.…”