Information entropy and thermal entropy: apples and oranges

Abstract

The frequent misunderstanding of information entropy is pointed out. It is shown that, contrary to fortuitous situations and common beliefs, there is no general interrelation between the information entropy and the thermodynamical entropy. We point out that the change of information entropy during measurement is determined by the resolution of the measurement instrument and these changes do not have a general and clear-cut interrelation with the change of thermal entropy. Moreover, these changes can be separated in space and time. We show classical physical considerations and examples to support this conclusion. Finally, we show that information entropy can violate the Third Law of Thermodynamics which is another indication of major differences from thermal entropy.

The Auto-Regressive Model and Spectrum Information Entropy Judgment Method for High Intensity Focused Ultrasound Echo Signal

“…The more orderly a system, the lower the information entropy; conversely, the more chaotic a system, the higher the information entropy. Therefore, information entropy is an important indicator to measure the order of the system [20][21][22].…”

Caracterización de la movilidad compartida de las empresas de redes de transporte en México: entropía, homeostasis, negentropía

“…De esta manera, sí la información o entropía negativa está asociada con una medida del orden [7] definida "por una expresión isomorfa con la entropia negativa de la termodinámica" [8]; entonces, "(…) la falta de información corresponde a un desorden real en los grados de libertad buscados" [9]. No obstante, para Kish y Ferry "(…) there is no general interrelation between the information entropy and the thermodynamical entropy (…) the information entropy is a subjective, measurement-system-dependent feature while the thermodynamical entropy is an objective property of the physical system" [10].…”

Information Content of the Model for Calculating the Finite Precision of Measurements

“…Not unimportant is the fact that the choice of any of the various existing systems of units, in principle, does not affect the stated features of Equation (12). This can be shown using Equation (8). Imagine that the number of dimensionless criteria and numbers in the extended system of units (numbered "2") is equal to μ2 and 2•μSI = μ2.…”

“…For several decades, publications on the theory of information and its application in physics indicate two mutually exclusive approaches to assessing the nature of information entropy. According to the first, information entropy is only an abstract, mathematically well-formalized concept and a subjective, measurement-system-dependent feature [6][7][8]. As is usually the case in science, the opposite opinion exists.…”