Strong evidence of an information-theoretical conservation principle linking all discrete systems

Abstract

Diverse discrete systems share common global properties that lack a unifying theoretical explanation. However, constraining the simplest measure of total information (Hartley–Shannon) in a statistical mechanics framework reveals a principle, the conservation of Hartley–Shannon information (CoHSI) that directly predicts both known and unsuspected common properties of discrete systems, as borne out in the diverse systems of computer software, proteins and music. Discrete systems fall into two categories distinguished by their structure: heterogeneous systems in which there is a distinguishable order of assembly of the system’s components from an alphabet of unique tokens (e.g. proteins assembled from an alphabet of amino acids), and homogeneous systems in which unique tokens are simply binned, counted and rank ordered. Heterogeneous systems are characterized by an implicit distribution of component lengths, with sharp unimodal peak (containing the majority of components) and a power-law tail, whereas homogeneous systems reduce naturally to Zipf’s Law but with a drooping tail in the distribution. We also confirm predictions that very long components are inevitable for heterogeneous systems; that discrete systems can exhibit simultaneously both heterogeneous and homogeneous behaviour; and that in systems with more than one consistent token alphabet (e.g. digital music), the alphabets themselves show a power-law relationship.

A new diversity index

“…We will also seek to extend the work to explore the behaviour of indices where the linear term in equation ( 3) is replaced by a higher order polynomial, as well as to extend the application to include measures of phylogenetic distance. A further aim of this work is to explore the behaviour of indices of this kind when sampling distributions whose frequencies appear to be predicted on a combinational basis, and which appear to follow a universal distribution [21].…”

Parallelising semantic checking in an IDE: A way toward improving profits and sustainability, while maintaining high-quality software development

“…Hatton proposed [15] that the Conservation of Hartley-Shannon Information might play the same role in discrete systems as the Conservation of Energy does in physical systems, proving [16] Zipf's Law in the case of homogeneous systems and showing strong evidence for unusually long components being an inevitable by-product of the total size of the system. He validated the claims on 100 million lines of code in 7 programming languages and 24 Fortran 90 packages.…”

“…To process these test suites, we created a new "TITAN Project (Java)" with the name of the TTCN-3 project, copied the source code from the downloaded compressed files into its "src" folder, and converted all XSD files 15 . We manually reviewed all problems detected by Titan on these projects and reported the incorrect ones to the development team 16 .…”