Counting distinctions: on the conceptual foundations of Shannon’s information theory

Abstract

The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the partition. The logical concept of entropy based on partition logic is the normalized counting measure of the set of distinctions of a partition on a finite set-just as the usual logical notion of probability based on the Boolean logic of subsets is the normalized counting measure of the subsets (events). Thus logical entropy is a measure on the set of ordered pairs, and all the compound notions of entropy (join entropy, conditional entropy, and mutual information) arise in the usual way from the measure (e.g., the inclusion-exclusion principle)-just like the corresponding notions of probability. The usual Shannon entropy of a partition is developed by replacing the normalized count of distinctions (dits) by the average number of binary partitions (bits) necessary to make all the distinctions of the partition.

The Objective Indefiniteness Interpretation of Quantum Mechanics

“…We can now describe how the dual logic of partitions captures at the logical level a vision of reality with objectively inde…nite (or indistinct) 8 entities. The key step is to interpret a subset such as a block B in a partition, not as a subset of the distinct elements u 2 B, but as a single objectively indistinct element that, with further distinctions, could become any of the fully distinct elements u 2 B.…”

“…The …rst two are the "Bell states" which are the two graphs of bijections U ! U and have the maximum entanglement if entanglement is measured by the logical divergence d (Pr(x; y)jj Pr (x) Pr (y)) [8]. All the 9 "separated" states have zero "entanglement" by the same measure.…”

“…Moreover, Boole developed a logical …nite probability theory out of his logic of subsets [1], and the analogous theory developed out of the logic of partitions is a logical version of information theory. [8] The concepts and operations of partition logic and logical information theory are developed in the rather austere set-theoretic context; they needed to be "lifted" to the richer environment of vector spaces. This lifting program from sets to vector spaces is part of the mathematical folklore (e.g., used intuitively by von Neumann).…”

An introduction to partition logic

“…Partition logic is at the same mathematical level as subset logic since the semantic models for both are constructed from (partitions on or subsets of) arbitrary unstructured sets with no topologies, no ordering relations and no compatibility or accessibility relations on the sets. 2 Just as Boole developed logical finite probability (the normalized counting measure on subsets of a finite universe) as a quantitative treatment of subset logic, applying the analogous mathematical steps to partition logic yields a logical notion of entropy (the normalized counting measure on the partition relation complementary to the equivalence relation as a binary relation on a finite universe) so that information theory can be refounded on a logical basis [8] in partition logic.…”

“…These results and some other easy reductions are given in the following table where the interior of the subset of U ×U in the first column yields the dit set of the binary operation given in the second column. 8 …”