On the maximum Shannon entropy of the full state space of a T-code decoder

Abstract

The state space of a decoder for a variable length code is the code's decoding tree. This paper looks at the probabilities with which such a decoder is found in any one particular node of the tree when decoding an infinite random string. The paper then looks at the special case of T-codes, and shows that these probabilities may be derived through a recurrence relation based on the codes' recursive construction scheme. The resulting Shannon entropy of the decoder is shown to be a sum of individual contributions from each construction step. These contributions only depend on the properties of the step itself, but not on the previous construction history.