Calculation of Precise Constants in a Probability Model of Zipf’s Law Generation and Asymptotics of Sums of Multinomial Coefficients

Abstract

Let 0 , 1 , . . . , be a full set of outcomes (symbols) and let positive , = 0, . . . , , be their probabilities (∑ =0 = 1). Let us treat 0 as a stop symbol; it can occur in sequences of symbols (we call them words) only once, at the very end. The probability of a word is defined as the product of probabilities of its symbols. We consider the list of all possible words sorted in the nonincreasing order of their probabilities. Let ( ) be the probability of the th word in this list. We prove that if at least one of the ratios log / log , , ∈ {1, . . . , }, is irrational, then the limit lim →∞ ( )/ −1/ exists and differs from zero; here is the root of the equation ∑ =1 = 1. The limit constant can be expressed (rather easily) in terms of the entropy of the distribution ( 1 , . . . , ).