Analysis of an information-theoretic model for communication

Abstract

We study the cost-minimization problem posed by Ferrer i Cancho and Solé in their model of communication that aimed at explaining the origin of Zipf's law [PNAS 100, 788 (2003)].Direct analysis shows that the minimum cost is min{λ, 1 − λ}, where λ determines the relative weights of speaker's and hearer's costs in the total, as shown in several previous works using different approaches. The nature and multiplicity of the minimizing solution changes discontinuously at λ = 1/2, being qualitatively different for λ < 1/2, λ > 1/2, and λ = 1/2. Zipf's law is found only in a vanishing fraction of the minimum-cost solutions at λ = 1/2 and therefore is not explained by this model. Imposing the further condition of equal costs yields distributions substantially closer to Zipf's law, but significant differences persist. We also investigate the solutions reached by the previously used minimization algorithm and find that they correctly recover global minimum states at the transition.

Zipf's law emerges asymptotically during phase transitions in communicative systems

“…or, in terms of the Zipfian exponent, 1 ≤ α ≤ 2, which is typically found to be the case [5,18]. Since there are infinitely many points within the boundaries of this α domain, this supports the notion [19] that Zipf's law (i.e., α ≈ 1 cases) is found only in a vanishing fraction of the total minimum-cost solutions at λ = 1/2, i.e., any reasonably sized Zipf distribution has essentially zero probability of appearing in the set of minimum-cost matrices at λ = 1/2. Nevertheless, assuming that a language is defined by a lexicon composed of at least two unique words, Zipf's law is rapidly approached in an asymptotic fashion for lexicons composed of five or more words (Fig.…”

Zipf’s Law: Balancing Signal Usage Cost and Communication Efficiency

“…For instance, for square matrices, Zipf’s law results from the optimal languages which satisfy equal efforts, i.e., λ = 0.5. The importance of equal cost was emphasized in earlier works [ 4 , 26 ]. The exponent defined by Eq (21) changes with the system size ( n or m ), and so the resulting power law “adapts” to linguistic dynamics and language evolution in general.…”

Large-Scale Analysis of Zipf’s Law in English Texts

“…More fascinatingly, the same law has been claimed in other codes of communication, as in music [7] or for the timbres of sounds [8], and also in disparate discrete systems where individual units or agents gather into different classes [9], for example, employees into firms [10], believers into religions [11], insects into plants [12], units of mass into animals present in ecosystems [13], visitors or links into web pages [14], telephone calls to users [15], or abundance of proteins (in a single cell) [16]. The attempts to find an explanation have been diverse [3,15,[17][18][19][20][21][22][23][24], but no solution has raised consensus [20,25,26]. Despite its quantitative character, Zipf's law has been usually checked for in a qualitative way, plotting the logarithm of the frequency n versus the logarithm of the rank r and looking for some domain with a roughly linear behavior, with slope more or less close to −1.…”