Zipf’s Law: Balancing Signal Usage Cost and Communication Efficiency

Abstract

We propose a model that explains the reliable emergence of power laws (e.g., Zipf’s law) during the development of different human languages. The model incorporates the principle of least effort in communications, minimizing a combination of the information-theoretic communication inefficiency and direct signal cost. We prove a general relationship, for all optimal languages, between the signal cost distribution and the resulting distribution of signals. Zipf’s law then emerges for logarithmic signal cost distributions, which is the cost distribution expected for words constructed from letters or phonemes.

Ethology as a physical science

“…This may reflect a trade-off between efficient stereotyped behaviour and relatively inefficient but necessary flexible behaviour. A similar efficiency argument has been advanced to underlie Zipf's law 62 , the power-law describing the rank-frequency distributions of word use 63 and heavy-tailed distributions of behaviours have also been observed in spontaneous worm behaviour 64 . However, in some circumstances at least, heavy-tailedness observed at a population level reflects heterogeneity among animals that are individually stereotyped 65,66 .…”

Zipf’s word frequency law in natural language: A critical review and future directions

“…Mandelbrot (1953) shows how the Zipfian distribution could arise from minimizing information-theoretic notions of cost (Mandelbrot (1962, 1966), ideas further developed by D. Manin (2009), Ferrer i Cancho and colleagues (Ferrer i Cancho, 2005a, 2005b; Ferrer i Cancho & Solé, 2003) and, more recently, Salge et al (2013).…”

“…To give a brief picture of the range of explanations that have been worked out, such distributions have been argued to arise from random concatenative processes (Conrad & Mitzenmacher, 2004; Li, 1992; Miller, 1957), mixtures of exponential distributions (Farmer & Geanakoplos, 2006), scale-invariance (Chater & Brown, 1999), (bounded) optimization of entropy (Mandelbrot, 1953) or Fisher information (Hernando, Puigdomènech, Villuendas, Vesperinas & Plastino, 2009), the invariance of such power laws under aggregation (see Farmer & Geanakoplos, 2006), multiplicative stochastic processes (see Mitzenmacher, 2004), preferential reuse (Simon, 1955; Yule, 1944), symbolic descriptions of complex stochastic systems (Corominas-Murtra & Solé, 2010), random walks on logarithmic scales (Kawamura & Hatano, 2002), semantic organization (Guiraud, 1968; D. Manin, 2008), communicative optimization (Ferrer i Cancho, 2005a, b; Ferrer i Cancho & Solé, 2003; Mandelbrot, 1962; Salge, Ay, Polani, & Prokopenko, 2013; Zipf, 1936, 1949), random division of elements into groups (Baek, Bernhardsson & Minnhagen 2011), first- and second-order approximation of most common (e.g., normal) distributions (Belevitch, 1959), and optimized memory search (Parker-Rhodes & Joyce, 1956), among many others.…”

Phase transitions in Pareto optimal complex networks

“…with λ ∈ [0, 1] a tunable parameter that weights the impact of each contribution. This is precisely the strategy in previous accounts of these problems [25][26][27][28][29][30][31]. If λ = 1 only efficiency constraints will be at work, whereas λ = 0 would ignore this component.…”

“…This problem has been addressed by explicitly introducing efficiency measures E (such as average path length) along with cost constraints C (such as number of connections of a given graph) [25][26][27]. A similar example in another field models languages as a network of associations between objects and words, and considers language evolution through a least effort process [28][29][30]. Here, the cost-efficiency conflict is mapped onto coding/decoding efforts for users of an economic (while ambiguous) language.…”