Zipf’s law revisited: Spoken dialog, linguistic units, parameters, and the principle of least effort

Abstract

The ubiquitous inverse relationship between word frequency and word rank is commonly known as Zipf’s law. The theoretical underpinning of this law states that the inverse relationship yields decreased effort in both the speaker and hearer, the so-called principle of least effort. Most research has focused on showing an inverse relationship only for written monolog, only for frequencies and ranks of one linguistic unit, generally word unigrams, with strong correlations of the power law to the observed frequency distributions, with limited to no attention to psychological mechanisms such as the principle of least effort. The current paper extends the existing findings, by not focusing on written monolog but on a more fundamental form of communication, spoken dialog, by not only investigating word unigrams but also units quantified on syntactic, pragmatic, utterance, and nonverbal communicative levels by showing that the adequacy of Zipf’s formula seems ubiquitous, but the exponent of the power law curve is not, and by placing these findings in the context of Zipf’s principle of least effort through redefining effort in terms of cognitive resources available for communication. Our findings show that Zipf’s law also applies to a more natural form of communication—that of spoken dialog, that it applies to a range of linguistic units beyond word unigrams, that the general good fit of Zipf’s law needs to be revisited in light of the parameters of the formula, and that the principle of least effort is a useful theoretical framework for the findings of Zipf’s law.

Language-like efficiency and structure in house finch song

“…I will limit my interpretation of this result, given ongoing debates about the cause of the rank-frequency law, but it is notable that the goodness-of-fit of the Zipf-Mandelbrot distribution to house finch song (R 2 . 0:99) is within the range of human language [21,91].…”

Consciousness influences the enhancement of visual statistical learning in Zipfian distributions.

“…Zipf’s law has been applied to various aspects of our daily life. Although no comprehensive explanation for these phenomena has been proposed to date, some studies have demonstrated that they can be explained by minimization in communication effort (Ferrer i Cancho & Solé, 2003; Linders & Louwerse, 2023). The current findings that VSL enhancement in Zipfian distributions is mediated by human consciousness suggest that the mechanisms of human visual cognition may produce phenomena in everyday situations that follow Zipfian distributions, allowing us to recognize our environments efficiently as cognitive misers (Fiske & Taylor, 2020).…”

Commercialization of state-owned broadcast networks in China

“…We discern that the α for numbers rests at 0.75. By juxtaposing this finding with the spectrum of α values characterizing other members of Zipf's familyrange from 0.72 for the dialog [35], 1 for the word [15], and 1 for the gene expression [55] to 2 encapsulating the ecological size spectra [56]. Meanwhile, the firm size retains 1 [28], the internet's α gravitates to 0.81 [49], music embodies 0.85 [57], citations veer across a range spanning from 2.4 to 3.1 [58],…”

“…For several decades, plentiful domains have engaged in quantitative examinations of Zipf's Law [28][29][30][31][32][33][34][35][36]. They perceive themselves as complex systems comprised of numerous individual objects whose behaviors interdepend and evolve dynamically, thus giving rise to emergent collective behaviors [37].…”

“…Typically, when an R 2 exceeds 0.9, it signifies a strong fit between the regression model and the observed data, suggesting a high degree of conformity. An R 2 exceeding 0.8 indicates a reasonably acceptable fit, while values below these thresholds may indicate a lack of compliance with the underlying distribution [35]. It is important to emphasize that R 2 alone [48] is often sufficient to assess the level of agreement between the fitted model and the actual data, and has proven effective in numerous quantitative studies [15,28,49].…”