Word lengths are optimized for efficient communication

Abstract

We demonstrate a substantial improvement on one of the most celebrated empirical laws in the study of language, Zipf's 75-y-old theory that word length is primarily determined by frequency of use. In accord with rational theories of communication, we show across 10 languages that average information content is a much better predictor of word length than frequency. This indicates that human lexicons are efficiently structured for communication by taking into account interword statistical dependencies. Lexical systems result from an optimization of communicative pressures, coding meanings efficiently given the complex statistics of natural language use.information theory | rational analysis O ne widely known and apparently universal property of human language is that frequent words tend to be short. This law was popularized by Harvard linguist George Kingsley Zipf, who observed that "the magnitude of words tends, on the whole, to stand in an inverse (not necessarily proportionate) relationship to the number of occurrences" (1).Zipf theorized that this pattern resulted from a pressure for communicative efficiency. Information can be conveyed as concisely as possible by giving the most frequently used meanings the shortest word forms, much like in variable-length (e.g., Huffman) codes. This strategy provided one key exemplar of Zipf's principle of least effort, a grand "principle that governs our entire individual and collective behavior of all sorts, including the behavior of our language" (2). Zipf's idea of assigning word length by frequency can be maximally concise and efficient if words occur independently from a stationary distribution. However, natural language use is highly nonstationary as word probabilities change depending on their context. A more efficient code for meanings can therefore be constructed by respecting the statistical dependencies between words. Here, we show that human lexical systems are such codes, with word length primarily determined by the average amount of information a word conveys in context. The exact forms of the frequency-length relationship (3, 4) and the distribution of word lengths (5) have been quantitatively evaluated previously. In contrast, information content offers an empirically supported and rationally motived alternative to Zipf's frequency-length relationship.A lexicon that assigns word lengths based on information content differs from Zipf's theory in two key ways. First, such a lexicon would not be the most concise one possible as it would not shorten highly informative words, even if shorter distinctive wordforms were available. Second, unlike Zipf's system, assigning word length based on information content keeps the information rate of communication as constant as possible (6). A tendency for this type of "smoothing out" peaks and dips of informativeness is known as uniform information density and has been observed in choices made during online language production (7-10). Formally, uniform information density holds that language users make choices that keep the...

Evolution of word meanings through metaphorical mapping: Systematicity over the past millennium

“…Our analyses provide evidence that the large majority of directions of metaphorical extensions over history can be explained by a compact set of variables, which suggests that metaphor provides an efficient way of compressing emerging meanings into an existing lexicon, without requiring the construction of word forms de novo. Our work thus extends previous studies about communicative and cognitive constraints on synchronic features of language such as word length (Piantadosi et al, 2011;Zipf, 1949) and semantic structures (Kemp & Regier, 2012;Regier et al, 2015) to explain the evolution of metaphorical polysemy in the lexicon (cf. Blank and Koch, 1999;Geeraerts, 1997).…”

Informativity affects consonant duration and deletion rates

“…Piantadosi et al (2011) showed that informativity affects the length (in segments) of words in the lexicons of several European languages, and Seyfarth (2014) found that informativity affects word duration. Adams et al (2009) found that informativity affects the duration and deletion ratios of German codas.…”

“…Informativity has been shown to affect both duration and deletion (Cohen Priva 2008;Cohen Priva and Jurafsky 2008;Adams et al 2009), word-length (measured in segments; Piantadosi et al 2011), and word duration, after word predictability and phonological structure have been controlled for (Seyfarth 2014).…”

“…Predictable words tend to have shorter duration than their phonological structure would otherwise predict (Bell et al 2009). But words that are usually predictable (that have low informativity) also tend to have fewer segments than words that are usually unpredictable, as demonstrated cross-linguistically by Piantadosi et al (2011). While the duration of some word token can be affected by a particular speaker's performance, speakers cannot easily change the number of segments in a given word in order to make it shorter or longer, if they want to be understood.…”

Predicting head-marking variability in Yucatec Maya relative clause production

“…For information-theoretic approaches to language production in particular, the empirical base is further limited, since this work tends to require larger databases on the basis of which informativity can be estimated (Bell, Brenier, Gregory, Girand, & Jurafsky, 2009 ;Piantadosi et al, 2011 ;Resnik, 1996 ). To the extent that cross-linguistic investigations within information-theoretic frameworks exist, they have thus mostly focused on lexical and sublexical properties, i.e., levels of linguistic description for which units are more frequent (e.g., Graff & Jaeger, 2009;Pellegrino, Coupé, & Marsico, 2011 ;Piantadosi et al, 2011 ;Qian & Jaeger, 2012 ;Wedel et al, 2013 ). Above the lexical level, some suggestive cross-linguistic support for communicative effi ciency in fact comes from the comprehension research fi nding that comprehenders e x pe c t speakers to produce reduced forms when the meaning the form conveys is contextually expected, and less reduced forms otherwise.…”