Optimization Models of Natural Communication

Ferrer-i-Cancho, Ramon

doi:10.1080/09296174.2017.1366095

Cited by 33 publications

(37 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The reason for (1) is that our theory (e.g., Q) assumes a tree structure [19,28] and that we wanted to avoid the statistical problem of mixing trees with other kinds of graphs, e.g., the potential number of crossings depends on the number of edges [19,27,63]. The reason for (2) is that crossings are impossible in a star tree [63]. Condition (2) implies that the syntactic dependency structure has at least four vertices (otherwise all the possible trees are star trees).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

Gómez-Rodríguez

Ferrer-i-Cancho

2017

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

The structure of a sentence can be represented as a network where vertices are words and edges indicate syntactic dependencies. Interestingly, crossing syntactic dependencies have been observed to be infrequent in human languages. This leads to the question of whether the scarcity of crossings in languages arises from an independent and specific constraint on crossings. We provide statistical evidence suggesting that this is not the case, as the proportion of dependency crossings of sentences from a wide range of languages can be accurately estimated by a simple predictor based on a null hypothesis on the local probability that two dependencies cross given their lengths. The relative error of this predictor never exceeds 5% on average, whereas the error of a baseline predictor assuming a random ordering of the words of a sentence is at least 6 times greater. Our results suggest that the low frequency of crossings in natural languages is neither originated by hidden knowledge of language nor by the undesirability of crossings per se, but as a mere side effect of the principle of dependency length minimization.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Here we provide a quick overview of a crossing theory developed in a series of articles [18,19,27,28,63]. It is correct to state that C cannot exceed the number of pairs of different edges, namely…”

Section: Crossing Theorymentioning

confidence: 99%

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

Gómez-Rodríguez

Ferrer-i-Cancho

2017

Phys. Rev. E

Self Cite

View full text Add to dashboard Cite

show abstract

“…Zipf's law is an example of power-law model for the relationship between two variables [4]. Zipf's law for word frequencies can be explained by information theoretic models of communication [5] and is a robust pattern of language that presents invariance with text length in a sufficiently long text [6], and little sensitivity with respect to the linguistic units considered [7]. The focus of this paper is to test the robustness of two statistical laws in linguistics that have been studied less intensively:…”

Section: Introductionmentioning

confidence: 99%

Polysemy and brevity versus frequency in language

Fernández

Hernández-Fernández

Català

et al. 2019

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

The pioneering research of G. K. Zipf on the relationship between word frequency and other word features led to the formulation of various linguistic laws. The most popular is Zipf's law for word frequencies. Here we focus on two laws that have been studied less intensively: the meaning-frequency law, i.e. the tendency of more frequent words to be more polysemous, and the law of abbreviation, i.e. the tendency of more frequent words to be shorter. In a previous work, we tested the robustness of these Zipfian laws for English, roughly measuring word length in number of characters and distinguishing adult from child speech. In the present article, we extend our study to other languages (Dutch and Spanish) and introduce two additional measures of length: syllabic length and phonemic length. Our correlation analysis indicates that both the meaning-frequency law and the law of abbreviation hold overall in all the analyzed languages.

show abstract

“…Zipf's law for word frequencies, for instance, emerges in a critical balance between these two forces [39]. In these models, entropy minimization is linked with learnability: fewer word forms are easier to learn (see [80] for other cognitive costs associated with entropy). Whereas mutual information maximization is linked with expressivity via the form/meaning mappings available in a communication system.…”

Section: Entropy Diversity Across Languages Of the Worldmentioning

confidence: 99%

The Entropy of Words—Learnability and Expressivity across More than 1000 Languages

Bentz

Alikaniotis

Cysouw

et al. 2017

Entropy

Self Cite

View full text Add to dashboard Cite

Abstract:The choice associated with words is a fundamental property of natural languages. It lies at the heart of quantitative linguistics, computational linguistics and language sciences more generally. Information theory gives us tools at hand to measure precisely the average amount of choice associated with words: the word entropy. Here, we use three parallel corpora, encompassing ca. 450 million words in 1916 texts and 1259 languages, to tackle some of the major conceptual and practical problems of word entropy estimation: dependence on text size, register, style and estimation method, as well as non-independence of words in co-text. We present two main findings: Firstly, word entropies display relatively narrow, unimodal distributions. There is no language in our sample with a unigram entropy of less than six bits/word. We argue that this is in line with information-theoretic models of communication. Languages are held in a narrow range by two fundamental pressures: word learnability and word expressivity, with a potential bias towards expressivity. Secondly, there is a strong linear relationship between unigram entropies and entropy rates. The entropy difference between words with and without co-textual information is narrowly distributed around ca. three bits/word. In other words, knowing the preceding text reduces the uncertainty of words by roughly the same amount across languages of the world.

show abstract

Optimization Models of Natural Communication

Cited by 33 publications

References 82 publications

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

Polysemy and brevity versus frequency in language

The Entropy of Words—Learnability and Expressivity across More than 1000 Languages

Contact Info

Product

Resources

About