We consider languages generated by weighted context-free grammars. It is shown that the behavior of large texts is controlled by saddle-point equations for an appropriate generating function. We then consider ensembles of grammars, in particular the Random Language Model of [1]. This model is solved in the replicasymmetric ansatz, which is valid in the high-temperature, disordered phase. It is shown that in the phase in which languages carry information, the replica symmetry must be broken.Many complex systems have a generative, or linguistic, aspect. For example, protein structure is written in sequences of amino acids, a language of 20 different symbols. A large body of previous work has investigated the social aspect of linguistic systems, namely that different agents must find consensus regarding the meaning of symbols [2,3,4]. A complementary but necessary aspect of any linguistic system concerns the hidden structure within the sequences themselves, independent of communication. The most basic structural property is syntax: the rules that govern how symbols can be combined to create richer structures and thus carry information. In computer science and linguistics, generative grammar has proved to be a valuable formalism to describe syntax, in a generalized sense [5,6,7]. A generative grammar consists of an alphabet of hidden symbols, an alphabet of observable symbols, and a set of rules, which allow certain combinations of symbols to be replaced by others. From an initial start symbol S, one progressively applies the rules until only observable symbols remain; any sentence produced this way is said to be grammatical, and the set of all such sentences is called the language of the grammar. The sequence of rule applications is called a derivation. The Chomsky hierarchy distinguishes grammars based on the complexity of the grammatical rules. In this work, we restrict our attention to context-free grammars (CFGs), for which derivations are trees (Figure 1).There are many theoretical results on the capabilities of CFGs [7]. However, little is known about the statistical properties of large, typical grammars. Recently, there has been increasing interest in approaching the properties of syntax from the point of arXiv:1902.07516v2 [cond-mat.dis-nn]