Safe Specification of Operator Precedence Rules

Abstract

Abstract. In this paper we present an approach to specifying operator precedence based on declarative disambiguation constructs and an implementation mechanism based on grammar rewriting. We identify a problem with existing generalized context-free parsing and disambiguation technology: generating a correct parser for a language such as OCaml using declarative precedence specification is not possible without resorting to some manual grammar transformation. Our approach provides a fully declarative solution to operator precedence specification for contextfree grammars, is independent of any parsing technology, and is safe in that it guarantees that the language of the resulting grammar will be the same as the language of the specification grammar. We evaluate our new approach by specifying the precedence rules from the OCaml reference manual against the highly ambiguous reference grammar and validate the output of our generated parser.

One parser to rule them all

“…For each challenge we selected a programming language that exhibits it, and wrote a data-dependent grammar 3 , derived from the specification grammar of the language. For evaluation, we parsed real source files from the source distribution of the language and some popular open source libraries, see Table 2.…”

“…To test the correctness of parsers resulting from the desugaring of >, left, and right to data-dependent grammars, we tested their resulting parse trees against a GLL parser for the same natural grammar of Java, using our previous work on rewriting operator precedence rules [3]. Both parsers, using desugaring to data-dependent grammars and rewriting operator precedence rules, produced the same parse trees for all Java files, providing an evidence that our desugaring of operator precedence to data-dependent grammars implements the same semantics as the rewriting in [3]. Despite its prototype status, the data-dependent parser is at the moment on average only 25% slower than the rewritten one.…”

“…Despite its prototype status, the data-dependent parser is at the moment on average only 25% slower than the rewritten one. The main reason for performance difference is that in the rewriting technique [3] the precedence information is statically encoded in the grammar, and therefore there is no runtime overhead, while in the data-dependent version passing arguments and handling environment is done at runtime. The problem with the rewriting technique is that the rewriting process itself is rather slow and the resulting grammar is very large.…”

“…Given that * normally has higher precedence than +, the first derivation tree is desirable. We use >, left, and right to define priority and left-and rightassociativity, respectively [3]. Figure 6 (right) shows the disambiguated version of this grammar by specifying > and left, where -has the highest precedence, and * and + are left-associative.…”

“…In earlier work [3] we discussed the precedence ambiguity, and proposed a grammar rewriting that takes an ambiguous grammar with a set of precedence rules and produces a grammar that does not allow precedence-invalid derivations. Our current solution has the same semantics: it does not remove sentences when there is no precedence ambiguity, and can deal with corner cases found in programming languages such as OCaml.…”