Faster, Practical GLL Parsing

Abstract

Abstract. Generalized LL (GLL) parsing is an extension of recursivedescent (RD) parsing that supports all context-free grammars in cubic time and space. GLL parsers have the direct relationship with the grammar that RD parsers have, and therefore, compared to GLR, are easier to understand, debug, and extend. This makes GLL parsing attractive for parsing programming languages. In this paper we propose a more e cient Graph-Structured Stack (GSS) for GLL parsing that leads to significant performance improvement. We also discuss a number of optimizations that further improve the performance of GLL. Finally, for practical scannerless parsing of programming languages, we show how common lexical disambiguation filters can be integrated in GLL parsing. Our new formulation of GLL parsing is implemented as part of the Iguana parsing framework. We evaluate the e↵ectiveness of our approach using a highly-ambiguous grammar and grammars of real programming languages. Our results, compared to the original GLL, show a speedup factor of 10 on the highly-ambiguous grammar, and a speedup factor of 1.5, 1.7, and 5.2 on the grammars of Java, C#, and OCaml, respectively.

One parser to rule them all

“…We use a variation of GLL parsing that uses a more efficient GSS [2]. GLL parsing can be seen as a grammar traversal process that is guided by the input.…”

“…In single-phase parsing-scannerless or context-awarelongest match and keyword exclusion have to be applied during parsing, by using lexical disambiguation filters such as follow restrictions [32,36]. These disambiguation filters have parser-specific implementations [2,38]. In Section 3 we show how these filters can be mapped to data-dependent grammars.…”

“…These disambiguation filters are common in scannerless parsing [32] and have been implemented for various generalized parsers [2,38].…”

“…Therefore, it is natural to define them as part of the layout nonterminal. Figure 12 shows relevant parts of the layout definition (Layout) 2 we used to parse C# (follow restrictions enforce longest match). In addition to whitespace characters (Whitespace) and comments (starting with '/ * ' or '//'), the layout consists of declaration directives (Decl) and conditional directives (If).…”

“…In our parsing framework, a global variable is implemented by using parameters and return values to thread a value through a parse. In this case, 2 For readability reasons, we omit uses of Whitespace? (optional whitespace) before and after terminal '#', and uses of Whitespace after terminals 'define', 'undef', 'if', 'elif'.…”