Efficient and flexible incremental parsing

Abstract

Previously published algorithms for LR ( k ) incremental parsing are inefficient, unnecessarily restrictive, and in some cases incorrect. We present a simple algorithm based on parsing LR( k ) sentential forms that can incrementally parse an arbitrary number of textual and/or structural modifications in optimal time and with no storage overhead. The central role of balanced sequences in achieving truly incremental behavior from analysis algorithms is described, along with automated methods to support balancing during parse table generation and parsing. Our approach extends the theory of sentential-form parsing to allow for ambiguity in the grammar, exploiting it for notational convenience, to denote sequences, and to construct compact (“abstract”) syntax trees directly. Combined, these techniques make the use of automatically generated incremental parsers in interactive software development environments both practical and effective. In addition, we address information preservation in these environments: Optimal node reuse is defined; previous definitions are shown to be insufficient; and a method for detecting node reuse is provided that is both simpler and faster than existing techniques. A program representation based on self-versioning documents is used to detect changes in the program, generate efficient change reports for subsequent analyses, and allow the parsing transformation itself to be treated as a reversible modification in the edit log.

Determinization and minimization of finite acyclic automata by incremental techniques

“…Many of the limitations of LR(k) incremental parsing techniques, including inefficiency and unnecessary restrictiveness, are overcome in [36], which aims at making automatically generated incremental parsers in interactive software development environments both practical and effective.…”

Design of methodology for incremental compiler construction

“…There have been suggested many methods for incremental analysis. Algorithms of incremental compilation of context-free language class LL (Yang, 1993;Li, 1995;Li, 1996;Shilling, 1992;Murching, Prasad and Srikant, 1990;Ferro and Dion, 1994) and LR (Yang, 1994;Agrawal and Detrich, 1983;Tomita, 1987;Horspool, 1990;Wagner and Graham, 1997;Wagner and Graham, 1998;Wagner, 1998) have been published. Melichar and Vagner (2008) published an elementary text about how to create an incremental parser of LL(1) languages as a part of educational materials in English, based on previous publications of Li (1995), but suitably accompanied by examples and illustrations.…”

Bridging the chasm between MDE and the world of compilation

“…As an example, smart error recovery schemes [18] are used either to help programmers find the cause of an error or to suggest possible corrections. Similarly incremental/lazy parsing [19] is also receiving more and more interest. This comes from the fact that syntactic analysis is nowadays tightly coupled to programming environment (e.g., it enables on-the-fly syntactic checking).…”