Directly-Executable Earley Parsing

Aycock, John; Horspool, R. Nigel

doi:10.1007/3-540-45306-7_16

Cited by 12 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current research is making these algorithms faster [3,4,5,11,17]. My colleague and I have recently made an Earley parser run in time comparable to Bison.…”

Section: The Case For General Parsing Algorithmsmentioning

confidence: 95%

Why bison is becoming extinct

Aycock¹

2001

XRDS

Self Cite

View full text Add to dashboard Cite

At some point in your career, you're going to implement a computer language. You probably won't be implementing Java or C++. You may not even recognize it as a language. Truth be told, there are an awful lot of domain-specific languages, or "little languages" [7] in common use: configuration files, HTML/XML documents, shell scripts, network protocols, mail headers, command-line arguments. The list goes on. A number of programs allow you to write scripts to control their operation; infact, just the other day I downloaded a neural network simulator which provided a little programming language to steer the simulation.How will you implement your language? There's the ad hoc approach, of course, but it's not well suited to languages whose design is complex or frequently changing. You also end up writing code to perform tasks which can be effectively automated.You might also consider using existing languages like Tcl [18] and Python [6]. These languages are designed to either be embedded in an existing application, or easily extended. This is a good solution when it can be used, saving a lot of time and effort. However, there may be concerns about tying your language to one which is itself changing, or the syntax and semantics of your language may not match those of such a "host" language.A third approach is to use compiler tools to implement your language. Most were designed for the implementation of large programming languages, but the same principles and techniques apply equally well to little languages. This article is the story of one such tool -- a parser generator tool -- and more importantly, what sort of tool is going to replace it, and why.

show abstract

“…Current research is making these algorithms faster [3,4,5,11,17]. My colleague and I have recently made an Earley parser run in time comparable to Bison.…”

Section: The Case For General Parsing Algorithmsmentioning

confidence: 95%

Why bison is becoming extinct

Aycock¹

2001

XRDS

Self Cite

View full text Add to dashboard Cite

show abstract

“…Only manually re-writing the grammar (that will end-up resembling a lot like the PEG version, studied below) can produce an unambiguous Earley parse. Later enhancements to the algorithm, have focused on empty-rule handling [3] parallelism [11], complexity [30,27] and performance [2,31,32]. The ambiguous output of Earley is widely considered a feature (especially for Natural Language Processing) rather than a problem, with the exception of precedence handling through external directives [27].…”

Section: Problem and Related Workmentioning

confidence: 99%

“…Since their introduction in 2004, Parsing Expression Grammars have been gaining widespread adoption both in industry and academia. More than 400 subsequent works 1 cite B. Ford's original paper, while a total of 29 implementations in 14 different programming languages are reported in active use 2 . Nevertheless reviewing PEG-related bibliography reveals that the original argumentation in favor of PEGs has actually been weakened by subsequent work, regarding basic parsing features such as (a) recursion handling and (b) associativity support.…”

mentioning

confidence: 99%

Parsing multi-ordered grammars with the Gray algorithm

Papoulias

2019

Preprint

View full text Add to dashboard Cite

Background. Context-free grammars (CFGs) and Parsing-expression Grammars (PEGs) are the two main formalisms used by formal specifications and parsing frameworks to describe programming languages. They mainly differ in the definition of the choice operator, describing language alternatives. CFGs support the use of non-deterministic choice (i.e., unordered choice), where all alternatives are equally explored. PEGs support a deterministic choice (i.e., ordered choice), where alternatives are explored in strict succession. In practice the two formalisms, are used through concrete classes of parsing algorithms (such as Left-to-right, rightmost derivation (LR) for CFGs and Packrat parsing for PEGs), that follow the semantics of the formal operators. Problem Statement. Neither the two formalisms, nor the accompanying algorithms are sufficient for a complete description of common cases arising in language design. In order to properly handle ambiguity, recursion, precedence or associativity, parsing frameworks either introduce implementation specific directives or ask users to refactor their grammars to fit the needs of the framework/algorithm/formalism combo. This introduces significant complexity even in simple cases and results in incompatible grammar specifications. Our Proposal. We introduce Multi-Ordered Grammars (MOGs) as an alternative to the CFG and PEG formalisms. MOGs aim for a better exploration of ambiguity, ordering, recursion and associativity during language design. This is achieved by (a) allowing both deterministic and non-deterministic choices to co-exist, and (b) introducing a form of recursive and scoped ordering. The formalism is accompanied by a new parsing algorithm (Gray) that extends chart parsing (normally used for Natural Language Processing) with the proposed MOG operators. Results. We conduct two case-studies to assess the expressiveness of MOGs, compared to CFGs and PEGs. The first consists of two idealized examples from literature (an expression grammar and a simple procedural language). The second examines a real-world case (the entire Smalltalk grammar and eleven new Smalltalk extensions) probing the complexities of practical needs. We show that in comparison, MOGs are able to reduce complexity and naturally express language constructs, without resorting to implementation specific directives. Conclusion. We conclude that combining deterministic and non-deterministic choices in a single grammar specification is indeed not only possible but also beneficial. Moreover, augmented by operators for recursive and scoped ordering the resulting multi-ordered formalism presents a viable alternative to both CFGs and PEGs. Concrete implementations of MOGs can be constructed by extending chart parsing with MOG operators for recursive and scoped ordering.

show abstract

“…An Earley parser variation described in [4] moves in a different direction: the authors of [4] propose a version of Earley parser that is suitable for parsing reflective (more often referred as adaptable) grammars, that can be modified during parsing to augment the input language. A separate paper [5], describes how to translate Earley grammars into C programming language, thus eliminating many of dynamic elements of the original parser and improving the overall parsing performance.…”

Section: Related Workmentioning

confidence: 99%

Parsing with Earley Virtual Machines

Šaikūnas¹

2017

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Abstract-Earley parser is a well-known parsing method used to analyse context-free grammars. While being less efficient in practical contexts than other generalized context-free parsing algorithms, such as GLR, it is also more general. As such it can be used as a foundation to build more complex parsing algorithms.We present a new, virtual machine based approach to parsing, heavily based on the original Earley parser. We present several variations of the Earley Virtual Machine, each with increasing feature set. The final version of the Earley Virtual Machine is capable of parsing context-free grammars with data-dependant constraints and support grammars with regular right hand sides and regular lookahead.We show how to translate grammars into virtual machine instruction sequences that are then used by the parsing algorithm. Additionally, we present two methods for constructing shared packed parse forests based on the parse input.

show abstract

Directly-Executable Earley Parsing

Cited by 12 publications

References 19 publications

Why bison is becoming extinct

Why bison is becoming extinct

Parsing multi-ordered grammars with the Gray algorithm

Parsing with Earley Virtual Machines

Contact Info

Product

Resources

About