Easy-First Dependency Parsing with Hierarchical Tree LSTMs

Abstract

We suggest a compositional vector representation of parse trees that relies on a recursive combination of recurrent-neural network encoders. To demonstrate its effectiveness, we use the representation as the backbone of a greedy, bottom-up dependency parser, achieving very strong accuracies for English and Chinese, without relying on external word embeddings. The parser's implementation is available for download at the first author's webpage.

LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese

“…Several studies were proposed for improving the accuracy of this method such as Ma et al [20] using beam search strategy to effectively explore the search space of parsing process. Another method was proposed by Kiperwasser & Goldberg [11] which uses deep neural network to learn model's parameters, entire dependency tree was encoded by LSTMs and was applied as deep features for effective learning.…”

“…In this paper, we used an extended version of the easy-first algorithm [11], which employs LSTMs to represent a dependency structure as a vector and used multilayer perceptron (MLP) to score the parse actions. The dependency structure can be described as follows [11]:…”

“…Many works showed that neuralnetwork methods can achieve state-of-the-art results on different tasks of Natural Language Processing such as Named Entity Recognition (NER) [5], Machine Translation [6], [8]. In dependency parsing, there are studies using deep neuralnetworks to encode features without hand-crafted definition [9], [10], [11] and [12].…”

“…We found that [18] did not incorporate pre-trained word embeddings in the vector representation of words. Although this feature can improve the accuracy of dependency parsing because of its rich context information [11] [12] and [17].…”

“…The parser we used is a variant of transition-based parser called tree LSTM easy-first [11] which is fast speed and similar to human annotation [19]. The tree LSTM easy-first is reported as the state-of-the-art parser for Chinese [11]. Because Chinese syntax structures is very similar to Vietnamese, we believe that this parser can give a good accuracy for Vietnamese dependency parsing.…”