Stack-Pointer Networks for Dependency Parsing

Abstract

We introduce a novel architecture for dependency parsing: stack-pointer networks (STACKPTR). Combining pointer networks (Vinyals et al., 2015) with an internal stack, the proposed model first reads and encodes the whole sentence, then builds the dependency tree top-down (from root-to-leaf) in a depth-first fashion. The stack tracks the status of the depthfirst search and the pointer networks select one child for the word at the top of the stack at each step. The STACKPTR parser benefits from the information of the whole sentence and all previously derived subtree structures, and removes the leftto-right restriction in classical transitionbased parsers. Yet, the number of steps for building any (including non-projective) parse tree is linear in the length of the sentence just as other transition-based parsers, yielding an efficient decoding algorithm with O(n 2 ) time complexity. We evaluate our model on 29 treebanks spanning 20 languages and different dependency annotation schemas, and achieve state-of-theart performance on 21 of them.

Hierarchical Pointer Net Parsing

“…3.1 Pointer Networks for Parsing. Ma et al (2018) and Lin et al (2019) both use a pointer network as the backbone of their parsing models and achieve state-of-the-art performance in dependency and discourse parsing tasks, respectively. As shown in Figures 2(a) and 3, in both cases, the parsing algorithm is implemented in a top-down depth-first order.…”

“…Given a sentence of length n, the number of decoding steps to build a parse tree is linear. The attention mechanism at each decoding step computes an attention vector of length n. The overall decoding complexity is O(n 2 ), which is same as the StackPointer Parser (Ma et al, 2018).…”

“…Recently, transition-based top-down parsing with Pointer Networks (Vinyals et al, 2015) has attained state-of-the-art results in both dependency and discourse parsing tasks with the same computational efficiency (Ma et al, 2018;Lin et al, 2019); thanks to the encoder-decoder architecture that makes it possible to capture information from the whole text and the previously derived subtrees, while limiting the number of parsing steps to linear. However, the decoder of these parsers has a sequential structure, which may not yield the most appropriate inductive bias for deriving a hierarchical structure.…”

Cross-Lingual Dependency Parsing with Unlabeled Auxiliary Languages

“…The encoder and the decoder jointly form the parsing model and we consider two alternatives 2 from (Ahmad et al, 2019): "SelfAtt-Graph" and "RNN-Stack". The "SelfAtt-Graph" parser consists of a modified self-attentional encoder (Shaw et al, 2018) and a graph-based deep bi-affine decoder (Dozat and Manning, 2017), while the "RNN-Stack" parser is composed of a Recurrent Neural Network (RNN) based encoder and a stack-pointer decoder (Ma et al, 2018).…”

“…To train the parser, we adopt both cross-entropy objectives for these two types of parsers as in (Dozat and Manning, 2017;Ma et al, 2018). The encoder and the decoder are jointly trained to optimize the probability of the dependency trees (y) given sentences (x):…”