Efficient Constituency Parsing by Pointing

Abstract

We propose a novel constituency parsing model that casts the parsing problem into a series of pointing tasks. Specifically, our model estimates the likelihood of a span being a legitimate tree constituent via the pointing score corresponding to the boundary words of the span. Our parsing model supports efficient top-down decoding and our learning objective is able to enforce structural consistency without resorting to the expensive CKY inference. The experiments on the standard English Penn Treebank parsing task show that our method achieves 92.78 F1 without using pre-trained models, which is higher than all the existing methods with similar time complexity. Using pre-trained BERT, our model achieves 95.48 F1, which is competitive with the state-of-theart while being faster. Our approach also establishes new state-of-the-art in Basque and Swedish in the SPMRL shared tasks on multilingual constituency parsing.

A Conditional Splitting Framework for Efficient Constituency Parsing

“…Train Valid Test From the results in Table 4, we see that our model achieves the highest F1 in French, Hungarian and Korean and higher than the best baseline by 0.06, 0.15 and 0.13, respectively. Our method also rivals existing SoTA methods on other languages even though some of them use predicted POS tags (Nguyen et al, 2020) or bigger models (75M parameters) (Kitaev and Klein, 2018). Meanwhile, our model is smaller (31M), uses no extra information and runs 40% faster.…”

“…At each step t, the decoder autoregressively predicts the split point k t in the input by conditioning on the current input span (i t , j t ) and previous splitting decisions (i, j) ) k) <t . This conditional splitting formulation (decision at step t depends on previous steps) can help our model to find better trees compared to non-conditional top-down parsers (Stern et al, 2017a;Shen et al, 2018;Nguyen et al, 2020), thus bridging the gap between the global (but expensive) and the local (but efficient) models. The labels L(T ) can be modeled by using a label classifier, as described later in the next section.…”

“…where e char t , e word t are respectively the character and word embeddings of token t. Similar to (Kitaev and Klein, 2018;Nguyen et al, 2020), we use a character LSTM to compute the character embedding of a token. We experiment with both randomly initialized and pretrained token embeddings.…”

“…• The conditional probabilities of the splitting decisions are optimized using a cross entropy loss and structural consistency is maintained through a global pointing mechanism. The training process can be fully parallelized without requiring structured inference as in (Shen et al, 2018;Gómez and Vilares, 2018;Nguyen et al, 2020).…”

“…where l t is the label of the text span (i t , j t ) encompassing tokens from index i t to index j t . Previous approaches to syntactic parsing (Stern et al, 2017a;Kitaev and Klein, 2018;Nguyen et al, 2020) train a neural model to score each possible span and then apply a greedy or dynamic programming algorithm to find the parse tree. In other words, these methods are span-based formulation.…”