Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

Abstract

Although current CCG supertaggers achieve high accuracy on the standard WSJ test set, few systems make use of the categories’ internal structure that will drive the syntactic derivation during parsing. The tagset is traditionally truncated, discarding the many rare and complex category types in the long tail. However, supertags are themselves trees. Rather than give up on rare tags, we investigate constructive models that account for their internal structure, including novel methods for tree-structured prediction. Our best tagger is capable of recovering a sizeable fraction of the long-tail supertags and even generates CCG categories that have never been seen in training, while approximating the prior state of the art in overall tag accuracy with fewer parameters. We further investigate how well different approaches generalize to out-of-domain evaluation sets.

Proof Net Structure for Neural Lambek Categorial Parsing

“…A key aspect of our parser is that it makes use of a structured decomposition of lexical categories in categorial grammars. In this sense, our work follows up on the intuition of recent "constructive" supertaggers, which have been explored for a type-logical grammar (Kogkalidis et al, 2019) and for CCG (Bhargava and Penn, 2020;Prange et al, 2021). Such supertaggers construct categories out of the atomic categories of the grammar; this challenges the classical approach to supertagging, where lexical categories are treated as opaque, rendering the task of supertagging equivalent to large-tagset POS tagging.…”

Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets

“…By convention, the model is limited to predicting only tags that appeared at least 10 times in the training data, yielding 425 tags + the UNK tag. We use the non-constructive BERT-based (Devlin et al, 2019) model from (Prange et al, 2021) with its default hyperparameters. The tagger was trained on 927,497 tokens and obtained a dev accuracy of 96.1%.…”

“…Many linguistic phenomena follow power law distributions and thus feature a long tail of individually rare events, which, as we will show, makes it nontrivial to measure calibration error with existing methods, including marginal calibration error (MCE), which requires sufficient samples of each class to produce a reliable estimate (Kumar et al, 2019). We evaluate two English sentence taggers 1 with closed sets of 100s of tags that disambiguate word tokens: a Combinatory Categorial Grammar (CCG) syntactic supertagger with 426 tags (Prange et al, 2021), and a Lexical Semantic Recognition (LSR) tagger with 598 tags (Liu et al, 2021).…”

A New Representation for Span-based CCG Parsing

“…Combinatory Categorial Grammar (CCG) (Steedman, 2000) is a mildly context-sensitive grammar formalism. Several neural CCG parsing methods have been proposed so far (Lewis and Steedman, 2014;Xu et al, 2015;Vaswani et al, 2016;Xu, 2016;Yoshikawa et al, 2017;Steedman, 2019, 2020;Bhargava and Penn, 2020;Tian et al, 2020;Prange et al, 2021;Liu et al, 2021). Currently, neural span-based models (Cross and Huang, 2016;Stern et al, 2017;Gaddy et al, 2018;Kitaev and Klein, 2018) have been successful in the field of constituency parsing.…”

“…Furthermore, as a by-product of our representation, the parsing models can assign outof-vocabulary (OOV) categories, which have not appeared in training data. This characteristic has been attracting attention in CCG parsing research (Bhargava and Penn, 2020;Prange et al, 2021;Liu et al, 2021). Our experimental result shows that an off-the-shelf span-based parser with our representation is comparable with previous CCG parsers and can generate correct OOV categories.…”