Content Differences in Syntactic and Semantic Representation

Abstract

Syntactic analysis plays an important role in semantic parsing, but the nature of this role remains a topic of ongoing debate. The debate has been constrained by the scarcity of empirical comparative studies between syntactic and semantic schemes, which hinders the development of parsing methods informed by the details of target schemes and constructions. We target this gap, and take Universal Dependencies (UD) and UCCA as a test case. After abstracting away from differences of convention or formalism, we find that most content divergences can be ascribed to: (1) UCCA's distinction between a Scene and a non-Scene;(2) UCCA's distinction between primary relations, secondary ones and participants; (3) different treatment of multi-word expressions, and (4) different treatment of inter-clause linkage. We further discuss the long tail of cases where the two schemes take markedly different approaches. Finally, we show that the proposed comparison methodology can be used for fine-grained evaluation of UCCA parsing, highlighting both challenges and potential sources for improvement. The substantial differences between the schemes suggest that semantic parsers are likely to benefit downstream text understanding applications beyond their syntactic counterparts.

TUPA at MRP 2019: A Multi-Task Baseline System

“…We also train and test TUPA with BERT embeddings on v1.0 of the UCCA English Web Treebank (EWT) reviews dataset (Hershcovich et al, 2019a). While the EWT reviews are included in the MRP shared task UCCA data, the different format and preprocessing makes for slightly different scores, so we report the scores for comparability with previous work in Table 5.…”

“…TUPA (w/o BERT) 71.0 72.1 47.0 TUPA (w/ BERT) 75.2 76.1 54.8 Table 5: Test UCCA F-score scores (in %) on all edges, primary edges and remote edges, on the UCCA EWT reviews data. TUPA (w/o BERT) is from (Hershcovich et al, 2019a). TUPA (w/ BERT) is averaged over three separately trained models in each setting, differing only by random seed (standard deviation < 0.03).…”

Semantic Role Labeling as Syntactic Dependency Parsing

“…3 SD is semantically-friendly as noted by Schuster andManning (2016, p. 2371), "Since its first version, SD representation has had the status of being both a syntactic and a shallow semantic representation", thus it is suited for the development of our joint modeling of syntactic and semantic structures. Indeed, Universal Dependencies (UD; Nivre et al, 2016), which builds upon SD, has been compared with and aligned to meaning representations including UCCA (Hershcovich et al, 2019) and AMR (Szubert et al, 2018). 4…”

Benchmarking Meaning Representations in Neural Semantic Parsing

“…Abend and Rappoport (2017) conducted a survey on ungrounded MRs to assess their achievements and shortcomings. Hershcovich et al (2019) evaluated the similarities and divergences in the content encoded by ungrounded MRs and syntactic representation. Lin and Xue (2019) carried out a careful analysis on AMR and MRS to understand the factors contributing to the discrepancy in their parsing accuracy.…”