A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling

Abstract

Semantic role labeling (SRL) aims to identify the predicate-argument structure of a sentence. Inspired by the strong correlation between syntax and semantics, previous works pay much attention to improve SRL performance on exploiting syntactic knowledge, achieving significant results. Pipeline methods based on automatic syntactic trees and multi-task learning (MTL) approaches using standard syntactic trees are two common research orientations. In this paper, we adopt a simple unified span-based model for both span-based and word-based Chinese SRL as a strong baseline. Besides, we present a MTL framework that includes the basic SRL module and a dependency parser module. Different from the commonly used hard parameter sharing strategy in MTL, the main idea is to extract implicit syntactic representations from the dependency parser as external inputs for the basic SRL model. Experiments on the benchmarks of Chinese Proposition Bank 1.0 and CoNLL-2009 Chinese datasets show that our proposed framework can effectively improve the performance over the strong baselines. With the external BERT representations, our framework achieves new state-of-the-art 87.54 and 88.5 F1 scores on the two test data of the two benchmarks, respectively. In-depth analysis are conducted to gain more insights on the proposed framework and the effectiveness of syntax.

Syntax-Aware Opinion Role Labeling with Dependency Graph Convolutional Networks

“…Taking the example in Figure 1, the Target span is often incompletely recognized without syntactic dependency relations, missing either "facing Chavez" or "challenge". For the similar SRL task, many previous works have proposed to incorporate syntax into the neural models He et al, 2018;Xia et al, 2019a). In contrast, few studies in the recent years explore this line of research for ORL.…”

“…extract neural features from a welltrained SRL model as SRL-aware word representations, and then feed them into the input layer of ORL, aiming to alleviate the error propagation problem. Many previous works have shown that syntactic information is of great value for SRL and other NLP tasks (He et al, 2018;Zhang et al, 2019c;Strubell et al, 2018;Xia et al, 2019a;Miwa and Bansal, 2016;. Xia et al (2019b) use the relative position between predicate words and other words in a dependency tree to represent syntactic information, while Roth and Lapata (2016) employ LSTM to obtain the embedding of a dependency path.…”

“…Both TreeLSTM and GCN are commonly used techniques to encode parse trees (Miwa and Bansal, 2016;Bastings et al, 2017). and Xia et al (2019a) extract the hidden states from the LSTM encoder of the parser model as syntax-aware word representations, and feed them to downstream tasks as extra inputs.…”

“…On the one hand, compared with explicit 1-best parse trees, edge-weighted graphs convey richer structural information by providing all latent syntactic structures, and avoid error propagation as well. On the other hand, compared with the implicit hidden states of the parser encoder Xia et al, 2019a), an edgeweighted graph, denoted as an attention matrix, explicitly captures the modification strength of word pairs. The original GCN is designed for directly modeling graph-structured data (Kipf and Welling, 2016).…”

Semantic Role Labeling as Syntactic Dependency Parsing

“…This is an instance of multi-task learning (MTL;Caruana, 1993Caruana, , 1997. MTL has been successfully applied to SRL (Collobert and Weston, 2008;Collobert et al, 2011;Shi et al, 2016) in many state-of-the-art systems (Strubell et al, 2018;Swayamdipta et al, 2018;Cai and Lapata, 2019;Xia et al, 2019a). A potential future extension is to learn multiple syntactic (Søgaard and Goldberg, 2016) and semantic representations (Peng et al, 2017;Hershcovich et al, 2018) beyond dependency trees and PropBank-style SRL at the same time.…”