Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks

Abstract

Recent studies on event detection (ED) have shown that the syntactic dependency graph can be employed in graph convolution neural networks (GCN) to achieve state-of-the-art performance. However, the computation of the hidden vectors in such graph-based models is agnostic to the trigger candidate words, potentially leaving irrelevant information for the trigger candidate for event prediction. In addition, the current models for ED fail to exploit the overall contextual importance scores of the words, which can be obtained via the dependency tree, to boost the performance. In this study, we propose a novel gating mechanism to filter noisy information in the hidden vectors of the GCN models for ED based on the information from the trigger candidate. We also introduce novel mechanisms to achieve the contextual diversity for the gates and the importance score consistency for the graphs and models in ED. The experiments show that the proposed model achieves state-of-the-art performance on two ED datasets.

CLEVE: Contrastive Pre-training for Event Extraction

“…Traditional EE methods (Ji and Grishman, 2008;Gupta and Ji, 2009;Li et al, 2013) rely on manually-crafted features to extract events. In recent years, the neural models become mainstream, which automatically learn effective features with neural networks, including convolutional neural networks (Nguyen and Grishman, 2015;Chen et al, 2015), recurrent neural networks (Nguyen et al, 2016), graph convolutional networks (Nguyen and Grishman, 2018;Lai et al, 2020). With the recent successes of BERT (Devlin et al, 2019), PLMs have also been used for EE (Wang et al, 2019a,b;Yang et al, 2019;Wadden et al, 2019;Tong et al, 2020).…”

“…Previous work has shown that event-related structures are helpful in extracting new events (Lai et al, 2020) as well as discovering and generalizing to new event schemata (Huang et al, 2016(Huang et al, , 2018Huang and Ji, 2020). Hence we conduct event structure pre-training on a GNN as graph encoder to learn transferable event-related structure representations with recent advances in graph contrastive pre-training (Qiu et al, 2020;You et al, 2020;.…”

“…This is done by contrastive learning with the connected word pairs as positive samples and unrelated words as negative samples. Moreover, considering event structures are also helpful in extracting events (Lai et al, 2020) and generalizing to new event schemata (Huang et al, 2018), we need to learn transferable event structure representations. Hence we further introduce a graph neural network (GNN) as the graph encoder to encode AMR structures as structure representations.…”

Fine-Grained Event Trigger Detection

“…ED has been studied extensively in the last decade, featuring feature-based models (Ahn, 2006;Ji and Grishman, 2008;Li et al, 2013Li et al, , 2015, deep learning models (Chen et al, 2015; 2016b,a; Nguyen and Grishman, 2016; Liu et al, 2018;Yan et al, 2019;Ngo et al, 2020;Lai et al, 2020b), and few/zero-shot learning models (Huang et al, 2018;Lai and Nguyen, 2019;Lai et al, 2020a). The rapid development of such models has been facilitated by the availability of the ED datasets in different domains, including the general domain with the popular ACE and TAC KBP datasets (Walker et al, 2006;Mitamura et al, 2015Mitamura et al, , 2016, the biomedical domain (Kim et al, 2009(Kim et al, , 2011, literature (Sims et al, 2019), cybersecurity (Satyapanich et al, 2020;Man Duc Trong et al, 2020), and the open domain (Araki and Mitamura, 2018;Liu et al, 2019b).…”

“…Event triggers represent the most important words (usually single verbs or nominalizations) in the sentences that evoke the events. The current state-of-the-art methods for ED feature the deep learning models where many new network architectures are introduced in the last couple of years Chen et al, 2015;Liu et al, , 2019aLai et al, 2020b).…”