Building Context-aware Clause Representations for Situation Entity Type Classification

Abstract

Capabilities to categorize a clause based on the type of situation entity (e.g., events, states and generic statements) the clause introduces to the discourse can benefit many NLP applications. Observing that the situation entity type of a clause depends on discourse functions the clause plays in a paragraph and the interpretation of discourse functions depends heavily on paragraph-wide contexts, we propose to build context-aware clause representations for predicting situation entity types of clauses. Specifically, we propose a hierarchical recurrent neural network model to read a whole paragraph at a time and jointly learn representations for all the clauses in the paragraph by extensively modeling context influences and inter-dependencies of clauses. Experimental results show that our model achieves the state-of-the-art performance for clause-level situation entity classification on the genrerich MASC+Wiki corpus, which approaches human-level performance.

Proceedings of the Fifteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-15)

“…Xu et al (2015) propose a shortest dependency path LSTM for sentence representation in the task of relation classification. Dai and Huang (2018) propose a BiLSTM based model that combines paragraph vectors and word vectors into clause embeddings for a situation entity classification task. Connective prediction has often been addressed: Ji and Eisenstein (2015) and Rutherford et al (2017) use recursive neural networks with parse trees as input to predict connectives and discourse relations, with solid improvements on PDTB.…”

Learning Clause Representation from Dependency-Anchor Graph for Connective Prediction

“…Xu et al (2015) propose a shortest dependency path LSTM for sentence representation in the task of relation classification. Dai and Huang (2018) propose a BiLSTM based model that combines paragraph vectors and word vectors into clause embeddings for a situation entity classification task. Connective prediction has often been addressed: Ji and Eisenstein (2015) and Rutherford et al (2017) use recursive neural networks with parse trees as input to predict connectives and discourse relations, with solid improvements on PDTB.…”

Zero-shot fine-grained entity typing in information security based on ontology

“…Several studies on entity typing have been conducted in the past [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]. In general, these studies can be divided into unsupervised and semi-supervised groups depending on the approach used.…”

“…By contrast, the semi-supervised approach focuses on learning the representation of categories or entities. Dai et al [24] proposed context-aware clause representations to predict the situation entity types of the clauses. In recent years, language pretrained models have been developed rapidly.…”

“…In addition, from the related literature on fine-grained entity typing [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25], we determined that few researchers have considered the problem of misclassification in coarse-grained entity typing tasks. However, the inaccuracy in the results of coarse-grained typing certainly results in performance degradation in fine-grained entity typing tasks.…”