Classifying Semantic Clause Types: Modeling Context and Genre Characteristics with Recurrent Neural Networks and Attention

Abstract

Detecting aspectual properties of clauses in the form of situation entity types has been shown to depend on a combination of syntactic-semantic and contextual features. We explore this task in a deeplearning framework, where tuned word representations capture lexical, syntactic and semantic features. We introduce an attention mechanism that pinpoints relevant context not only for the current instance, but also for the larger context. Apart from implicitly capturing task relevant features, the advantage of our neural model is that it avoids the need to reproduce linguistic features for other languages and is thus more easily transferable. We present experiments for English and German that achieve competitive performance. We present a novel take on modeling and exploiting genre information and showcase the adaptation of our system from one language to another.

Explaining Arguments with Background Knowledge

“…Detecting aspectual properties of clauses in the form of semantic clause types has been shown to depend on a combination of syntactic, semantic and contextual features. We explore the task in a deep-learning framework, where tuned word representations capture lexical, syntactic and semantic features [6,7]. Given a clause in its context (previous clauses and previously predicted labels), the model predicts its semantic type (i.e., state, event, generic, generalizing sentence).…”

Building Context-aware Clause Representations for Situation Entity Type Classification

“…In contrast, we focus on deriving dynamic clause representations informed by paragraph-level contexts and model context influences more extensively. Becker et al (2017) proposed a GRU based neural network model that predicts the SE type for one clause each time, by encoding the content of the target clause using a GRU and incorporating several sources of context information, includ-ing contents and labels of preceding clauses as well as genre information, using additional separate GRUs (Chung et al, 2014). This model is different from our approach that processes one paragraph (with a sequence of clauses) at a time and extensively models inter-dependencies of clauses.…”

“…Note that to diminish the effects of randomness in training neural network models and report stable experimental results, we ran each of the proposed models as well as our own baseline models ten times and reported the averaged performance across the ten runs. (Friedrich et al, 2016;Becker et al, 2017), we used the same 80:20 traintest split with balanced genre distributions. Preprocessing: As described in (Friedrich et al, 2016), texts were split into clauses using SPADE (Soricut and Marcu, 2003).…”

“…• CRF (Friedrich et al, 2016): a CRF model that relies heavily on features extracted from the target clause itself. • GRU (Becker et al, 2017): a GRU based neural network model that incorporates context information by using separate GRU units and predicts the SE type for one clause each time. • Humans (Friedrich et al, 2016): one annotator's performance when using two other an- We report accuracy (Acc), macro-average F1-score (Macro) and class-wise F1 scores for STATE (STA), EVENT (EVE), REPORT (REP), GENERIC (GENI), GENERALIZING (GENA), QUESTION (QUE) and IMPERATIVE (IMP).…”

“…Macro Acc CRF (Friedrich et al, 2016) 69.3 74.7 GRU (Becker et al, 2017) 68. notators' annotation as "gold labels". It has been reported that labeling SE types is a nontrivial task even for humans.…”