Named Entity Recognition and Relation Extraction Using Enhanced Table Filling by Contextualized Representations

Abstract

In this study, we propose a method designed to extract named entities and relations from unstructured text based on table representations. To extract named entities, the proposed method computes representations for entity mentions and long-range dependencies using contextualized representations without hand-crafted features or complex neural network architectures. To extract relations, it applies a tensor dot product to predict all relation labels simultaneously without considering dependencies among relation labels. These advancements significantly simplify the proposed model and the associated algorithm for the extraction of named entities and relations. Despite its simplicity, the experimental results demonstrate that the proposed approach outperformed the state of the-art methods on multiple datasets. Compared with existing table-filling approaches, the proposed method achieved high performance solely by independently predicting the relation labels. In addition, we found that incorporating dependencies of relation labels into the system obtained little performance gain, indicating the effectiveness and sufficiency of the tensor dot-product mechanism for relation extraction in the proposed architecture. Experimental analyses were also performed to explore the benefits of joint training with named entity recognition in relation extraction in our design. We concluded that joint training with named entity recognition assists relation extraction to improve the span-level representation of entities.

Joint Entity and Relation Extraction Based on Table Labeling Using Convolutional Neural Networks

“…Although the system is simple and effective, it ignores the dependencies among predicted relation labels. As noted in Ma et al (2022), this does not improve the performance with label dependencies incorporated through refined decoding orders.…”

“…Yan et al (2021) applied a partition filter to divide neurons into multiple partitions and generated taskspecific features based on a linear combination of these partitions. Moreover, Ma et al (2022) for RE. This study applies their system as a strong baseline and explores the effect of incorporating local dependencies at the top of BERT.…”

“…This study applies their system as a strong baseline and explores the effect of incorporating local dependencies at the top of BERT. Because the proposed system is an extension of TablERT, we recap this approach (Ma et al, 2022) in the following subsection.…”

“…TablERT (Ma et al, 2022) is a simple and effective method for a combined system applying both NER and RE. As shown on the right side of Figure 1, the method uses the upper triangular part of a table to represent the label spaces of NER and RE.…”

“…The core concept is to extract entities and relations by filling a table with entity labels in the diagonal cells and relation labels in the off-diagonal cells. Based on this concept, Ma et al (2022) proposed TablERT, which is a combined system of NER and RE based on a pretrained BERT. TablERT predicts the diagonal cells sequentially and offdiagonal cells simultaneously.…”