ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning

Abstract

While relation extraction is an essential task in knowledge acquisition and representation, and new-generated relations are common in the real world, less effort is made to predict unseen relations that cannot be observed at the training stage. In this paper, we formulate the zeroshot relation extraction problem by incorporating the text description of seen and unseen relations. We propose a novel multi-task learning model, zero-shot BERT (ZS-BERT), to directly predict unseen relations without handcrafted attribute labeling and multiple pairwise classifications. Given training instances consisting of input sentences and the descriptions of their relations, ZS-BERT learns two functions that project sentences and relation descriptions into an embedding space by jointly minimizing the distances between them and classifying seen relations. By generating the embeddings of unseen relations and newcoming sentences based on such two functions, we use nearest neighbor search to obtain the prediction of unseen relations. Experiments conducted on two well-known datasets exhibit that ZS-BERT can outperform existing methods by at least 13.54% improvement on F1 score.

Prompt-based Zero-shot Relation Classification with Semantic Knowledge Augmentation

“…We conduct several experiments with ablation studies on three public datasets: FewRel (Han et al, 2018), Wiki-ZSL (Sorokin and Gurevych, 2017;Chen and Li, 2021) and NYT (Riedel et al, 2010) to show that our proposed model outperforms other existing state-of-the-art models, and our proposed model is more robust compared with the other models in zero-shot learning tasks.…”

“…In particular, by listing questions that define the relation's slot values (Levy et al, 2017;Cetoli, 2020). To avoid relying on question-answering models, some studies formulate relation extraction as a text entailment task and utilize the accessibility of the relation descriptions (Obamuyide and Vlachos, 2018;Qin et al, 2020;Gong and Eldardiry, 2021;Chen and Li, 2021). However, these models only utilize class names semantic information, losing the connections between relations.…”

“…We follow the experiment settings as (Chen and Li, 2021) to enable zero-shot relation classification tasks. We randomly select m unseen relations and remove all the instances related to these m relations in the training set to ensure that these m relations have not appeared in training data.…”

“…We compare our proposed model to several stateof-the-arts models in zero-shot learning tasks. For clean FewRel and Wiki-ZSL datasets, we compare our model with CNN (Zeng et al, 2014), Bi-LSTM (Zhang et al, 2015), Attentional Bi-LSTM (Zhou et al, 2016), R-BERT (Wu and He, 2019), ESIM (Chen et al, 2017), CIM (Rocktäschel et al, 2016) and ZS-BERT (Chen and Li, 2021). The seven baselines above are reported by (Chen and Li, 2021).…”

“…Third, some state-of-the-art models leverage side (auxiliary) information to tackle zero-shot tasks. They focus on class names/descriptions semantic information, losing the connection or relationships between seen relations and unseen relations (Gong and Eldardiry, 2021;Chen and Li, 2021).…”