Function-words Adaptively Enhanced Attention Networks for Few-Shot Inverse Relation Classification

Abstract

Existing unsupervised domain adaptation (UDA) studies focus on transferring knowledge in an offline manner. However, many tasks involve online requirements, especially in real-time systems. In this paper, we discuss Online UDA (OUDA) which assumes that the target samples are arriving sequentially as a small batch. OUDA tasks are challenging for prior UDA methods since online training suffers from catastrophic forgetting which leads to poor generalization. Intuitively, a good memory is a crucial factor in the success of OUDA. We formalize this intuition theoretically with a generalization bound where the OUDA target error can be bounded by the source error, the domain discrepancy distance, and a novel metric on forgetting in continuous online learning. Our theory illustrates the tradeoffs inherent in learning and remembering representations for OUDA. To minimize the proposed forgetting metric, we propose a novel source feature distillation (SFD) method which utilizes the source-only model as a teacher to guide the online training. In the experiment, we modify three UDA algorithms, i.e., DANN, CDAN, and MCC, and evaluate their performance on OUDA tasks with real-world datasets. By applying SFD, the performance of all baselines is significantly improved.

BMIPN: A Biased Multi-Granularity Interaction Prototype Network for Few-Shot Relation Extraction

“…We conducted our experiments on two large-scale benchmark datasets. The experimental results show that our model outperforms all baseline models, especially the recent state-of-the-art model [3]. Further studies demonstrate our ability to mitigate the similar relation problem effectively.…”

“…However, these models are devoted to learning each class's prototype representations individually, without considering the higher-order interactions between the different classes. Recently, FAEA [3] explores constructing an instance-level graph in the support set. However, they mainly focus on intra-class function word detection, without considering the impact of high-order interactions, limiting the task's performance.…”

“…We follow previous works [3,6,8,9] with N-way-K-shot setting, which is a typical few-shot setting. To be specific, N-way-K-shot mode contains a support set S and a query set Q.…”

“…Evaluation. We follow the evaluation metrics in previous work [3,4,5,6,8,12,21,23,24,27,28]. Firstly, the models' performance is measured by the accuracy (%) of the query set in N -way-K-shot tasks.…”

The danger of dichloromethane in the synthesis and application of imidazole-based dyes

“…For few-shot inverse relation classification, FAEA [265] employs a hybrid attention model to attend class-related words based on meta-learning. It leverages function-words enhanced attention to effectively compute representations for both support and query instances, facilitating message passing and similarity calculation between queries and prototypes.…”