Metric Learning for Dynamic Text Classification

Wohlwend, Jeremy; Elenberg, Ethan R.; Altschul, Samuel; Henry, Shawn; Leí, Tao

doi:10.18653/v1/d19-6116

Cited by 8 publications

(3 citation statements)

References 27 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We choose cosine-similarity function, which is equivalent to inner product for normalized vectors. Training Stage -The goal is to learn a latent space following metric learning [27], where matching query and FAQ pairs shall have smaller distance compared to the non-matching pairs. Let D train = {(x i , y + i , y − i,1 , .…”

Section: Proposed Approachmentioning

confidence: 99%

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

Banerjee¹,

Jain²,

Kulkarni³

2023

Preprint

View full text Add to dashboard Cite

In the domain of question-answering in NLP, the retrieval of Frequently Asked Questions (FAQ) is an important sub-area which is well researched and has been worked upon for many languages. Here, in response to a user query, a retrieval system typically returns the relevant FAQs from a knowledge-base. The efficacy of such a system depends on its ability to establish semantic match between the query and the FAQs in real-time. The task becomes challenging due to the inherent lexical gap between queries and FAQs, lack of sufficient context in FAQ titles, scarcity of labeled data and high retrieval latency. In this work, we propose a bi-encoder-based query-FAQ matching model that leverages multiple combinations of FAQ fields (like, question, answer, and category) both during model training and inference. Our proposed Multi-Field Bi-Encoder (MFBE) model benefits from the additional context resulting from multiple FAQ fields and performs well even with minimal labeled data. We empirically support this claim through experiments on proprietary as well as open-source public datasets in both unsupervised and supervised settings. Our model achieves around 27% and 20% better top-1 accuracy for the FAQ retrieval task on internal and open datasets, respectively over the best performing baseline.

show abstract

Section: Proposed Approachmentioning

confidence: 99%

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

Banerjee¹,

Jain²,

Kulkarni³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Conversely, the distance between the vectors of two samples belonging to different classes should be larger. In recent years, metric learning has been shown to be effective in a number of computer vision tasks, such as image retrieval (Zhong et al, 2021 ), object recognition (Sohn, 2016 ), and face recognition (Cao et al, 2013 ), and also for natural language processing tasks, such as text classification (Wohlwend et al, 2019 ) and entity linking (Liu et al, 2020 ).…”

Section: Background and Related Workmentioning

confidence: 99%

A metric learning-based method for biomedical entity linking

Le,

Nguyen

2023

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

Biomedical entity linking task is the task of mapping mention(s) that occur in a particular textual context to a unique concept or entity in a knowledge base, e.g., the Unified Medical Language System (UMLS). One of the most challenging aspects of the entity linking task is the ambiguity of mentions, i.e., (1) mentions whose surface forms are very similar, but which map to different entities in different contexts, and (2) entities that can be expressed using diverse types of mentions. Recent studies have used BERT-based encoders to encode mentions and entities into distinguishable representations such that their similarity can be measured using distance metrics. However, most real-world biomedical datasets suffer from severe imbalance, i.e., some classes have many instances while others appear only once or are completely absent from the training data. A common way to address this issue is to down-sample the dataset, i.e., to reduce the number instances of the majority classes to make the dataset more balanced. In the context of entity linking, down-sampling reduces the ability of the model to comprehensively learn the representations of mentions in different contexts, which is very important. To tackle this issue, we propose a metric-based learning method that treats a given entity and its mentions as a whole, regardless of the number of mentions in the training set. Specifically, our method uses a triplet loss-based function in conjunction with a clustering technique to learn the representation of mentions and entities. Through evaluations on two challenging biomedical datasets, i.e., MedMentions and BC5CDR, we show that our proposed method is able to address the issue of imbalanced data and to perform competitively with other state-of-the-art models. Moreover, our method significantly reduces computational cost in both training and inference steps. Our source code is publicly available here.

show abstract

“…These methods typically learn generally useful text representations from a large corpus of unlabeled text and use them for a specific target task with limited supervision (Howard and Ruder, 2018;Devlin et al, 2019;Lan et al, 2020;Peters et al, 2018). Metric learning (Wohlwend et al, 2019) is related to our work, but they focus on few-shot learning and we work on improving unsupervised text classifiers. Related contemporaneous work has proposed methods to generate more relevant label names from a given set (Meng et al, 2020;Schick et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Label Refinement Improves Dataless Text Classification

Chu¹,

Stratos²,

Gimpel³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Dataless text classification is capable of classifying documents into previously unseen labels by assigning a score to any document paired with a label description. While promising, it crucially relies on accurate descriptions of the label set for each downstream task. This reliance causes dataless classifiers to be highly sensitive to the choice of label descriptions and hinders the broader application of dataless classification in practice. In this paper, we ask the following question: how can we improve dataless text classification using the inputs of the downstream task dataset? Our primary solution is a clustering based approach. Given a dataless classifier, our approach refines its set of predictions using k-means clustering. We demonstrate the broad applicability of our approach by improving the performance of two widely used classifier architectures, one that encodes text-category pairs with two independent encoders and one with a single joint encoder. Experiments show that our approach consistently improves dataless classification across different datasets and makes the classifier more robust to the choice of label descriptions. 1

show abstract

Metric Learning for Dynamic Text Classification

Cited by 8 publications

References 27 publications

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

A metric learning-based method for biomedical entity linking

Unsupervised Label Refinement Improves Dataless Text Classification

Contact Info

Product

Resources

About