FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance

Abstract

Frequently Asked Question (FAQ) retrieval is an important task where the objective is to retrieve an appropriate Question-Answer (QA) pair from a database based on a user's query. We propose a FAQ retrieval system that considers the similarity between a user's query and a question as well as the relevance between the query and an answer. Although a common approach to FAQ retrieval is to construct labeled data for training, it takes annotation costs. Therefore, we use a traditional unsupervised information retrieval system to calculate the similarity between the query and question. On the other hand, the relevance between the query and answer can be learned by using QA pairs in a FAQ database. The recentlyproposed BERT model is used for the relevance calculation. Since the number of QA pairs in FAQ page is not enough to train a model, we cope with this issue by leveraging FAQ sets that are similar to the one in question. We evaluate our approach on two datasets. The first one is localgovFAQ, a dataset we construct in a Japanese administrative municipality domain. The second is StackExchange dataset, which is the public dataset in English. We demonstrate that our proposed method outperforms baseline methods on these datasets.

Balancing via Generation for Multi-Class Text Classification Improvement

“…SUC is a fundamental, multi-class, highly imbalanced textual classification problem. For example, it is widely used for intent (class) detection in goaloriented dialogue systems (Henderson et al, 2014;Bohus and Rudnicky, 2009), and for frequently asked question (FAQ) retrieval (Sakata et al, 2019;Gupta and Carvalho, 2019;Wang et al, 2017).…”

A survey on the techniques, applications, and performance of short text semantic similarity

“…Two sub‐networks receive two inputs respectively, convert them into a vector, and then calculate the distance between the two vectors by some distance metric. The analysis of the BERT model is also added in the paper, BERT is a model that has received widespread attention recently and has been successfully used in answer‐question application 67,101 . The paper compares the LSTM models, BERT model with the traditional non‐DL measures.…”

“…The model not only makes use of a simple context but also uses merging structured semantic information, which can provide rich semantics for language representation. Sakata et al 67 use BERT model to calculate the similarity between the user's query and answer. Their method has robust and high‐performance retrieval.…”

Curriculum Learning Strategies for IR

“…Transfer learning via large pre-trained Transformers [46]-the prominent case being BERT [7]-has lead to remarkable empirical successes on a range of NLP problems. The BERT approach to learn textual representations has also significantly improved the performance of neural models for several IR tasks [33,37,54,55,57], that for a long time struggled to outperform classic IR models [53]. In this work we use the no-CL BERT as a strong baseline for the conversation response ranking task.…”