Question Similarity Calculation for FAQ Answering

Song, Wanpeng; Min, Feng; Gu, Naijie; Wenyin, Liu

doi:10.1109/skg.2007.32

Cited by 20 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we focus on the FAQ answering task. Previous work on answering a question given FAQ pages (FAQ-Finder [8], Auto-FAQ [28], [2,11,13,21,23]) was based on traditional feature engineering for surfacing statistical/semantic similarities between query and questions. Most of these works considered similarity between query and questions, very few considered query-answer similarity.…”

Section: Related Workmentioning

confidence: 99%

Optimized Transformer Models for FAQ Answering

Damani

Narahari

Chatterjee

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Informational chatbots provide a highly effective medium for improving operational efficiency in answering customer queries for any enterprise. Chatbots are also preferred by users/customers since unlike other alternatives like calling customer care or browsing over FAQ pages, chatbots provide instant responses, are easy to use, are less invasive and are always available. In this paper, we discuss the problem of FAQ answering which is central to designing a retrieval-based informational chatbot. Given a set of FAQ pages s for an enterprise, and a user query, we need to find the best matching question-answer pairs from s. Building such a semantic ranking system that works well across domains for large QA databases with low runtime and model size is challenging. Previous work based on feature engineering or recurrent neural models either provides low accuracy or incurs high runtime costs. We experiment with multiple transformer based deep learning models, and also propose a novel MT-DNN (Multi-task Deep Neural Network)-based architecture, which we call Masked MT-DNN (or MMT-DNN). MMT-DNN significantly outperforms other state-of-the-art transformer models for the FAQ answering task. Further, we propose an improved knowledge distillation component to achieve ∼2.4x reduction in model-size and ∼7x reduction in runtime while maintaining similar accuracy. On a small benchmark dataset from SemEval 2017 CQA Task 3, we show that our approach provides an NDCG@1 of 83.1. On another large dataset of ∼281K instances corresponding to ∼30K queries from diverse domains, our distilled 174 MB model provides an NDCG@1 of 75.08 with a CPU runtime of mere 31 ms establishing a new state-of-the-art for FAQ answering.

show abstract

Section: Related Workmentioning

confidence: 99%

Optimized Transformer Models for FAQ Answering

Damani

Narahari

Chatterjee

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Attribute Recognition: Once entities present in the user question are identified, questions stored in the attribute-tuples of the identified entity node are extracted. A similarity measure of the given question with each of the extracted questions [12] is computed as follows:…”

Section: Answer Retrieval (Anr) Modulementioning

confidence: 99%

EDUQA: Educational Domain Question Answering System Using Conceptual Network Mapping

Agarwal¹,

Sachdeva²,

Yadav³

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Most of the existing question answering models can be largely compiled into two categories: i) open domain question answering models that answer generic questions and use largescale knowledge base along with the targeted web-corpus retrieval and ii) closed domain question answering models that address focused questioning area and use complex deep learning models. Both the above models derive answers through textual comprehension methods. Due to their inability to capture the pedagogical meaning of textual content, these models are not appropriately suited to the educational field for pedagogy. In this paper, we propose an on-the-fly conceptual network model that incorporates educational semantics. The proposed model preserves correlations between conceptual entities by applying intelligent indexing algorithms on the concept network so as to improve answer generation. This model can be utilized for building interactive conversational agents for aiding classroom learning.

show abstract

“…One branch of work calculated the semantic similarity of questions based on Natural Language Processing (NLP). For example, Song et al [34] first identified keywords in questions, and then calculated semantic similarity between questions based on the keywords. They also extended the semantic similarity to statistic similarity, which was calculated using the cosine similarity between two question vectors.…”

Section: Question Similarity Calculationmentioning

confidence: 99%

“…In order to conduct a state-of-the-art machine learning prediction experiment, we survey existing studies for the features used in student performance prediction [22,24], mouse trajectory feature definition [33,42], and possible methods in question similarity calculation [1,4,34,36,37]. We summarize this knowledge with our dataset and task, then we propose our prediction framework in Figure 2.…”

Section: Overall Prediction Frameworkmentioning

confidence: 99%

Why you should(n't) build your own game engine

Rivela¹

2019

ACM SIGGRAPH 2019 Talks

View full text Add to dashboard Cite

Modeling student learning and further predicting the performance is a well-established task in online learning and is crucial to personalized education by recommending different learning resources to different students based on their needs. Interactive online question pools (e.g., educational game platforms), an important component of online education, have become increasingly popular in recent years. However, most existing work on student performance prediction targets at online learning platforms with a well-structured curriculum, predefined question order and accurate knowledge tags provided by domain experts. It remains unclear how to conduct student performance prediction in interactive online question pools without such well-organized question orders or knowledge tags by experts. In this paper, we propose a novel approach to boost student performance prediction in interactive online question pools by further considering student interaction features and the similarity between questions. Specifically, we introduce new features (e.g., think time, first attempt, and first drag-and-drop) based on student mouse movement trajectories to delineate students' problem-solving details. In addition, heterogeneous information network is applied to integrating students' historical problem-solving information on similar questions, enhancing student performance predictions on a new question. We evaluate the proposed approach on the dataset from a real-world interactive question pool using four typical machine learning models. The result shows that our approach can achieve a much higher accuracy for student performance prediction in interactive online question pools than the traditional way of only using the statistical features (e.g., students' historical question scores) in various models. We further discuss the performance consistency of our approach across different prediction models and question classes, as well as the importance of the proposed interaction features in detail.

show abstract

Question Similarity Calculation for FAQ Answering

Cited by 20 publications

References 8 publications

Optimized Transformer Models for FAQ Answering

Optimized Transformer Models for FAQ Answering

EDUQA: Educational Domain Question Answering System Using Conceptual Network Mapping

Why you should(n't) build your own game engine

Contact Info

Product

Resources

About