Forum Duplicate Question Detection by Domain Adaptive Semantic Matching

Abstract

Community Question Answering (CQA) forums, such as Stack Overflow, Stack Exchange and Massive Open Online Course (MOOC) forums, spend a lot of manpower and time to manage duplicate questions on the forum. Mismatch of duplicate questions makes users keep asking ''new'' questions, and the continuous accumulation of duplicate questions may interfere with their information searching again, affecting user satisfaction. Neural Networks (NN) models for parsing semantics provide the possibility of end-to-end duplicate question detection. Whereas, due to lack of domain data and expertise, NN models for semantic parsing are rarely directly applied to CQA duplicate question detection. This paper proposes a Semantic Matching Model (SMM) integrated with the multi-task transfer learning framework for multi-domain forum duplicate question detection. By designing the word-to-sentence interaction mechanism based on the word-to-word interaction, SMM can automatically choose to ignore or pay attention to potential similar words according to the semantics at the sentence level. The experiments on the benchmark data set and MOOC forum data set state that SMM outperforms baselines, its interaction mechanism is effective and it has an advantage in cross-domain duplicate question detection.

Enhancing User Experience on Q&A Platforms: Measuring Text Similarity Based on Hybrid CNN-LSTM Model for Efficient Duplicate Question Detection

“…Methodology Dataset Huang et al [50] Pre-Trained word embedding by using Perceptron of Multi-Layer Twitter Agarwal et al [30] Pre-Trained word embedding by combining deep learning and statistical features MSPR K. Dey et al [51] Set of Lexical, Syntactic, Semantic, and Pragmatic Features using resources Word Net, POS Tagger Twitter, MSPR R. Ferreira et al [52] Represent pair of sentences as a combination of similarity measures by using a dependency parser MSPR D. Liang et al [17] Pre-trained Glove vectors on Wikipedia by using Gated Recurrent Network Quora Yushi Homma et al [40] Pre-trained 300 D. Glove vectors on Wikipedia by using Siamese Gated Recurrent Model Quora Wei Bao et al [53] Measuring Semantic Textual Similarity by using Attentive Siamese LSTM SamEval 2014 ZHUOJIA XU et al [11] Semantic Matching Model merged framework of multi-tasking transfer learning for multi-domain forum duplicate detection of questions.…”

“…Deep learning has recently been shown to be useful for several software engineering tasks, such as code clone identification [6] [7], bug report detection [8] [9], and predicting semantically linkable information [10] in questioning and Answering websites. ZHUOJIA et al [11] used a semantics modeling matching technique. Liting Wang et al [12] utilized CNN, RNN, and LSTM on six groups of datasets.…”

Identifying Similar Question Pairs Using Machine Learning Techniques

“…This motivated us to use XGBoost in our work. Z. Xu et al (8) discussed a Semantic Matching Model (SMM) incorporated with the framework of multitasking transfer for detecting duplicate questions. It introduced the word-to-sentence interaction approach depending on which possible similar words are either ignored or paid attention to.…”