EICA Team at SemEval-2017 Task 3: Semantic and Metadata-based
            Features for Community Question Answering

Xie, Yi; Wang, Maoquan; Ma, Jing; Jiang, Jian; Zhao, Lu

doi:10.18653/v1/s17-2047

Cited by 8 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 summarizes the results of different methods on SemEval 2017 dataset. For our methods, the term "single" denotes that we only consider word-to-word matches as in Equation 14, while "multi" means that we consider Method MAP MRR Baseline (IR) 9.18 10.11 Baseline (random) 5.77 7.69 (Tian et al 2017) 10.64 11.09 (Zhang et al 2017a) 13.23 14.27 (Xie et al 2017) 13.48 16.04 (Filice, Da Martino, and Moschitti 2017) 14.35 16.07 (Koreeda et al 2017) 14.71 16.48 (Nandi et al 2017) 15 (Filice, Da Martino, and Moschitti 2017;Xie et al 2017;Nandi et al 2017) and neural networks (Tian et al 2017;Zhang et al 2017a;Koreeda et al 2017). For singlescale model, the MAP is increased from 14.67 to 17.25, while for multi-scale model, the number is increased from 14.80 to 17.91.…”

Section: Training Hyper-parametersmentioning

confidence: 99%

Adversarial Training for Community Question Answer Selection Based on Multi-Scale Matching

Xiao¹,

Khabsa²,

Wang³

et al. 2019

AAAI

View full text Add to dashboard Cite

Community-based question answering (CQA) websites represent an important source of information. As a result, the problem of matching the most valuable answers to their corresponding questions has become an increasingly popular research topic. We frame this task as a binary (relevant/irrelevant) classification problem, and present an adversarial training framework to alleviate label imbalance issue.We employ a generative model to iteratively sample a subset of challenging negative samples to fool our classification model. Both models are alternatively optimized using REIN-FORCE algorithm. The proposed method is completely different from previous ones, where negative samples in training set are directly used or uniformly down-sampled. Further, we propose using Multi-scale Matching which explicitly inspects the correlation between words and ngrams of different levels of granularity. We evaluate the proposed method on SemEval 2016 and SemEval 2017 datasets and achieves state-of-the-art or similar performance.

show abstract

Section: Training Hyper-parametersmentioning

confidence: 99%

Adversarial Training for Community Question Answer Selection Based on Multi-Scale Matching

Xiao¹,

Khabsa²,

Wang³

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…For classification tasks like question similarity across community QA forums, machine learning classification algorithms like Support Vector Machines (SVMs) have been used (Šaina et al, 2017;Nandi et al, 2017;Xie et al, 2017;Mihaylova et al, 2016;Wang and Poupart, 2016;. Recently, advances in deep neural network architectures have also led to the use of Convolutional Neural Networks (CNNs) (Šaina et al, 2017;Mohtarami et al, 2016) which perform reasonably well for selection of the correct answer amongst cQA formus.…”

Section: Related Workmentioning

confidence: 99%

“…Other works in the space include use of Random Forests (Wang and Poupart, 2016); topic models to match the questions at both the term level and topic level (Zhang et al, 2014). There have also been works on translation based retrieval models (Jeon et al, 2005;Zhou et al, 2011); Xg-Boost (Feng et al, 2017) and Feedforward Neural Networks (NN) (Wang and Poupart, 2016 (Wang and Poupart, 2016;Mohtarami et al, 2016;Nandi et al, 2017); and Metadata-based features (Mohtarami et al, 2016;Mihaylova et al, 2016;Xie et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

Fermi at SemEval-2019 Task 8: An elementary but effective approach to Question Discernment in Community QA Forums

Syed

Indurthi

Shrivastava³

et al. 2019

Proceedings of the 13th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

Online Community Question Answering Forums (cQA) have gained massive popularity within recent years. The rise in users for such forums have led to the increase in the need for automated evaluation for question comprehension and fact evaluation of the answers provided by various participants in the forum. Our team, Fermi, participated in sub-task A of Task 8 at SemEval 2019-which tackles the first problem in the pipeline of factual evaluation in cQA forums, i.e., deciding whether a posed question asks for a factual information, an opinion/advice or is just socializing. This information is highly useful in segregating factual questions from non-factual ones which highly helps in organizing the questions into useful categories and trims down the problem space for the next task in the pipeline for fact evaluation among the available answers. Our system uses the embeddings obtained from Universal Sentence Encoder combined with XGBoost for the classification subtask A. We also evaluate other combinations of embeddings and off-the-shelf machine learning algorithms to demonstrate the efficacy of the various representations and their combinations. Our results across the evaluation test set gave an accuracy of 84% and received the first position in the final standings judged by the organizers.

show abstract

“…Some of the earlier works on cQA include the use of classification models -Support Vector Machines(SVMs) (Šaina et al, 2017;Nandi et al, 2017;Xie et al, 2017;Mihaylova et al, 2016;Wang and Poupart, 2016; for Similarity tasks; Convolutional Neural Networks (CNNs) for Similarity tasks (Šaina et al, 2017;Mohtarami et al, 2016) and for answer selection (Zhang et al, 2017); Long-Short Term Memory (LSTM) model for answer selection (Zhang et al, 2017;Feng et al, 2017;Mohtarami et al, 2016); Random Forests (Wang and Poupart, 2016); LDA topic language model to match the questions at both the term level and topic level (Zhang et al, 2014); translation based retrieval models (Jeon et al, 2005;Zhou et al, 2011); XgBoost (Feng et al, 2017) and Feedforward Neural Network (NN) (Wang and Poupart, 2016 (Mikolov et al, 2013), GloVe 6 (Pennington et al, 2014) etc.) (Wang and Poupart, 2016;Mohtarami et al, 2016;Nandi et al, 2017); Metadata-based features (like user information, answer length, question length, question marks in answer, question to comment length etc.)…”

Section: Related Workmentioning

confidence: 99%

CodeForTheChange at SemEval-2019 Task 8: Skip-Thoughts for Fact Checking in Community Question Answering

Avvaru

Pandey²

2019

Proceedings of the 13th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

Community Question Answering (cQA) is one of the popular Natural Language Processing (NLP) problems being targeted by researchers across the globe. Couple of the unanswered questions in the domain of cQA are 'can we label the questions/answers as factual or not?' and 'Is the given answer by the user to a particular factual question is correct and if it is correct, can we measure the correctness and factuality of the given answer?'. We have participated in SemEval-2019 Task 8 which deals with these questions. In this paper, we present the features used, approaches followed for feature engineering, models experimented with and finally the results. Our primary submission with accuracy (official metric for Se-mEval Task 8) of 0.65 in Subtask B (Answer Classification) and 0.63 in Subtask A (Question Classification) stood at 6 th and 16 th places respectively.

show abstract

EICA Team at SemEval-2017 Task 3: Semantic and Metadata-based Features for Community Question Answering

Cited by 8 publications

References 19 publications

Adversarial Training for Community Question Answer Selection Based on Multi-Scale Matching

Adversarial Training for Community Question Answer Selection Based on Multi-Scale Matching

Fermi at SemEval-2019 Task 8: An elementary but effective approach to Question Discernment in Community QA Forums

CodeForTheChange at SemEval-2019 Task 8: Skip-Thoughts for Fact Checking in Community Question Answering

Contact Info

Product

Resources

About