Answer Classification via Machine Learning in Community Question Answering

Abstract

As a new type of knowledge sharing platform, the community question answer website realizes the acquisition and sharing of knowledge, and is loved and sought after by the majority of users. But for multi-answer questions, answer quality assessment becomes a challenge. The answer selection in CQA (Community Question Answer) was proposed as a challenge task in the SemEval competition, which gave a data set and proposed two subtasks. Task-A is to give a question (including short title and extended description) and its answers, and divide each answer into absolutely relevant (good), potentially relevant (potential) and bad or irrelevant (bad, dialog, non-English, other). Task-B is to give a YES/NO type question (including short title and extended description) and some answers. Based on the answer of the absolute correlation type (good), judge whether the answer to the whole question should be yes, no or uncertain. This paper first preprocesses this data set, and then uses natural language processing technology to perform word segmentation, part-of-speech tagging and named entity recognition on the data set, and then perform feature extraction on the preprocessed data set. Finally, SVM and random forest are used to classify on the basis of feature extraction, and the classification results are analyzed and compared. The experiments in this paper show that SVM and random forest methods have good results on the data set, and exceed the multi-classifier ensemble learning method and hierarchical classification method proposed by the predecessors.