Do Not Trust the Trolls: Predicting Credibility in Community Question Answering Forums

Nakov, Preslav; Mihaylova, Tsvetomila; Màrquez, Lluı́s; Shiroya, Yashkumar; Koychev, Ivan

doi:10.26615/978-954-452-049-6_072

Cited by 17 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, given the abundance of data for English, we would like to try cross-language approaches (Chew et al 2011; Da San Martino et al 2017; Joty et al 2017), combining English and Arabic data for both training and testing. Finally, given the recent proliferation of false information online, it might be worth trying to verify the factuality of the answers given in a cQA forum (Karadzhov et al 2017; Nakov et al 2017b; Mihaylova et al 2018), as well as detecting potential forum trolls (Mihaylov et al 2015a, 2015b; Mihaylov and Nakov 2016a; Mihaylov et al 2018).…”

Section: Resultsmentioning

confidence: 99%

Arabic community question answering

Nakov¹,

Màrquez²,

Moschitti³

et al. 2018

Nat. Lang. Eng.

Self Cite

View full text Add to dashboard Cite

We analyze resources and models for Arabic community Question Answering (cQA). In particular, we focus on CQA-MD, our cQA corpus for Arabic in the domain of medical forums. We describe the corpus and the main challenges it poses due to its mix of informal and formal language, and of different Arabic dialects, as well as due to its medical nature. We further present a shared task on cQA at SemEval, the International Workshop on Semantic Evaluation, based on this corpus. We discuss the features and the machine learning approaches used by the teams who participated in the task, with focus on the models that exploit syntactic information using convolutional tree kernels and neural word embeddings. We further analyze and extend the outcome of the SemEval challenge by training a meta-classifier combining the output of several systems. This allows us to compare different features and different learning algorithms in an indirect way. Finally, we analyze the most frequent errors common to all approaches, categorizing them into prototypical cases, and zooming into the way syntactic information in tree kernel approaches can help solve some of the most difficult cases. We believe that our analysis and the lessons learned from the process of corpus creation as well as from the shared task analysis will be helpful for future research on Arabic cQA.

show abstract

Section: Resultsmentioning

confidence: 99%

Arabic community question answering

Nakov¹,

Màrquez²,

Moschitti³

et al. 2018

Nat. Lang. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, there has been a lot of research interest in studying disinformation and bias in the news and in social media. This includes challenging the truthiness of claims [6,52,81], of news [17,33,36,37,42,60,61], of news sources [8], of social media users [3,26,49,50,51,59], and of social media [18,19,64,82], as well as studying credibility, influence, and bias [7,8,20,45,49,51]. The interested reader can also check several recent surveys that offer a general overview on "fake news" [46], or focus on topics such as the process of proliferation of true and false news online [75], on fact-checking [71], on data mining [67], or on truth discovery in general [47].…”

Section: Related Workmentioning

confidence: 99%

Can We Spot the "Fake News" Before It Was Even Written?

Nakov¹

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Given the recent proliferation of disinformation online, there has been also growing research interest in automatically debunking rumors, false claims, and "fake news." A number of fact-checking initiatives have been launched so far, both manual and automatic, but the whole enterprise remains in a state of crisis: by the time a claim is finally factchecked, it could have reached millions of users, and the harm caused could hardly be undone. An arguably more promising direction is to focus on fact-checking entire news outlets, which can be done in advance. Then, we could fact-check the news before it was even written: by checking how trustworthy the outlets that published it is. We describe how we do this in the Tanbih news aggregator, which makes readers aware of what they are reading. In particular, we develop media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting, and stance with respect to various claims and topics.

show abstract

“…Their proposed model captures information from the answer content (what is said and how), from the author profile (who says it), from the rest of the community forum (where it is said), and from external authoritative sources of information (external support). (Nakov et al, 2017) proposed models for credibility assessment in community question answering forums. However, credibility is different from veracity as it is a subjective perception about whether a statement is credible, rather than verifying whether it is true/false as a matter of fact.…”

Section: Related Workmentioning

confidence: 99%

ColumbiaNLP at SemEval-2019 Task 8: The Answer is Language Model Fine-tuning

Chakrabarty

Muresan

2019

Proceedings of the 13th International Workshop on Semantic Evaluation

View full text Add to dashboard Cite

Community Question Answering forums are very popular nowadays, as they represent effective means for communities to share information around particular topics. But the information shared on these forums is often not correct or misleading. This paper presents the ColumbiaNLP submission for the SemEval-2019 Task 8: Fact-Checking in Community Question Answering Forums. We show how fine-tuning a language model on a large unannotated corpus of old threads from Qatar Living forum helps us to classify question types (factual, opinion, socializing) and to judge the factuality of answers on the shared task labeled data from the same forum. Our system finished 4th and 2nd on Subtask A (question type classification) and B (answer factuality prediction), respectively, based on the official metric of accuracy.

show abstract

Do Not Trust the Trolls: Predicting Credibility in Community Question Answering Forums

Cited by 17 publications

References 23 publications

Arabic community question answering

Arabic community question answering

Can We Spot the "Fake News" Before It Was Even Written?

ColumbiaNLP at SemEval-2019 Task 8: The Answer is Language Model Fine-tuning

Contact Info

Product

Resources

About