Argument discovery via crowdsourcing

Nguyen, Quoc Viet Hung; Duong, Chi Thang; Nguyên, Thành Tâm; Weidlich, Matthias; Aberer, Karl; Yin, Hongzhi; Zhou, Xiaofang

doi:10.1007/s00778-017-0462-9

Cited by 44 publications

(15 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, crowdsourcing is used to obtain a great number of labels quickly and inexpensively, as is the case of Hovy et al (). In relation to the problems solved by these applications, we find that most of them are related to open challenges in natural language processing (for an introduction to some of them, we refer the reader to Hirschberg and Manning ()): sentiment analysis of online media (Brew, Greene, & Cunningham, ; Salter‐Townshend & Murphy, ); joke's humor classification (Costa et al, ); temporal relation classification (Ng & Kan, ); word sense (Passonneau et al, ); marketing messaging classification on Twitter (Machedon et al, ); part‐of‐speech tagging (Hovy et al, ); identifying fake Amazon reviews (Fornaciari & Poesio, ); sequence labeling (Nguyen, Wallace, et al, ; Rodrigues et al, ); estimation of discourse segmentation (Huang et al, ); emotion estimation from narratives (Duan et al, ); crowdsourced translation (Yan et al, ); entity disambiguation (Li, Yang, et al, ; Nguyen, Duong, et al, ; Zhou et al, ); topic models (Liu et al, ; Rodrigues et al, ); personal assistants (Shin & Paek, ; Yang et al, ); corpus creation for Arabic dialects (Alshutayri & Atwell, ); and context sensitive tasks (Fang et al, ).…”

Section: Publication Areasmentioning

confidence: 99%

Machine learning from crowds: A systematic review of its applications

Rodrigo

Aledo

Gámez

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Crowdsourcing opens the door to solving a wide variety of problems that previously were unfeasible in the field of machine learning, allowing us to obtain relatively low cost labeled data in a small amount of time. However, due to the uncertain quality of labelers, the data to deal with are sometimes unreliable, forcing practitioners to collect information redundantly, which poses new challenges in the field. Despite these difficulties, many applications of machine learning using crowdsourced data have recently been published that achieved state of the art results in relevant problems. We have analyzed these applications following a systematic methodology, classifying them into different fields of study, highlighting several of their characteristics and showing the recent interest in the use of crowdsourcing for machine learning. We also identify several exciting research lines based on the problems that remain unsolved to foster future research in this field.

show abstract

Section: Publication Areasmentioning

confidence: 99%

Machine learning from crowds: A systematic review of its applications

Rodrigo

Aledo

Gámez

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…For instance, some statistical features such as the number of retweets or replies are considered [1,20,11]. Similarly, some user-level features are also used such as the credibility or readability of the users [1,9,14]. A different approach is to examine network level features to detect rumors.…”

Section: Related Workmentioning

confidence: 99%

Provenance-Based Rumor Detection

Duong

Nguyen

Wang

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

With the advance of social media networks, people are sharing contents in an unprecedented scale. This makes social networks such as microblogs an ideal place for spreading rumors. Although different types of information are available in a post on social media, traditional approaches in rumor detection leverage only the text of the post, which limits their accuracy in detection. In this paper, we propose a provenanceaware approach based on recurrent neural network to combine the provenance information and the text of the post itself to improve the accuracy of rumor detection. Experimental results on a real-world dataset show that our technique is able to outperform state-of-the-art approaches in rumor detection.

show abstract

“…In particular, Sarma et al [62] decompose crowdsourcing tasks to decrease the difficulty of questions, thereby improving the chance for workers to provide correct answers. Although such decomposition is useful for large-scale data, it might render the answer matrix sparser, which requires further customization [42,47,48]. Joglekar et al [25], on the other hand, measure the confidence interval of worker error rates, making the classification of worker types more fine-grained and thus the filtering of faulty workers more accurate.…”

Section: Crowd-expert Collaborationmentioning

confidence: 99%

Answer validation for generic crowdsourcing tasks with minimal efforts

et al. 2017

Self Cite

View full text Add to dashboard Cite

Crowdsourcing has been established as an essential means to scale human computation in diverse Web applications, reaching from data integration to information retrieval. Yet, crowd workers have wide-ranging levels of expertise. Large worker populations are heterogeneous and comprise a significant amount of faulty workers. As a consequence, quality insurance for crowd answers is commonly seen as the Achilles heel of crowdsourcing. Although various techniques for quality control have been proposed in recent years, a post-processing phase in which crowd answers are validated is still required. Such validation, however, is typically conducted by experts, whose availability is limited and whose work incurs comparatively high costs. This work aims at guiding an expert in the validation of crowd answers. We present a probabilistic model that helps to identify the most beneficial validation questions in terms of both improvement in result correctness and detection of faulty workers. By seeking expert feedback on the most problematic cases, we are able to obtain a set of high-quality answers, even if the expert does not validate the complete answer set. Our approach is applicable for a broad range of crowdsourcing tasks, including classification and counting. Our comprehensive evaluation using both real-world and synthetic datasets demonstrates that our techniques save up to 60% of expert efforts compared to baseline methods when striving for perfect result correctness. In absolute terms, for most cases, we B Nguyen Quoc Viet Hung

show abstract

Argument discovery via crowdsourcing

Cited by 44 publications

References 58 publications

Machine learning from crowds: A systematic review of its applications

Machine learning from crowds: A systematic review of its applications

Provenance-Based Rumor Detection

Answer validation for generic crowdsourcing tasks with minimal efforts

Contact Info

Product

Resources

About