Massive multiplayer human computation for fun, money, and survival

Abstract

Crowdsourcing is an effective tool to solve hard tasks. By bringing 100,000s of people to work on simple tasks that only humans can do, we can go far beyond traditional models of data analysis and machine learning. As technologies and processes mature, crowdsourcing is becoming mainstream. It powers many leading Internet companies and a wide variety of novel projects: from content moderation and business listing verification to real-time SMS translation for disaster response. However, quality assurance can be a major challenge. In this paper CrowdFlower presents various crowdsourcing applications, from business to ethics, to money and survival, all of which showcase the power of labor-on-demand, otherwise known as the human cloud.

Analysis of named entity recognition and linking for tweets

“…The annotation task was performed using Crowdflower [40]. Our interface design was to show each volunteer the text of the tweet, any URL links contained therein, and a set of candidate targets from DBpedia.The volunteers were encouraged to click on the URL links from the tweet, to gain addition context and thus ensure that the correct DBpedia URI is chosen by them.…”