Crowd mining

Abstract

Harnessing a crowd of Web users for data collection has recently become a wide-spread phenomenon. A key challenge is that the human knowledge forms an open world and it is thus difficult to know what kind of information we should be looking for. Classic databases have addressed this problem by data mining techniques that identify interesting data patterns. These techniques, however, are not suitable for the crowd. This is mainly due to properties of the human memory, such as the tendency to remember simple trends and summaries rather than exact details.Following these observations, we develop here for the first time the foundations of crowd mining. We first define the formal settings. Based on these, we design a framework of generic components, used for choosing the best questions to ask the crowd and mining significant patterns from the answers. We suggest general implementations for these components, and test the resulting algorithm's performance on benchmarks that we designed for this purpose. Our algorithm consistently outperforms alternative baseline algorithms.

Foundations of Crowd Data Sourcing

“…In contrast, individual data is typically not recorded in a systematic manner, and can only be collected by posing questions to people. Such data can thus be modeled as per-crowd-member knowledge bases, which are not materialized and are accessed by restricted means of interacting with the crowd [1,3]. See a concrete such model in Section 3.1.…”

“…For example, in [7], the error model captures the increase in the error probability of increasingly di cult tasks, as described in Section 4.1. The error probability can be estimated based on preliminary tests with gold-standard data (where the ground truth is known) [6,19] or by comparing the answers of di↵erent users to the same question [3,14]. Error estimations for individual tasks allow estimating and adjusting the overall probability of error, by assigning tasks with higher uncertainty to more crowd members.…”

“…First, the crowd could be engaged in harvesting new or missing data; and second, the crowd can process data that was already collected, by providing their judgments, comparing, cleaning and matching data items. Both types of tasks have been studied in previous literature, e.g., [3,6,7,9,12,13,14,15,17,19,24,26,28]. However, much of the work has been focused on the technical and practical aspects of crowdsourcing.…”

“…In Section 3, we consider a recently developed approach for harvesting the crowd, namely, crowd mining [3], which focuses on identifying statistically significant patterns within the habits and preferences of the crowd. Crowd mining can be used, e.g., for identifying popular combinations of food dishes (useful to the dietician from our previous example).…”

“…-Faithful representation of the trends in the target population -relevant for the harvesting of individual data, where results are typically computed based on the answers of only a small fraction of the population [3]. -Coverage -the number of collected or processed data items [20].…”