Leveraging transitive relations for crowdsourced joins

Abstract

The development of crowdsourced query processing systems has recently attracted a significant attention in the database community. A variety of crowdsourced queries have been investigated. In this paper, we focus on the crowdsourced join query which aims to utilize humans to find all pairs of matching objects from two collections. As a human-only solution is expensive, we adopt a hybrid human-machine approach which first uses machines to generate a candidate set of matching pairs, and then asks humans to label the pairs in the candidate set as either matching or non-matching. Given the candidate pairs, existing approaches will publish all pairs for verification to a crowdsourcing platform. However, they neglect the fact that the pairs satisfy transitive relations. As an example, if o1 matches with o2, and o2 matches with o3, then we can deduce that o1 matches with o3 without needing to crowdsource (o1, o3). To this end, we study how to leverage transitive relations for crowdsourced joins. We present a hybrid transitive-relations and crowdsourcing labeling framework which aims to crowdsource the minimum number of pairs to label all the candidate pairs. We propose a heuristic labeling order and devise a parallel labeling algorithm to efficiently crowdsource the pairs following the order. We evaluate our approaches in both simulated environment and a real crowdsourcing platform. Experimental results show that our approaches with transitive relations can save much more money and time than existing methods, with a little loss in the result quality.

A framework for protecting worker location privacy in spatial crowdsourcing

“…While crowdsourcing has been present in both the research community (e.g., [25]) and industry (e.g., oDesk and Amazon Mechanical Turk), spatial crowdsourcing only recently received attention (e.g., [21], [15] and [14]). (k) ANW , Yelp-Linear…”

Crowdsourcing algorithms for entity resolution

“…If our database contains n records we may need to ask O(n 2 ) pairwise questions for ER. In general, we can exploit the transitive nature of the equality relation to reduce the number of questions we ask the crowd [16]. For example, consider the Stanford example described above.…”

“…Recently, Wang et al [16] posited that the strategy of asking questions in decreasing order of pairwise probability is the optimal strategy for this problem. However, we show in this paper that the claim is not correct.…”

Interactive join query inference with JIM

“…Crowdsourced joins have been mainly defined in terms of entity resolution, where joining two datasets means finding all pairs of tuples that refer to the same entity [4,5]. Conversely, Jim can handle arbitrary n-ary join predicates, thus targeting a quite different and more intricate goal for the crowd i.e., inferring such join predicates from a set of positive and negative labels.…”

“…As a further difference, the existing systems that allow join processing with the crowd [4,5] do not take into account the labels already given by the user to adjust the order of presenting new tuples for labeling. On the other hand, Jim continuously interleaves the user's feedback and the inference process.…”