CrowdMap: Crowdsourcing Ontology Alignment with Microtasks

Sarasua, Cristina; Simperl, Elena; Noy, Natalya F.

doi:10.1007/978-3-642-35176-1_33

Cited by 124 publications

(107 citation statements)

References 19 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Multi-user scenarios include CrowdMap [19] for ontology matching, ZenCrowd [8] for entity linking, and Zhang et al [23] for database schema matching, which use crowdsourcing on a web platform.…”

Section: Multi-user Feedbackmentioning

confidence: 99%

Pay-As-You-Go Multi-user Feedback Model for Ontology Matching

Cruz

Loprete

Palmonari

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract.Using a pay-as-you-go strategy, we allow for a community of users to validate or invalidate mappings obtained by an automatic ontology matching system using consensus for each mapping. The ultimate objectives are effectiveness-improving the quality of the obtained alignment (set of mappings) measured in terms of F-measure as a function of the number of user interactionsand robustness-making the system as much as possible impervious to user validation errors. Our strategy consists of two major steps: candidate mapping selection, which ranks mappings based on their perceived quality so that users are presented first with those mappings with lowest quality, and feedback propagation, which seeks to validate or invalidate those mappings that are perceived to be similar to the mappings already presented to the users for validation. The purpose of these two strategies is twofold: achieve greater improvements earlier and minimize overall user interaction. There are three important features of our approach: the use of a dynamic ranking mechanism to adapt to the new conditions after each user interaction, the presentation of each mapping for validation more than once-revalidation-because of possible user errors, and the immediate propagation of the user input on a mapping without first achieving consensus for that mapping. We study extensively the effectiveness and robustness of our approach as several of these parameters change, namely the error and revalidation rates, as a function of the number of iterations, to provide conclusive guidelines for the design and implementation of multi-user feedback ontology matching systems.

show abstract

Section: Multi-user Feedbackmentioning

confidence: 99%

Pay-As-You-Go Multi-user Feedback Model for Ontology Matching

Cruz

Loprete

Palmonari

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Waitelonis et al (2011) explored similar principles to curate DBpedia content. In microtask crowdsourcing, Demartini et al (2012) worked on the identification of links between text entities and DBpedia URIs, while Sarasua et al (2012) focused on the post-processing of ontology mappings generated by alignment algorithms. Kontokostas et al (2013) proposed a contest to attract volunteers to assess Linked Data triples, which Acosta et al (2013) combined with the use of microtasks.…”

Section: Validation and Enhancement Of Knowledgementioning

confidence: 99%

Crowdsourcing and the Semantic Web: A Research Manifesto

Sarasua

Simperl

Noy

et al. 2015

HCj

Self Cite

View full text Add to dashboard Cite

Our goal with this research manifesto is to define a roadmap to guide the evolution of the new research field that is emerging at the intersection between crowdsourcing and the Semantic Web. We analyze the confluence of these two disciplines by exploring their relationship. First, we focus on how the application of crowdsourcing techniques can enhance the machine-driven execution of Semantic Web tasks. Second, we look at the ways in which machine-processable semantics can benefit the design and management of crowdsourcing projects. As a result, we are able to describe a list of successful or promising scenarios for both perspectives, identify scientific and technological challenges, and compile a set of recommendations to realize these scenarios effectively. This research manifesto is an outcome of the Dagstuhl Seminar 14282: Crowdsourcing and the Semantic Web.

show abstract

“…The tools used in our experiments and the results are available online, including the outcome of the contest, 12 the gold standard and microtask data (HITs and results). 13 …”

Section: Creation Of Gold Standardmentioning

confidence: 99%

Crowdsourcing Linked Data Quality Assessment

Acosta

Zaveri

Simperl³

et al. 2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper we look into the use of crowdsourcing as a means to handle Linked Data quality problems that are challenging to be solved automatically. We analyzed the most common errors encountered in Linked Data sources and classified them according to the extent to which they are likely to be amenable to a specific form of crowdsourcing. Based on this analysis, we implemented a quality assessment methodology for Linked Data that leverages the wisdom of the crowds in different ways: (i) a contest targeting an expert crowd of researchers and Linked Data enthusiasts; complemented by (ii) paid microtasks published on Amazon Mechanical Turk. We empirically evaluated how this methodology could efficiently spot quality issues in DBpedia. We also investigated how the contributions of the two types of crowds could be optimally integrated into Linked Data curation processes. The results show that the two styles of crowdsourcing are complementary and that crowdsourcing-enabled quality assessment is a promising and affordable way to enhance the quality of Linked Data.

show abstract

CrowdMap: Crowdsourcing Ontology Alignment with Microtasks

Cited by 124 publications

References 19 publications

Pay-As-You-Go Multi-user Feedback Model for Ontology Matching

Pay-As-You-Go Multi-user Feedback Model for Ontology Matching

Crowdsourcing and the Semantic Web: A Research Manifesto

Crowdsourcing Linked Data Quality Assessment

Contact Info

Product

Resources

About