Addressing the resource bottleneck to create large-scale annotated texts

Chamberlain, Jon; Poesio, Massimoand Kruschwitz

doi:10.3115/1626481.1626511

Cited by 10 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Following the idea of Phrase Detectives (Chamberlain et al, 2008), in the GMB a game with a purpose (GWAP) was introduced to annotate parts of speech, antecedents of pronouns, noun compound relations (Bos and Nissim, 2015), and word senses (Venhuizen et al, 2013). The quality of annotations harvested from gamification was generally high, but the amount of annotations relatively low-it would literally take years to annotate the entire GMB corpus.…”

Section: Be Careful With the Crowdmentioning

confidence: 99%

Thirty Musts for Meaning Banking

Bos,

Abzianidze

2020

Preprint

View full text Add to dashboard Cite

Meaning banking-creating a semantically annotated corpus for the purpose of semantic parsing or generation-is a challenging task. It is quite simple to come up with a complex meaning representation, but it is hard to design a simple meaning representation that captures many nuances of meaning. This paper lists some lessons learned in nearly ten years of meaning annotation during the development of the Groningen Meaning Bank and the Parallel Meaning Bank . The paper's format is rather unconventional: there is no explicit related work, no methodology section, no results, and no discussion (and the current snippet is not an abstract but actually an introductory preface). Instead, its structure is inspired by work of Traum (2000) and Bender (2013). The list starts with a brief overview of the existing meaning banks (Section 1) and the rest of the items are roughly divided into three groups: corpus collection (Section 2 and 3, annotation methods (Section 4-11), and design of meaning representations (Section 12-30). We hope this overview will give inspiration and guidance in creating improved meaning banks in the future.

show abstract

Section: Be Careful With the Crowdmentioning

confidence: 99%

Thirty Musts for Meaning Banking

Bos,

Abzianidze

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…he, she, it ), rather than plural pronouns and pronouns referring to abstract entities. Nonetheless, current efforts of creating large‐scale annotated resources for anaphora, of which the crowdsourcing method presented in Chamberlain et al. (2008) is a key example, will certainly boost performance of machine learning approaches in the near future.…”

Section: Constructing Semantic Representationsmentioning

confidence: 99%

A Survey of Computational Semantics: Representation, Inference and Knowledge in Wide‐Coverage Text Understanding

Bos

2011

Language and Linguist. Compass

View full text Add to dashboard Cite

The aim of computational semantics is to capture the meaning of natural language expressions in representations suitable for performing inferences, in the service of understanding human language in written or spoken form. First-order logic is a good starting point, both from the representation and inference point of view. But even if one makes the choice of first-order logic as representation language, this is not enough: the computational semanticist needs to make further decisions on how to model events, tense, modal contexts, anaphora and plural entities. Semantic representations are usually built on top of a syntactic analysis, using unification, techniques from the lambda-calculus or linear logic, to do the book-keeping of variable naming. Inference has many potential applications in computational semantics. One way to implement inference is using algorithms from automated deduction dedicated to first-order logic, such as theorem proving and model building. Theorem proving can help in finding contradictions or checking for new information. Finite model building can be seen as a complementary inference task to theorem proving, and it often makes sense to use both procedures in parallel. The models produced by model generators for texts not only show that the text is contradiction-free; they also can be used for disambiguation tasks and linking interpretation with the real world. To make interesting inferences, often additional background knowledge is required (not expressed in the analysed text or speech parts). This can be derived (and turned into first-order logic) from raw text, semistructured databases or large-scale lexical databases such as WordNet. Promising future research directions of computational semantics are investigating alternative representation and inference methods (using weaker variants of first-order logic, reasoning with defaults), and developing evaluation methods measuring the semantic adequacy of systems and formalisms.

show abstract

“…As we will see, it is one of the contentions of this article that the collaborative approach to resource creation can also result in a better understanding of the complexity of language interpretation. This work is meant to be the definitive reference article on Phrase Detectives, collecting in a single publication material previously only found in separate papers such as , Chamberlain et al [2008aChamberlain et al [ , 2008bChamberlain et al [ , 2009aChamberlain et al [ , 2009b, and Kruschwitz et al [2009] and additional material not presented before, including a cost comparison between games, traditional annotation, and crowdsourcing, and a discussion of recent developments such as the Facebook version of the game. Our objective is to provide an assessment of the methodology and to summarize the lessons we learned so that other researchers may decide whether this methodology is appropriate for other HLT tasks.…”

Section: Introductionmentioning

confidence: 99%

Phrase detectives

Poesio

Chamberlain

Kruschwitz

et al. 2013

ACM Trans. Interact. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

We are witnessing a paradigm shift in Human Language Technology (HLT) that may well have an impact on the field comparable to the statistical revolution: acquiring large-scale resources by exploiting collective intelligence. An illustration of this new approach is Phrase Detectives, an interactive online game with a purpose for creating anaphorically annotated resources that makes use of a highly distributed population of contributors with different levels of expertise.The purpose of this article is to first of all give an overview of all aspects of Phrase Detectives, from the design of the game and the HLT methods we used to the results we have obtained so far. It furthermore summarizes the lessons that we have learned in developing this game which should help other researchers to design and implement similar games.

show abstract

Addressing the resource bottleneck to create large-scale annotated texts

Cited by 10 publications

References 7 publications

Thirty Musts for Meaning Banking

Thirty Musts for Meaning Banking

A Survey of Computational Semantics: Representation, Inference and Knowledge in Wide‐Coverage Text Understanding

Phrase detectives

Contact Info

Product

Resources

About