Phrase detectives

Poesio, Massimo; Chamberlain, Jon; Kruschwitz, Udo; Robaldo, Livio; Ducceschi, Luca

doi:10.1145/2448116.2448119

Cited by 69 publications

(23 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Search Terms Used Sicart (2010) Morality & games Lankoski (2013) Morality & "Video games" Siu & Reidl (2016) Design & "Games with a purpose" Poesio, Chamberlain, Kruschwitz, Robaldo, and Ducceschi (2015) Design & "Games with a purpose" Flanagan (2009) Added afterward Schrier (2016a) Added due to being late-breaking and not searchable yet Narvaez (2006) "Moral education" & design Berkowitz & Bier (2007) "Moral education" & design Sicart (2009a) Morality & "video games" & "design" Mah, Taylor, Hoang, and Cook (2014) Morality & Learning & "Design principles" Konstantinidis, Tsiatsos, and Pomportsis (2009) Teaching & morality & "Design principles" von Ahn & Dabbish (2008) Design & "Games with a Purpose" Dulacka (2012) Design & "Games with a Purpose" Mayhew and King (2008) "Moral education" & design Pe-Than, Goh, and Lee 2014"Crowdsourcing games" & design Deng, Joshi, and Galliers (2016) Crowdsourcing & Ethics Appendix B. Round One Design Principle Categories From the Literature Review.…”

Section: Appendixmentioning

confidence: 99%

Designing Games for Moral Learning and Knowledge Building

Schrier

2017

Games and Culture

View full text Add to dashboard Cite

How can we better learn about and teach moral thinking and skills? How can we solve moral problems? One possible way is to create and use moral learning games, or games that enable players to work on moral scenarios, make moral choices, and gain relevant skills. One possible subcategory of these games is moral knowledge games, or games that aim to solve real-world moral problems and create new knowledge about morality. This article systematically analyzes relevant literature and related games and media to uncover a preliminary set of design principles for creating moral learning games and moral knowledge games. Frameworks such as the Elemental Tetrad, Mechanics, Dynamics, and Aesthetics (MDA), and Ethics Practice and Implementation Categorization Framework (EPIC) were used to analyze individual games and media. Ten different categories of principles emerged, along with 95 possible subprinciples. Implications, next steps, and limitations of this analysis were also discussed.

show abstract

Section: Appendixmentioning

confidence: 99%

Designing Games for Moral Learning and Knowledge Building

Schrier

2017

Games and Culture

View full text Add to dashboard Cite

show abstract

“…However, in reality, these experts may have varying levels of expertise depending on the domains, and thus may disagree on labelling in certain cases (Aroyo and Welty, 2013). A rapid and costeffective alternative is to obtain labels through crowdsourcing (Snow et al, 2008;Poesio et al, 2013Poesio et al, , 2017. In crowdsourcing, each instance is presented to multiple expert or non-expert annotators for labelling.…”

Section: Introductionmentioning

confidence: 99%

Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks

Li¹,

Myrman²,

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

When constructing models that learn from noisy labels produced by multiple annotators, it is important to accurately estimate the reliability of annotators. Annotators may provide labels of inconsistent quality due to their varying expertise and reliability in a domain. Previous studies have mostly focused on estimating each annotator's overall reliability on the entire annotation task. However, in practice, the reliability of an annotator may depend on each specific instance. Only a limited number of studies have investigated modelling perinstance reliability and these only considered binary labels. In this paper, we propose an unsupervised model which can handle both binary and multi-class labels. It can automatically estimate the per-instance reliability of each annotator and the correct label for each instance. We specify our model as a probabilistic model which incorporates neural networks to model the dependency between latent variables and instances. For evaluation, the proposed method is applied to both synthetic and real data, including two labelling tasks: text classification and textual entailment. Experimental results demonstrate our novel method can not only accurately estimate the reliability of annotators across different instances, but also achieve superior performance in predicting the correct labels and detecting the least reliable annotators compared to stateof-the-art baselines. 1

show abstract

“…The lack of fallacy-annotated linguistic resources and thus the need for creating and labeling a new dataset from scratch motivated us to investigate serious games (also games with a purpose)-a scenario in which a task is gamified and users (players) enjoy playing a game without thinking much of the burden of annotations (von Ahn and Dabbish, 2008;Mayer et al, 2014). Serious games have been successful in NLP tasks that can be easily represented by images Kazemzadeh et al, 2014) or that can be simplified to assessing a single word or a pair of propositions (Nevěřilová, 2014;Poesio et al, 2013). More complex tasks such as argument un-derstanding, reasoning, or composing pose several design challenges centered around the key question: how to make data creation and annotation efforts fun and entertaining in the first place.…”

Section: Introductionmentioning

confidence: 99%

Argotario: Computational Argumentation Meets Serious Games

Habernal¹,

Hannemann²,

Pollak³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

An important skill in critical thinking and argumentation is the ability to spot and recognize fallacies. Fallacious arguments, omnipresent in argumentative discourse, can be deceptive, manipulative, or simply leading to 'wrong moves' in a discussion. Despite their importance, argumentation scholars and NLP researchers with focus on argumentation quality have not yet investigated fallacies empirically. The nonexistence of resources dealing with fallacious argumentation calls for scalable approaches to data acquisition and annotation, for which the serious games methodology offers an appealing, yet unexplored, alternative. We present Argotario, a serious game that deals with fallacies in everyday argumentation. Argotario is a multilingual, open-source, platform-independent application with strong educational aspects, accessible at www.argotario.net.

show abstract

Phrase detectives

Cited by 69 publications

References 55 publications

Designing Games for Moral Learning and Knowledge Building

Designing Games for Moral Learning and Knowledge Building

Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks

Argotario: Computational Argumentation Meets Serious Games

Contact Info

Product

Resources

About