A Task-Centered Framework for Computationally-Grounded Science Collaborations

Gil, Yolanda; Michel, Felix; Ratnakar, Varun; Hauder, Matheus; Duffy, Christopher; Dugan, Hilary A.; Hanson, Paul C.

doi:10.1109/escience.2015.76

Cited by 5 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some work has shown that volunteer workers provide higher data quality than paid workers in crowd-sourcing tasks. For example, Gil et al [12] found that quality control measures were required for paid crowd-workers to attain data quality similar to volunteer crowd-workers in a text annotation task. Similarly, Borromeo et al [2] found that volunteer crowd-workers provided accurate data in both simple and complex data extraction tasks, while paid crowdworker's accuracy suffered in more complex tasks.…”

Section: Effort In Online Experimentation and Citizen Sciencementioning

confidence: 99%

Pay Attention, Please

August

Reinecke

2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Participant engagement in online studies is key to collecting reliable data, yet achieving it remains an often discussed challenge in the research community. One factor that might impact engagement is the formality of language used to communicate with participants throughout the study. Prior work has found that language formality can convey social cues and power hierarchies, affecting people's responses and actions. We explore how formality influences engagement, measured by attention, dropout, time spent on the study and participant performance, in an online study with 369 participants on Mechanical Turk (paid) and LabintheWild (volunteer). Formal language improves participant attention compared to using casual language in both paid and volunteer conditions, but does not affect dropout, time spent, or participant performance. We suggest using more formal language in studies containing complex tasks where fully reading instructions is especially important. We also highlight trade-offs that different recruitment incentives provide in online experimentation.

show abstract

Section: Effort In Online Experimentation and Citizen Sciencementioning

confidence: 99%