What You Show is What You Get! Gestures for Microtask Crowdsourcing

Abstract

Crowdsourcing is a valuable tool to gather human input which enables the development of reliable artificial intelligence systems. Microtask platforms like Prolific and Amazon's Mechanical Turk have flourished by creating environments where crowd workers can provide such human input in a diverse and representative manner. Such marketplaces have evolved to support several hundreds of workers in earning their primary livelihood through crowd work. Crowd workers, however, often perform these tasks in sub-optimal work environments with poor ergonomics. Additionally, many of the various microtasks require input via the standard method of a mouse and keyboard and are repetitive in nature. As such, crowd workers who primarily earn their livelihoods in microtask marketplaces are at risk of injuries such as carpal tunnel syndrome. By changing the input modality from a mouse and keyboard to gesture-driven input, crowd workers can complete their work while simultaneously improving or safeguarding their physical health. Through three distinct microtasks, we constructed a dataset that enables the exploration of the physical and mental health of crowd workers while using gestures. In this work, we present the process of constructing this dataset, how we applied it, and the future applications we foresee.

AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning

“…Concurrent to this paper, a few other groups investigate the intersection of crowdsourcing and LLMs. One paper explores the impact of incorporating LLMs into crowdsourcing workflows on requester and crowdworker stakeholders [3]. Another implements three strategies from crowdsourcing using LLMs for data processing tasks [116].…”