Gesticulate for Health’s Sake! Understanding the Use of Gestures as an Input Modality for Microtask Crowdsourcing

Abstract

Human input is pivotal in building reliable and robust artificial intelligence systems. By providing a means to gather diverse, high-quality, representative, and cost-effective human input on demand, microtask crowdsourcing marketplaces have thrived. Despite the unmistakable benefits available from online crowd work, the lack of health provisions and safeguards, along with existing work practices threatens the sustainability of this paradigm. Prior work has investigated worker engagement and mental health, yet no such investigations into the effects of crowd work on the physical health of workers have been undertaken. Crowd workers complete their work in various sub-optimal work environments, often using a conventional input modality of a mouse and keyboard. The repetitive nature of microtask crowdsourcing can lead to stress-related injuries, such as the well-documented carpal tunnel syndrome. It is known that stretching exercises can help reduce injuries and discomfort in office workers. Gestures, the act of using the body intentionally to affect the behavior of an intelligent system, can serve as both stretches and an alternative form of input for microtasks. To better understand the usefulness of the dual-purpose input modality of ergonomically-informed gestures across different crowdsourced microtasks, we carried out a controlled 2 x 3 between-subjects study (N=294). Considering the potential benefits of gestures as an input modality, our results suggest a real trade-off between worker accuracy in exchange for potential short to long-term health benefits.

SignUpCrowd: Using Sign-Language as an Input Modality for Microtask Crowdsourcing

“…Authors developed a system that allows the interpretation of sign language-to-caption text, and also provides an opportunity for deaf and mute individuals to assist those that are unable to read sign language. More recently, and in closely related work Allen, Hu, and Gadiraju (2022) proposed the use of gestures as an input modality for microtask crowdsourcing.…”

What You Show is What You Get! Gestures for Microtask Crowdsourcing

“…To this end, we conducted a set of experiments to explore the performance and perception of gesture inputs for microtasks [1]. We used three distinct microtasks, informed by the taxonomy described by Gadiraju et al [11].…”

“…The models are sourced from MediaPipe. 1 The landmarks are converted into more directly interpretable data via multiple methods using Kalidokit. 2 Details on the augmentation can be found in the library documentation.…”