Turking in a Global Labour Market

Martín, David; O’Neill, Jacki; Gupta, Neha; Hanrahan, Benjamin V.

doi:10.1007/s10606-015-9241-6

Cited by 71 publications

(84 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Modern crowdwork is a relatively novel type of work if looked at through the prism of technology and the Internet but it is also in many ways simply the modern twist on home-based (or sometimes mobile) piece-work. In this section we draw heavily on our previously studied in-depth ethnographic studies of US and Indian crowdworkers on AMT [17,31,32]. When crowdworkers are studied in depth it is obvious that so much of what they talk about and how they talk about it is classic 'shop talk' -i.e.…”

Section: Why: What Motivates Crowdworkers?mentioning

confidence: 99%

“…For the most part this section will deal with qualitative insights about how crowdworkers organise their working lives, their worksites and their actual crowdworking. To do this we will draw again most specifically on the ethnographic studies of Indian and US Turkers [17,31,32]. The study by Gupta et al [17] of Indian Turkers is our richest source of observational data as the lead author actually visited a number of Turkers, saw their worksites and observed them Turking.…”

Section: How: Social and Organisational Aspects Of Doing Crowdworkmentioning

confidence: 99%

“…The study by Gupta et al [17] of Indian Turkers is our richest source of observational data as the lead author actually visited a number of Turkers, saw their worksites and observed them Turking. In the work by Martin and colleagues [31,32] we draw on forum discussion of these matters to provide insights into the ways in which people work, the circumstances of their work, how they manage work and life and what other resources and technologies they use in organising their work. We cannot make statements with absolute certainty about the correlation between our findings of crowdworkers with that of workers using other platforms that we have not directly studied, but given that a number of these crowdworkers do crowdwork on other platforms with a similar social and organisational set up, and given other studies (e.g.…”

Section: How: Social and Organisational Aspects Of Doing Crowdworkmentioning

confidence: 99%

See 2 more Smart Citations

Understanding the Crowd: Ethical and Practical Matters in the Academic Use of Crowdsourcing

Martín

Carpendale

Gupta

et al. 2017

Evaluation in the Crowd. Crowdsourcing and Human-Centered Experiments

Self Cite

View full text Add to dashboard Cite

Section: Why: What Motivates Crowdworkers?mentioning

confidence: 99%

Section: How: Social and Organisational Aspects Of Doing Crowdworkmentioning

confidence: 99%

Section: How: Social and Organisational Aspects Of Doing Crowdworkmentioning

confidence: 99%

See 1 more Smart Citation

Understanding the Crowd: Ethical and Practical Matters in the Academic Use of Crowdsourcing

Martín

Carpendale

Gupta

et al. 2017

Evaluation in the Crowd. Crowdsourcing and Human-Centered Experiments

Self Cite

View full text Add to dashboard Cite

“…Frequently absent from these discussions is the notion that the participants themselves are worthy of direct consideration. In discussing the ethical obligations owed to individuals as workers, Martin, O'Neill, Gupta, and Hanrahan (2016) describe this distancing between requesters and Turkers:…”

Section: Invisible Participantsmentioning

confidence: 99%

Invisible Participants: Recommendations for Researchers Using Crowdsourcing Platforms for Human Subjects Research

Morrissey¹,

Yamasaki²,

Levitan³

2019

Preprint

View full text Add to dashboard Cite

Increasingly, behavioral scientists rely on crowdsourcing platforms (e.g., Amazon’s MTurk) to collect large datasets rapidly and inexpensively. However, by heightening the distance between participants and researchers, these platforms render the participants “invisible,” and thereby raise ethical concerns related to participant autonomy. Moreover, because many participants use these platforms to earn a living, researchers must consider issues of fair compensation, and the relationship between employment and research participation. We present an overview of ethical considerations raised by the invisibility of crowdsourced research participants, including recommendations for researchers to consider in deciding whether and how to utilize these platforms. Specifically, we recommend that researchers: explore alternatives that distinguish research from employment; strengthen participant-research communication, including by recognizing and utilizing participant expertise in study design; and, explore alternatives that do not require “payment,” or, barring that, determine and pay appropriate compensation. We conclude by locating these ethical considerations in the context of broader calls for more thoughtful data collection.

show abstract

“…Authors of [49] focused on the Indian and US worker communities and highlighted the effects of current crowdsourcing platforms and marketplaces on crowd worker experience. In [48], authors studied how crowd workers on AMT use web forums to create communities where they share experiences and voice crowd work-related problems.…”

Section: Crowd Work Context and Barriersmentioning

confidence: 99%

Modus Operandi of Crowd Workers

Gadiraju

Checco

Gupta

et al. 2017

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

Self Cite

View full text Add to dashboard Cite

The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that of work environments; defined as the hardware and software affordances at the disposal of crowd workers which are used to complete microtasks on crowdsourcing platforms. In this paper, we reveal the significant role of work environments in the shaping of crowd work. First, through a pilot study surveying the good and bad experiences workers had with UI elements in crowd work, we revealed the typical issues workers face. Based on these findings, we then deployed over 100 distinct microtasks on CrowdFlower, addressing workers in India and USA in two identical batches. These tasks emulate the good and bad UI element designs that characterize crowdsourcing microtasks. We recorded hardware specifics such as CPU speed and device type, apart from software specifics including the browsers used to complete tasks, operating systems on the device, and other properties that define the work environments of crowd workers. Our findings indicate that crowd workers are embedded in a variety of work environments which influence the quality of work produced. To confirm and validate our data-driven findings we then carried out semi-structured interviews with a sample of Indian and American crowd workers from this platform. Depending on the design of UI elements in microtasks, we found that some work environments support crowd workers more than others. Based on our overall findings resulting from all the three studies, we introduce ModOp, a tool that helps to design crowdsourcing microtasks that are suitable for diverse crowd work environments. We empirically show that the use of ModOp results in reducing the cognitive load of workers, thereby improving their user experience without effecting the accuracy or task completion time.

show abstract

Turking in a Global Labour Market

Cited by 71 publications

References 19 publications

Understanding the Crowd: Ethical and Practical Matters in the Academic Use of Crowdsourcing

Understanding the Crowd: Ethical and Practical Matters in the Academic Use of Crowdsourcing

Invisible Participants: Recommendations for Researchers Using Crowdsourcing Platforms for Human Subjects Research

Modus Operandi of Crowd Workers

Contact Info

Product

Resources

About