Investigating the Accessibility of Crowdwork Tasks on Mechanical Turk

Uzor, Stephen; Jacques, Jason T.; Dudley, John J.; Kristensson, Per Ola

doi:10.1145/3411764.3445291

Cited by 20 publications

(11 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, Exp3 does not perform well in any settings (Figs. [5][6][7][8][9][10][11]. This is because choosing the incentives randomly based on their weights does not work well when the time budget is small.…”

Section: Resultsmentioning

confidence: 99%

“…We can include task quantity, task quality, task completion time, or some subset of them in the utility function. 6 For 5 The Stepped fKUBE algorithm will be presented in detail in Section 6.1. 6 A note when choosing an aspect to be included in the metric is that it should not only measure the effectiveness of contest implementations (i.e., incentives) appropriately, but it should also be possible to evaluate the effectiveness easily and ideally in an automatic manner.…”

Section: The Incentive Selection Problemmentioning

confidence: 99%

“…6 For 5 The Stepped fKUBE algorithm will be presented in detail in Section 6.1. 6 A note when choosing an aspect to be included in the metric is that it should not only measure the effectiveness of contest implementations (i.e., incentives) appropriately, but it should also be possible to evaluate the effectiveness easily and ideally in an automatic manner. That is because after deploying a contest of a specific implementation in a period of time, the effectiveness of this implementation should be calculated quickly before deploying another contest (to another group of participants) in the next period.…”

Section: The Incentive Selection Problemmentioning

confidence: 99%

“…Crowdsourcing has emerged as an efficient approach for obtaining solutions to a wide variety of problems by engaging a large number of Internet users from many places in the world [2][3][4][5][6][7]. Crowdsourcing is attractive to organisations and companies not only because it provides cheap labour, but also because it helps to solve problems quickly and effectively [8][9][10].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient and adaptive incentive selection for crowdsourcing contests

et al. 2022

View full text Add to dashboard Cite

The success of crowdsourcing projects relies critically on motivating a crowd to contribute. One particularly effective method for incentivising participants to perform tasks is to run contests where participants compete against each other for rewards. However, there are numerous ways to implement such contests in specific projects, that vary in how performance is evaluated, how participants are rewarded, and the sizes of the prizes. Also, the best way to implement contests in a particular project is still an open challenge, as the effectiveness of each contest implementation (henceforth, incentive) is unknown in advance. Hence, in a crowdsourcing project, a practical approach to maximise the overall utility of the requester (which can be measured by the total number of completed tasks or the quality of the task submissions) is to choose a set of incentives suggested by previous studies from the literature or from the requester’s experience. Then, an effective mechanism can be applied to automatically select appropriate incentives from this set over different time intervals so as to maximise the cumulative utility within a given financial budget and a time limit. To this end, we present a novel approach to this incentive selection problem. Specifically, we formalise it as an online decision making problem, where each action corresponds to offering a specific incentive. After that, we detail and evaluate a novel algorithm, , to solve the incentive selection problem efficiently and adaptively. In theory, in the case that all the estimates in (except the estimates of the effectiveness of each incentive) are correct, we show that the algorithm achieves the regret bound of $\mathcal {O}(\sqrt {B/c})$ O ( B / c ) , where B denotes the financial budget and c is the average cost of the incentives. In experiments, the performance of is about 93% (up to 98%) of the optimal solution and about 9% (up to 40%) better than state-of-the-art algorithms in a broad range of settings, which vary in budget sizes, time limits, numbers of incentives, values of the standard deviation of the incentives’ utilities, and group sizes of the contests (i.e., the numbers of participants in a contest).

show abstract

Section: Resultsmentioning

confidence: 99%

Section: The Incentive Selection Problemmentioning

confidence: 99%

Section: The Incentive Selection Problemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient and adaptive incentive selection for crowdsourcing contests

et al. 2022

View full text Add to dashboard Cite

show abstract

“…As a result, future research in this domain might need to narrow the scope of inquiry to a certain cultural context, socioeconomic background, or industry. For instance, recent work has investigated WFH experiences of people with disabilities [18,31,102] and online gig workers [110]. Focus groups, interviews, or workshops with a certain organization or group may yield additional insights into the challenges that specific kinds of workers face and how these specific groups of workers negotiate boundaries to reshape their sense of place in the home.…”

Section: Limitations and Future Workmentioning

confidence: 99%

Topophilia, Placemaking, and Boundary Work

Janghee

Beck

Voida

2022

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

The COVID-19 pandemic fundamentally changed the nature of work by shifting most in-person work to a predominantly remote modality as a way to limit the spread of the coronavirus. In the process, the shift to working-from-home rapidly forced the large-scale adoption of groupware technologies. Although prior empirical research examined the experience of working-from-home within small-scale groups and for targeted kinds of work, the pandemic provides HCI and CSCW researchers with an unprecedented opportunity to understand the psycho-social impacts of a universally mandated work-from-home experience rather than an autonomously chosen one. Drawing on boundary theory and a methodological approach grounded in humanistic geography, we conducted a qualitative analysis of Reddit data drawn from two work-from-home-related subreddits between March 2020 and January 2021. In this paper, we present a characterization of the challenges and solutions discussed within these online communities for adapting work to a hybrid or fully remote modality, managing reconfigured work-life boundaries, and reconstructing the home's sense of place to serve multiple, sometimes conflicting roles. We discuss how these findings suggest an emergent interplay among adapted work practice, reimagined physical (and virtual) spaces, and the establishment and continual re-negotiation of boundaries as a means for anticipating the long-term impact of COVID on future conceptualizations of productivity and work.

show abstract