Efficient crowdsourcing of unknown experts using bounded multi-armed bandits

Abstract

Increasingly, organisations flexibly outsource work on a temporary basis to a global audience of workers. This so-called crowdsourcing has been applied successfully to a range of tasks, from translating text and annotating images, to collecting information during crisis situations and hiring skilled workers to build complex software. While traditionally these tasks have been small and could be completed by non-professionals, organisations are now starting to crowdsource larger, more complex tasks to experts in their respective fields. These tasks include, for example, software development and testing, web design and product marketing. While this emerging expert crowdsourcing offers flexibility and potentially lower costs, it also raises new challenges, as workers can be highly heterogeneous, both in their costs and in the quality of the work they produce. Specifically, the utility of each outsourced task is uncertain and can vary significantly between distinct workers and even between subsequent tasks assigned to the same worker. Furthermore, in realistic settings, workers have limits on the amount of work they can perform and the employer will have a fixed budget for paying workers. Given this uncertainty and the relevant constraints, the objective of the employer is to assign tasks to workers in order to maximise the overall utility achieved. To formalise this expert crowdsourcing problem, we introduce a novel multi-armed bandit (MAB) model, the bounded MAB. Furthermore, we develop an algorithm to solve it efficiently, called bounded ε-first, which proceeds in two stages: exploration and exploitation. During exploration, it first uses εB of its total budget B to learn estimates of the workers' quality characteristics. Then, during exploitation, it uses the remaining (1 − ε)B to maximise the total utility based on those estimates. Using this technique allows us to derive an O (B 2 3 ) upper bound on its performance regret (i.e., the expected difference in utility between our algorithm and the optimum), which means that as the budget B increases, the regret tends to 0. In addition to this theoretical advance, we apply our algorithm to real-world data from oDesk, a prominent expert crowdsourcing site. Using data from real projects, including historic project budgets, expert costs and quality ratings, we show that our algorithm outperforms existing crowdsourcing methods by up to 300%, while achieving up to 95% of a hypothetical optimum with full information.

Mathematical Modeling of Crowdsourcing Systems: Incentive Mechanism and Rating System Design

“…Another approach determines the price automatically or dynamically via designing some efficient mechanisms. Typical examples include a multi-armed bandits based pricing mechanism [24], and auction based pricing mechanisms [25]. However these approaches do not solve the challenge of free-riding of workers, or denial of payment of requesters.…”

Interactive Machine Learning (iML)

“…Es soll jener Spielautomat ermittelt werden, der über die beste Auszahlungsrate verfügt, aber gleichzeitig sollen die Gewinne maximiert werden. Dabei werden zum einen Automaten berücksichtigt, die in der Vergangenheit hohe Gewinne erbracht haben, und zum anderen neue oder scheinbar schlechtere Automaten, die möglicherweise noch bessere Gewinne erbringen könnten [13]. Im Prinzip modellieren MABs also Agenten, die gleichzeitig versuchen, neue Kenntnisse zu erwerben (,,Exploration") und ihre Entscheidungen auf der Grundlage der vorhandenen Kenntnisse zu optimieren (,,Verwertung" [9].…”

A survey of the use of crowdsourcing in software engineering

“…Tran-Thanh et al [258] proposed a bounded multi-armed bandit model for expert crowdsourcing. Specifically, the proposed ε-first algorithm works in two stages: First, it explores the estimation of workers' quality by using part of the total budget; Second, it exploits the estimates of workers' quality to maximise the overall utility with the remaining budget.…”