Crowdsourcing with endogenous entry

Abstract

We investigate the design of mechanisms to incentivize high quality outcomes in crowdsourcing environments with strategic agents, when entry is an endogenous, strategic choice. Modeling endogenous entry in crowdsourcing markets is important because there is a nonzero cost to making a contribution of any quality which can be avoided by not participating, and indeed many sites based on crowdsourced content do not have adequate participation. We use a mechanism with monotone, rank-based, rewards in a model where agents strategically make participation and quality choices to capture a wide variety of crowdsourcing environments, ranging from conventional crowdsourcing contests with monetary rewards such as TopCoder, to crowdsourced content as in online Q&A forums.We begin by explicitly constructing the unique mixedstrategy equilibrium for such monotone rank-order mechanisms, and use the participation probability and distribution of qualities from this construction to address the question of designing incentives for two kinds of rewards that arise in the context of crowdsourcing. We first show that for attention rewards that arise in the crowdsourced content setting, the entire equilibrium distribution and therefore every increasing statistic including the maximum and average quality (accounting for participation), improves when the rewards for every rank but the last are as high as possible. In particular, when the cost of producing the lowest possible quality content is low, the optimal mechanism displays all but the poorest contribution. We next investigate how to allocate rewards in settings where there is a fixed total reward that can be arbitrarily distributed amongst participants, as in crowdsourcing contests. Unlike models with exogenous entry, here the expected number of participants can be increased by subsidizing entry, which could potentially improve the expected value of the best contribution. However, we show that subsidizing entry does not improve the expected quality of the best contribution, although it may improve the expected quality of the average contribution. In fact, we show that free entry is dominated by taxing entry-making all entrants pay a small fee, which is rebated to the winner along with whatever rewards were already assigned, can improve the quality of the best contribution over a winner-take-all contest with no taxes.

Incentives, Gamification, and Game Theory: An Economic Approach to Badge Design

“…There is now a growing literature on mechanism design and game-theoretic approaches to social computing and user-generated content systems [Jain and Parkes 2008;Chen et al 2009;Ghosh and McAfee 2011;Ghosh and Hummel 2011;Ghosh and McAfee 2012;Hummel 2012, 2013]. The key difference between this existing literature and our work is that prior work has largely focused on models and analysis for rewarding contributions, whereas we focus on the problem of rewarding contributors for their overall contributions to a site, rather than incentivizing desirable behavior in a single contribution decision.…”

“…The key difference between this existing literature and our work is that prior work has largely focused on models and analysis for rewarding contributions, whereas we focus on the problem of rewarding contributors for their overall contributions to a site, rather than incentivizing desirable behavior in a single contribution decision. That is, the research thus far models and prescribes reward allocation mechanisms for a single "unit" on a site-how to allocate points among the set of answers contributed to a single question [Chen et al 2009;Ghosh and McAfee 2012;Ghosh and Hummel 2012], or attention among the set of reviews for one product on Amazon or a particular restaurant on Yelp McAfee 2011, 2012;Ghosh and Hummel 2013]; or how to distribute prize money among the contestants in a single crowdsourcing contest [Chawla et al 2012;Archak and Sundarajan 2009;Ghosh and McAfee 2012]. While some of these models (such as in Ghosh and McAfee [2011] and Ghosh and Hummel [2012]) could arguably be extended to the contributor-reward problem, the analysis there regards mechanisms that are meaningful in the context of single contributions rather than overall contributor rewards.…”

“…The two most relevant papers from this literature are Chawla et al [2012] and Ghosh and McAfee [2012], which discuss mechanism design for eliciting optimal effort from agents in a (single) crowdsourcing contest. Chawla et al [2012] study the design and approximation of optimal crowdsourcing contests, but in a model with a fixed number of participants, whereas modeling endogenous entry-the fact that participation on a UGC site is a voluntary choice-is crucial and significantly changes the nature of our analysis.…”

“…Chawla et al [2012] study the design and approximation of optimal crowdsourcing contests, but in a model with a fixed number of participants, whereas modeling endogenous entry-the fact that participation on a UGC site is a voluntary choice-is crucial and significantly changes the nature of our analysis. Ghosh and McAfee [2012] address questions related to optimal reward design in a setting with endogenous entry. However, they do so in a model with perfectly observable output, a reasonable assumption in the specific context of crowdsourcing in which the principal posing the task can supply an accurate rank ordering of contributions.…”

A co-utility approach to the mesh economy: the crowd-based business model

“…According to Ghosh and Mc Afee (2012) quality and participation (with an associated cost of effort) are the key issues that arise in the crowdsourcing analysis. They argue that the level of effort an agent chooses to exert depends on the underlying incentives offered.…”