Simulation-Based Modeling and Evaluation of Incentive Schemes in Crowdsourcing Environments

Abstract

Abstract. Conventional incentive mechanisms were designed for business environments involving static business processes and a limited number of actors. They are not easily applicable to crowdsourcing and other social computing platforms, characterized by dynamic collaboration patterns and high numbers of actors, because the effects of incentives in these environments are often unforeseen and more costly than in a well-controlled environment of a traditional company.In this paper we investigate how to design and calibrate incentive schemes for crowdsourcing processes by simulating joint effects of a combination of different participation and incentive mechanisms applied to a working crowd. More specifically, we present a simulation model of incentive schemes and evaluate it on a relevant real-world scenario. We show how the model is used to simulate different compositions of incentive mechanisms and model parameters, and how these choices influence the costs on the system provider side and the number of malicious workers.

PRINGL – A domain-specific language for incentive management in crowdsourcing

“…The basic incentive modeling concepts that were used to design pringl were inspired by, or based upon concepts previously introduced in the set of our background papers: in [15] we presented a possible model of the abstraction interlayer (Section 3.2); the basic functionality of the interlayer's incentive modeling capabilities were simulated and tested in [16]; and finally, in [10,17] we present components of a framework that allows provisioning and communication with collectives of human workers. We are currently working on integrating pringl and the aforementioned components into a single end-to-end socio-technical system with incentive management capabilities.…”

Reward‐Based Participant Management for Crowdsourcing Rainfall Monitoring: An Agent‐Based Model Simulation

“…Mazzoleni et al (2018) used a simple crowd participation model to estimate the impacts of various motivators on the interest of agents in a water level reporting project. Scekic et al (2013) used an agent-based model (ABM) to simulate the traffic reporting behavior of the crowd under three different incentive/penalty schemes. For other quantitative studies on crowdsourcing participant management, see Buecheler et al (2011), Zou et al (2014), and Yu et al (2016).…”

“…When allocating monetary rewards for example, because of the varying reliability of crowd participants in providing data (ul Hassan & Curry, 2016), a uniform allocation of the reward budget across all participants would not be optimal. Further, the best possible incentive type and manner of allocation out of many availables (Scekic et al, 2013) are often not obvious and would depend on numerous factors such as the environmental awareness of participants, their response rates to incentives, their spatial distribution, and the ensuing administrative cost (Newman et al, 2012;Tokarchuk et al, 2012).…”

“…Moreover, those past studies either assumed overly simple representations of the crowdsourcing process (Borst, 2010;Mazzoleni et al, 2018) and disregarded participant heterogeneity or were such that spatial heterogeneity was not an important influencing factor (Borst, 2010;Buecheler et al, 2011;Scekic et al, 2013;Yu et al, 2016;Zou et al, 2014). In the geological and environmental context though, the effects of heterogeneity can be significant, and if so, need be accounted for.…”

Cloud Implementation Of Agent-Based Simulation Model In Evacuation Scenarios

“…Despite some initial security concerns and technical issues, an increasing number of institutions are considering moving their applications and services into "The Cloud." Recently, cloud-based technologies have been successfully used as effective support tools in complex simulations in emergency situations [21][22][23]. MapReduce programing model [24][25][26] is very popular tool for implementation of the crowd simulation in evacuation situations as two-stage mapper-reducer process.…”