Trustworthiness in enterprise crowdsourcing

Dwarakanath, Anurag; Shrikanth, N. C.; Kumar, Abhinav; Kass, Alex

doi:10.1145/2889160.2889225

Cited by 20 publications

(19 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work includes additional methods to improve worker responses, such as contest type tasks [13] and alternative worker compensations. We will explore the use of automated extraction tools to complementing the crowdsourcing methodology.…”

Section: Discussionmentioning

confidence: 99%

Çorba: crowdsourcing to obtain requirements from regulations and breaches

et al. 2019

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Çorba: crowdsourcing to obtain requirements from regulations and breaches

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Data regarding workforce reliability (in terms of complete task submission and thus participation), task registration speed, task completion duration, skills (on programming language knowledge list) and challenge types (on the nature and rewards of each contest) were analyzed and signalized as catalyst factors for success. Similarly, Dwarakanath et al [8] assessed the crowd trustworthiness based on submission quality, timeliness and ownership concluding that task requirements, user efficacy and reputation strongly influence the trustworthiness of the crowd. Karim et al in [13] proposed a recommendation system that can help mainly crowd workers (and as a side effect providers) on taking over the appropriate tasks based on technology requirements and their skills.…”

Section: F How To Crowdsource?mentioning

confidence: 99%

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

et al. 2020

View full text Add to dashboard Cite

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

show abstract

“…In the literature [25], Dubey A et al analyzed crowd developer's characteristics on Topcoder.com and upwork.com to predict the quality of tasks completed by them. Furthermore, they put forward the trustworthiness of software crowdsourcing to help enterprises get rid of certain risks [26]. Alelyani T and Yang Y [27] discussed the reliability that different workers showed on different tasks, so as to find out certain behavioral characteristics of the crowd developers.…”

Section: Research On Crowd Workersmentioning

confidence: 99%

A Data-Driven Game Theoretic Strategy for Developers in Software Crowdsourcing: A Case Study

Liao

Zhang

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Crowdsourcing has the advantages of being cost-effective and saving time, which is a typical embodiment of collective wisdom and community workers’ collaborative development. However, this development paradigm of software crowdsourcing has not been used widely. A very important reason is that requesters have limited knowledge about crowd workers’ professional skills and qualities. Another reason is that the crowd workers in the competition cannot get the appropriate reward, which affects their motivation. To solve this problem, this paper proposes a method of maximizing reward based on the crowdsourcing ability of workers, they can choose tasks according to their own abilities to obtain appropriate bonuses. Our method includes two steps: Firstly, it puts forward a method to evaluate the crowd workers’ ability, then it analyzes the intensity of competition for tasks at Topcoder.com—an open community crowdsourcing platform—on the basis of the workers’ crowdsourcing ability; secondly, it follows dynamic programming ideas and builds game models under complete information in different cases, offering a strategy of reward maximization for workers by solving a mixed-strategy Nash equilibrium. This paper employs crowdsourcing data from Topcoder.com to carry out experiments. The experimental results show that the distribution of workers’ crowdsourcing ability is uneven, and to some extent it can show the activity degree of crowdsourcing tasks. Meanwhile, according to the strategy of reward maximization, a crowd worker can get the theoretically maximum reward.

show abstract

Trustworthiness in enterprise crowdsourcing

Cited by 20 publications

References 26 publications

Çorba: crowdsourcing to obtain requirements from regulations and breaches

Çorba: crowdsourcing to obtain requirements from regulations and breaches

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

A Data-Driven Game Theoretic Strategy for Developers in Software Crowdsourcing: A Case Study

Contact Info

Product

Resources

About