Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Abstract

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

The Kirkpatrick model for training evaluation: bibliometric analysis after 60 years (1959–2020)

“…Various studies have performed bibliometric analysis using a 5Ws approach for informationgathering or problem-solving. For example, Calero et al (2020) conducted a bibliometric analysis of green and sustainable software using a 5Ws or 5Ws þ 1H approach (Galindo et al, 2019;Bibi et al, 2020) and Rahimi and Rosman (2020) used the 5Ws approach to investigate enterprise content management.…”

Crowdsourced Requirements Engineering Challenges and Solutions: A Software Industry Perspective

“…SW CS is the assignation of a worldwide pool of online developers who can be appointed on-demand to contribute to several types of software development tasks [5][6][7]. The process is facilitated by platforms that link requesters and online developers.…”