Crowdsourcing Software Task Assignment Method for Collaborative Development

Yu, DanDan; Zhou, Zhuang; Wang, Yi

doi:10.1109/access.2019.2905054

Cited by 23 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e three factors that the authors consider are the ability of the worker, active time of the worker, and the complexity of task module. e individuals are divided into collaborative groups, and then, for the optimal selection of worker to perform a task, the Hungarian algorithm is used [27]. e automatic detection for improper task is also proposed in the process of crowdsourcing.…”

Section: Related Workmentioning

confidence: 99%

MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in Crowdsourcing

Zhao

Khan

et al. 2021

Scientific Programming

View full text Add to dashboard Cite

Crowdsourcing in simple words is the outsourcing of a task to an online market to be performed by a diverse group of crowds in order to utilize human intelligence. Due to online labor markets and performing parallel tasks, the crowdsourcing activity is time- and cost-efficient. During crowdsourcing activity, selecting the proper labeled tasks and assigning them to an appropriate worker are a challenge for everyone. A mechanism has been proposed in the current study for assigning the task to the workers. The proposed mechanism is a multicriteria-based task assignment (MBTA) mechanism for assigning the task to the most suitable worker. This mechanism uses approaches for weighting the criteria and ranking the workers. These MCDM methods are Criteria Importance Through Intercriteria Correlation (CRITIC) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). Criteria have been made for the workers based on the identified features in the literature. Weight has been assigned to these selected features/criteria with the help of the CRITIC method. The TOPSIS method has been used for the evaluation of workers, with the help of which the ranking of workers is performed in order to get the most suitable worker for the selected tasks to be performed. The proposed work is novel in several ways; for example, the existing methods are mostly based on single criterion or some specific criteria, while this work is based on multiple criteria including all the important features. Furthermore, it is also identified from the literature that none of the authors used MCDM methods for task assignment in crowdsourcing before this research.

show abstract

Section: Related Workmentioning

confidence: 99%

MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in Crowdsourcing

Zhao

Khan

et al. 2021

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…Project Planning [25] Requirement Management [20] Programming/Coding [14,21,37] Software Testing [7,17] Software Integration [7] Debugging [40] Software Evolution [40] Generic development [8][9][10][11][12][13]15,16,18,19,[22][23][24][26][27][28][29][30][31][32][33][34][35][36]38,39,41,42] The activities mentioned in Table 5 are the only software engineering activities where freelancers were involved as reported by the literature studies. The formulation of the category labels and classification of the studies in these categories were made on the basis of the software engineering activity that the literature studies have focused on.…”

Section: Software Development Area Research Studymentioning

confidence: 99%

Freelancers in the Software Development Process: A Systematic Mapping Study

2020

View full text Add to dashboard Cite

[Context] Freelancers could catalyze the software development process by providing their niche skills to generate high quality outputs. They could help companies (including startups) to foster innovations by suggesting creative ideas and providing their expertise in implementing them (for instance, designing solutions, coding solutions etc.). Freelancers could effectively and efficiently work as a virtual member of the software development team. The company must make informed decisions about which task to allot to the freelancer, which freelancer to select, pricing the task, and evaluating the submitted work. On the other hand, the freelancer should make an informed decision about evaluating the monetary value of the task to be charged, trusting the requester, analyzing the skills requirement of the task (finding matches between skill requirement and skills processed), selecting the best task, and maintaining the highest level of reputation. However, the literature does not provide freelancers and the companies the guidelines that support their decision making. However, if freelancers are selected carefully for the most suitable task, the companies will benefit a lot in terms of improved software development metrics. [Objectives] The objective of this paper is to provide the research community the research trends in freelancer-supported software development. This helps to understand that which software development areas have higher concentrations of research efforts, which area has the support of empirical evidence to support management decision makings, and which area requires the research attention. [Method] The systematic study is conducted by planning the mapping protocol, executing the protocol, and reporting the findings using various visualization tools like bar charts and pie charts. The search process was planned to be executed using set of inclusion and exclusion conditions on four bibliographic databases (IEEExplore, Springerlink, Sciencedirect, and ACM digital library). The relevant papers are selected by applying inclusion and exclusion conditions. The google citations of the relevant papers are subject to the inclusion and exclusion conditions again to include the more relevant papers. Finally, the systematic schema was created and populated after analyzing the studies abstracts. [Results] The results indicate the following (a) The research focus is on generic software development (78%) rather on individual life cycle activities. (b) The number of empirical studies is limited (25%). (c) A number of studies proposing solutions and evaluating on live cases in industrial settings are missing from the literature. This is in comparison to the validation approaches (72%) i.e., solutions tested in laboratory settings. (d) At present, the literature has limited ability to provide the software companies (including startups) with the guidelines (in the form of opinions and experience reports) for involving freelancers in the software development process. (e) The reported challenges include Collaboration and Coordination (33%), Developer Recommendation (or selection) (19%), Team Formulation (14%), Task Recommendation (allocation) (14%), Task Decomposition (11%), Privacy and Security (Confidentiality) (11%), Budget Estimation (8%), Recognition (8%), Trust Issues (8%), Market Dynamism (6%), Intellectual Property Issues (6%), Participation of Crowd Worker (6%), and Capacity Utilization (3%). These challenges are highly interactive, and each challenge impacts all other challenges. (e) Recent focus of the researchers (total 7 studies in 2019) is on generic software development handling the collaboration and coordination (3 studies out of 7), Developer recommendation (2 studies out of 7), and task recommendation (2 studies out of 7). [Conclusion] The freelancer-driven software engineering research area has got the attraction of the researchers, but it will take a long time to gain maturity. This puts an urgent call for more empirical studies and evaluation-based solution research that could help companies (including startups) to foster innovations. Further, the research focus should be well distributed among the various development phases to address the unique challenges associated with individual activities. The accurate management of the freelancer in the software development could help companies and startups to foster innovations and remain competitive in the marketplace.

show abstract

“…Though in this study the authors do not refer to the specific types of tasks crowdsourced but rather examine the efficacy of task division practices. Similarly Yu et al [40] explored task assignment and division in collaborative crowdsourced development.…”

Section: B What To Crowdsource?mentioning

confidence: 99%

“…Since in this study we focused on analysing data from TopCoder platform, a pioneer in crowdsourced software development, platform usability was excluded as it could be considered the same for all types of challenges and activities studies. Additionally, crowd collaboration [40] was out of the scope of the study since we focused on competitive CSE, not collaborative and therefore factors addressing crowd collaboration were not applicable in this case. Another threat to validity is the selection of thresholds for transforming continuous variables to binary ones.…”

Section: Threats To Validitymentioning

confidence: 99%

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

et al. 2020

View full text Add to dashboard Cite

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

show abstract

Crowdsourcing Software Task Assignment Method for Collaborative Development

Cited by 23 publications

References 31 publications

MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in Crowdsourcing

MCDM Approach for Assigning Task to the Workers by Selected Features Based on Multiple Criteria in Crowdsourcing

Freelancers in the Software Development Process: A Systematic Mapping Study

Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Contact Info

Product

Resources

About