Analysis of the Key Factors for Software Quality in Crowdsourcing Development: An Empirical Study on TopCoder.com

Li, Ké; Xiao, Junchao; Wang, Yongji; Wang, Qing

doi:10.1109/compsac.2013.133

Cited by 32 publications

(26 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The payment and the number of requirements factors were identified as significant predictors for final submission quality. Li et al [105] also conducted a case study on TopCoder to identify important quality factors.…”

Section: Case Studiesmentioning

confidence: 99%

“…Most are based on one or several commercial platforms described above. Among them, the TopCoder platform has the most case studies reported upon in the literature [3], [5], [22], [62], [65], [103]- [105].…”

Section: Case Studiesmentioning

confidence: 99%

“…Li et al [105] identified 23 quality factors for crowdsourced software development from the perspective of platform and project, based on an empirical study of TopCoder. Four important aspects were identified in order to improve crowdsourced software quality, including the prosperity level of the platform, the scale of the task, the participants' skill levels and the design quality of the task.…”

Section: Quality Assurancementioning

confidence: 99%

“…The use of such an undefined work force inherently raises quality questions for crowdsourcing in general [267]- [269] as well as Crowdsourced Software Engineering [4], [5], [23], [105].…”

Section: Quality Assurancementioning

confidence: 99%

See 3 more Smart Citations

A survey of the use of crowdsourcing in software engineering

Mao

Capra

Harman

et al. 2017

Journal of Systems and Software

309

231

View full text Add to dashboard Cite

Crowdsourcing can be used to support software engineering activities and research into these activities. In this paper we provide a comprehensive survey of the use of crowdsourcing to support software engineering activities (Crowdsourced Software Engineering), seeking to cover all literature on this topic. We describe the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We also expose trends, issues and opportunities for Crowdsourced Software Engineering.Please cite as: Ke Mao, Licia Capra, Mark Harman and Yue Jia. A Survey of the Use of Crowdsourcing in SoftwareEngineering.

show abstract

Section: Case Studiesmentioning

confidence: 99%

Section: Case Studiesmentioning

confidence: 99%

Section: Quality Assurancementioning

confidence: 99%

“…The use of such an undefined work force inherently raises quality questions for crowdsourcing in general [267]- [269] as well as Crowdsourced Software Engineering [4], [5], [23], [105].…”

Section: Quality Assurancementioning

confidence: 99%

See 2 more Smart Citations

A survey of the use of crowdsourcing in software engineering

Mao

Capra

Harman

et al. 2017

Journal of Systems and Software

309

231

View full text Add to dashboard Cite

show abstract

“…From the literature [11], we know that there are two main factors that affect the success (in terms of on-time delivery and software quality) of a crowdcoding project: (1) the number of registered developers, and (2) the size of the software to be developed. The first factor refers to the chance of getting a better quality solution if the number of workers increases.…”

Section: Introductionmentioning

confidence: 99%

Correctness assessment of a crowdcoding project in a computer programming introductory course

Ferrán

Beghelli

Huerta-Cánepa

et al. 2017

Comp Applic In Engineering

View full text Add to dashboard Cite

Crowdcoding is a programming model that outsources a software project implementation to the crowd. As educators, we think that crowdcoding could be leveraged as part of the learning path of engineering students from a computer programming introductory course to solve local community problems. The benefits are twofold: on the one hand the students practice the concepts learned in class and, on the other hand, they participate in real-life problems. Nevertheless, several challenges arise when developing a crowdcoding platform, the first one being how to check the correctness of student's code without giving an extra burden to the professors in the course. To overcome this issue, we propose a novel system that does not resort to expert review; neither requires knowing the right answers beforehand. The proposed scheme automatically clusters the student's codes based solely on the output they produce. Our initial results show that the largest cluster contains the same codes selected as correct by the automated and human testing, as long as some conditions apply. K E Y W O R D Sautomated code correctness assessment, computer programming education, crowdcoding, software crowdsourcing | INTRODUCTIONCrowdsourcing gathers a vast and diverse group of people distributed around the world working towards a common goal resorting to the "wisdom of crowds." The assumption under this new collaborative work model is that the crowd can perform a task with higher speed and quality than any expert. In the crowdsourcing language, a requester is an organization or person that submits a task to a crowdsourcing platform. The workers are the people willing to carry out the task and submit their contributions to the same platform. Wikipedia is a wellknown example of crowdsourcing, but nowadays many organizations are using this collaborative work model to improve their business in diverse areas such as product design, drug development, mining, and software development [4,15].Crowdcoding (also known as Software Crowdsourcing) builds on the idea of crowdsourcing for software development projects. In a crowdcoding-based development, the original project (either as a whole or divided into smaller/simpler tasks) is presented as a coding challenge to an online community of software engineers. A group of reviewers ranks the submissions, and the best ones are selected to build the original project. One of the advantages attributed to crowdcoding is the lowering of defect rate thanks to the various development capacity provided by different programmers [7].We argue that crowdcoding can be successfully used in educative environments as a way to solve computer-related challenges from local communities. That is, resorting to students (workers) of computer programming courses to build 162 |

show abstract

Factors influencing sustainability aspects in crowdsourced software development: A systematic literature review

Haider,

Ilyas,

Khalid

et al. 2023

J Software Evolu Process

View full text Add to dashboard Cite

Crowdsource software development has become more and more popular in recent years in the software industry. Crowdsourcing is an open‐call technique for outsourcing tasks to a broad and undefined crowd. Crowdsourcing provides numerous advantages including reduced costs, fast project completion, talent identification, diversity of solutions, top‐quality, and access to problem‐solving creativity. Despite of the benefits gained from crowdsourcing, there are numerous issues like lack of experienced workers, lack of confidentiality, copyright issues, software sustainability, and so forth. There is also less focus on the long‐term sustainability of software development because of new ideas emerging in crowdsourcing software development. Furthermore, in literature, lack of guidelines towards sustainable software crowdsourcing is highlighted as one of the limitations in the software standards. This study aims to identify the factors that influence sustainability aspects in crowdsourced software development. We have conducted a systematic literature review for identification of these factors. In this paper, we present findings of the systematic literature review in the form of a list of 11 factors extracted from a sample of 45 finally selected papers. Among these factors, six of the factors are ranked as critical factors. These critical factors are “Lack of coding standard in documentation,” “Use of popular programming tools,” “Crowd Lack of knowledge and awareness about sustainability,” “Energy‐efficient coding,” “Lack of awareness about sustainable software engineering practices,” and “Lack of coordination/communication between client and crowd.”

show abstract

Analysis of the Key Factors for Software Quality in Crowdsourcing Development: An Empirical Study on TopCoder.com

Cited by 32 publications

References 11 publications

A survey of the use of crowdsourcing in software engineering

A survey of the use of crowdsourcing in software engineering

Correctness assessment of a crowdcoding project in a computer programming introductory course

Factors influencing sustainability aspects in crowdsourced software development: A systematic literature review

Contact Info

Product

Resources

About