Experimentation in Software Engineering

Wohlin, Claes; Runeson, Per; Höst, Martin; Ohlsson, Magnus C.; Regnell, Björn; Wesslén, Anders

doi:10.1007/978-1-4615-4625-2

Cited by 2,403 publications

(1,980 citation statements)

References 1 publication

Supporting

Mentioning

1,738

Contrasting

Unclassified

238

Order By: Relevance

“…In all but five studies, the authors did not state their relationship to the studied subjects (reflexivity). Only 12 papers reported the limitations of the studies, which is an important aspect of reporting empirical research [4,5,11]. Of the 63 studies, a third scored at least 10 out of 12 quality assessment points.…”

Section: Results Of Quality Assessmentmentioning

confidence: 99%

See 1 more Smart Citation

Reporting Empirical Research in Open Source Software: The State of Practice

Stol

Babar

2009

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. Background: The number of reported empirical studies of Open Source Software (OSS) has continuously been increasing. However, there has been no effort to systematically review the state of the practice of reporting empirical studies of OSS with respect to the recommended standards of performing and reporting empirical studies in software engineering. It is important to understand, how to report empirical studies of OSS in order to make them useful for practitioners and researchers.Research aim: The aim of our research is to gain insights in the state of the practice of reporting empirical studies of OSS in order to identify the gaps to be filled for improving the quality of evidence being provided for OSS. Method:To that end, we decided to systematically review the empirical studies of OSS. A total of 63 papers reporting empirical studies were selected from the four editions of the Proceedings of the International Conference on Open Source Systems. The data were extracted and synthesised from the selected papers for analysis. Results and conclusions:We have found that the quality of the reported OSSrelated empirical studies needs to be significantly improved. Based on the results of our systematic review and general principles of reporting good empirical research, we present a set of guidelines for reporting OSS-related empirical studies. The suggested guidelines are expected to help the research community to improve the quality of reported studies.

show abstract

Section: Results Of Quality Assessmentmentioning

confidence: 99%

“…action research, case study, ethnography, experiment [11]) and the data collection approach [6], or 'instrumentation' (e.g. interviews, questionnaire) [15].…”

Section: Methodsmentioning

confidence: 99%

Reporting Empirical Research in Open Source Software: The State of Practice

Stol

Babar

2009

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

“…The three strategies, 1) current experience based (Exp), 2) the proposed strategy [5] (Faz) and 3) the extended version of the proposed strategy (ExtFaz) were compared through a quasi-experiment [21]. Data were collected from six recent consecutive regression test sessions of six consecutive software versions, and the effect of the prioritization of execution order and selection with the tool were analyzed and compared with the actual outcome of respective session.…”

Section: Comparative Evaluation Of Strategies (Exp Faz Extfaz)mentioning

confidence: 99%

Improving Regression Testing Transparency and Efficiency with History-Based Prioritization -- An Industrial Case Study

Engström

Runeson

Ljung

2011

2011 Fourth IEEE International Conference on Software Testing, Verification and Validation

Self Cite

View full text Add to dashboard Cite

“…We analyse the validity threats for this study following common guidelines for empirical studies (Yin, 2003;Wohlin et al, 2012).…”

Section: Threats To Validitymentioning

confidence: 99%

How are functionally similar code clones syntactically different? An empirical study and a benchmark

Wagner

Abdulkhaleq

Bogicevic

et al. 2016

PeerJ Computer Science

View full text Add to dashboard Cite

Background. Today, redundancy in source code, so-called ''clones'' caused by copy &paste can be found reliably using clone detection tools. Redundancy can arise also independently, however, not caused by copy&paste. At present, it is not clear how only functionally similar clones (FSC) differ from clones created by copy&paste. Our aim is to understand and categorise the syntactical differences in FSCs that distinguish them from copy&paste clones in a way that helps clone detection research. Methods. We conducted an experiment using known functionally similar programs in Java and C from coding contests. We analysed syntactic similarity with traditional detection tools and explored whether concolic clone detection can go beyond syntax. We ran all tools on 2,800 programs and manually categorised the differences in a random sample of 70 program pairs. Results. We found no FSCs where complete files were syntactically similar. We could detect a syntactic similarity in a part of the files in <16% of the program pairs. Concolic detection found 1 of the FSCs. The differences between program pairs were in the categories algorithm, data structure, OO design, I/O and libraries. We selected 58 pairs for an openly accessible benchmark representing these categories. Discussion. The majority of differences between functionally similar clones are beyond the capabilities of current clone detection approaches. Yet, our benchmark can help to drive further clone detection research.

show abstract

Experimentation in Software Engineering

Cited by 2,403 publications

References 1 publication

Reporting Empirical Research in Open Source Software: The State of Practice

Reporting Empirical Research in Open Source Software: The State of Practice

Improving Regression Testing Transparency and Efficiency with History-Based Prioritization -- An Industrial Case Study

How are functionally similar code clones syntactically different? An empirical study and a benchmark

Contact Info

Product

Resources

About