Studying Software Engineers: Data Collection Techniques for Software Field Studies

Lethbridge, Timothy C.; Sim, Susan Elliott; Singer, Janice

doi:10.1007/s10664-005-1290-x

Cited by 393 publications

(265 citation statements)

References 52 publications

Supporting

Mentioning

244

Contrasting

Unclassified

Order By: Relevance

“…action research, case study, ethnography, experiment [11]) and the data collection approach [6], or 'instrumentation' (e.g. interviews, questionnaire) [15].…”

Section: Methodsmentioning

confidence: 99%

“…It can be argued that reporting the methods used for gathering and analysing data is important for evaluating the significance of the presented results of a study. Each data collection approach can help achieve certain research objectives and has its limitations [6], so a clear statement of the used data collection approach helps readers to understand the implications of the applied data collection approach.…”

Section: Data Collectionmentioning

confidence: 99%

“…Our results also show that there is a need for improving the quality of reporting empirical studies of OSS. To help improve the state of practice, we have made a few recommendations based on the standards and guidelines proposed to conduct empirical research in software engineering [4][5][6].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reporting Empirical Research in Open Source Software: The State of Practice

Stol

Babar

2009

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. Background: The number of reported empirical studies of Open Source Software (OSS) has continuously been increasing. However, there has been no effort to systematically review the state of the practice of reporting empirical studies of OSS with respect to the recommended standards of performing and reporting empirical studies in software engineering. It is important to understand, how to report empirical studies of OSS in order to make them useful for practitioners and researchers.Research aim: The aim of our research is to gain insights in the state of the practice of reporting empirical studies of OSS in order to identify the gaps to be filled for improving the quality of evidence being provided for OSS. Method:To that end, we decided to systematically review the empirical studies of OSS. A total of 63 papers reporting empirical studies were selected from the four editions of the Proceedings of the International Conference on Open Source Systems. The data were extracted and synthesised from the selected papers for analysis. Results and conclusions:We have found that the quality of the reported OSSrelated empirical studies needs to be significantly improved. Based on the results of our systematic review and general principles of reporting good empirical research, we present a set of guidelines for reporting OSS-related empirical studies. The suggested guidelines are expected to help the research community to improve the quality of reported studies.

show abstract

“…action research, case study, ethnography, experiment [11]) and the data collection approach [6], or 'instrumentation' (e.g. interviews, questionnaire) [15].…”

Section: Methodsmentioning

confidence: 99%

Section: Data Collectionmentioning

confidence: 99%

See 1 more Smart Citation

Reporting Empirical Research in Open Source Software: The State of Practice

Stol

Babar

2009

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

“…In the following subsections, we use the recommendations and guidance from several authors, to communicate our study design (Pinsonneault and Kraemer, 1993;Kitchenham et al, 1995;Basili et al, 1999;Lethbridge et al, 2005;Runeson and Host, 2009). Each subsection describes a particular perspective of our study design: (1) its Motivation; (2) its Plan; and (3) its Protocol.…”

Section: Methodsmentioning

confidence: 99%

A multiple comparative study of test-with development product changes and their effects on team speed and product quality

Bannerman

Martin

2010

Empir Software Eng

View full text Add to dashboard Cite

Researchers have typically studied the effects of Test-First Development (TFD), compared to Test-Last Development (TLD), across groups or projects, and for relatively short durations. We define Test-With Development (TWD) as more general than the fine-grained step of TFD, but also in contrast to the large-grained phase of TLD. With this definition, we performed a multiple comparative study to explore and describe TWD product changes, and the effects of those changes on two attributes related to team speed and two attributes related to product quality, within six long-term open-source projects. Our results indicate that when developers exercised some of their changes with automated tests, on average they made significantly larger changes over time while significantly reducing their product's complexity. And, when they exercised all of their changes with tests, on average they made significantly smaller changes over time. We interpret these results to indicate that practicing TWD supports faster simplification of a product. Therefore, we conclude that teams that need to simplify their product can benefit from practicing TWD.

show abstract

“…Nevertheless, obtaining a random sample is almost unachievable in software engineering studies because our community lacks good demographic information about populations of interest [17]. The respondents were chosen by us based on their background and experience with software architecture.…”

Section: Threats To Validitymentioning

confidence: 99%