Data quality in empirical software engineering

Bosu, Michael Franklin; MacDonell, Stephen G.

doi:10.1145/2460999.2461024

Cited by 22 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…c 2015 ACM 1049-331X/2015/05-ART20 $15.00 DOI: http://dx.doi.org/10.1145/2738037 compare the performance of the resulting models with other methods, the results are highly variable [Myrtveit and Stensrud 2012;Shepperd and Kadoda 2001], sometimes contradictory [Menzies and Shepperd 2012], and often difficult to interpret and validate [Kitchenham and Mendes 2009]. This is due in part to a number of factors: the use of datasets that are difficult to obtain or are proprietary and therefore not publically available [Mair et al 2005]; bias in dataset selection [Kitchenham and Mendes 2009]; initial data preparation methods that remove outlier examples and/or explanatory variables, making comparison with previous work difficult [ Bosu and MacDonell 2013]; the sampling methods used when assessing model quality; the presented error measurements [Myrtveit and Stensrud 2012;Myrtveit et al 2005]; bias in modeller expertise and parameter tuning [Song et al 2013]; and a lack of meaningful comparative models and statistics. Although these issues arise in many fields where inductive model building is performed, and have been recognised as relevant to software effort estimation, they appear both common and enduring in this domain.…”

Section: Introductionmentioning

confidence: 99%

A Baseline Model for Software Effort Estimation

Whigham

Owen

MacDonell

2015

ACM Trans. Softw. Eng. Methodol.

Self Cite

112

View full text Add to dashboard Cite

Software effort estimation (SEE) is a core activity in all software processes and development lifecycles. A range of increasingly complex methods has been considered in the past 30 years for the prediction of effort, often with mixed and contradictory results. The comparative assessment of effort prediction methods has therefore become a common approach when considering how best to predict effort over a range of project types. Unfortunately, these assessments use a variety of sampling methods and error measurements, making comparison with other work difficult. This article proposes an automatically transformed linear model (ATLM) as a suitable baseline model for comparison against SEE methods. ATLM is simple yet performs well over a range of different project types. In addition, ATLM may be used with mixed numeric and categorical data and requires no parameter tuning. It is also deterministic, meaning that results obtained are amenable to replication. These and other arguments for using ATLM as a baseline model are presented, and a reference implementation described and made available. We suggest that ATLM should be used as a baseline of effort prediction quality for all future model comparisons in SEE. . 2015. A baseline model for software effort estimation.

show abstract

Section: Introductionmentioning

confidence: 99%

A Baseline Model for Software Effort Estimation

Whigham

Owen

MacDonell

2015

ACM Trans. Softw. Eng. Methodol.

Self Cite

112

View full text Add to dashboard Cite

show abstract

“…While empirical software engineering researchers have paid attention to data quality, systematic reviews of data quality by Liebchen et al [14] [15] [20] and Bosu [1] [2] indicated that few researchers have explored data quality, and more research needs to be done in this area.…”

Section: Data Qualitymentioning

confidence: 99%

Initial evaluation of data quality in a TSP software engineering project data repository

Shirai

Nichols

Kasunic

2014

Proceedings of the 2014 International Conference on Software and System Process

View full text Add to dashboard Cite

To meet critical business challenges, software development teams need data to effectively manage product quality, cost, and schedule. The Team Software Process SM (TSP SM ) provides a framework that teams use to collect software process data in real time, using a defined disciplined process. This data holds promise for use in software engineering research. We combined data from 109 industrial projects into a database to support performance benchmarking and model development. But is the data of sufficient quality to draw conclusions? We applied various tests and techniques to identify data anomalies that affect the quality of the data in several dimensions. In this paper, we report some initial results of our analysis, describing the amount and the rates of identified anomalies and suspect data, including incorrectness, inconsistency, and credibility. To illustrate the types of data available for analysis, we provide three examples. The preliminary results of this empirical study suggest that some aspects of the data quality are good and the data are generally credible, but size data are often missing.

show abstract

“…The importance of the quality of data used by empirical studies has been acknowledged and assessed in the last years [13,14,15,16,17,18], [30,31], mostly due to the impact that it may have on the decisions taken. Some papers explicitly emphasize the importance of DQ in empirical software engineering datasets, as data imperfections can have unwanted impact on the data analysis and might lead to false conclusions [14], [16], [25].…”

Section: Related Workmentioning

confidence: 99%

“…Three literature reviews were carried out in this particular topic, showing interest and concern about how researchers are dealing with DQ problems [13,14], [30]. They all conclude that empirical software engineering community should pay more attention to this issue, which has long been neglected according to the results…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Applying a Data Quality Model to Experiments in Software Engineering

Valverde

Vallespir

Marotta

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Data collection and analysis are key artifacts in any software engineering experiment. However, these data might contain errors. We propose a Data Quality model specific to data obtained from software engineering experiments, which provides a framework for analyzing and improving these data. We apply the model to two controlled experiments, which results in the discovery of data quality problems that need to be addressed. We conclude that data quality issues have to be considered before obtaining the experimental results.

show abstract

Data quality in empirical software engineering

Abstract: Context

Cited by 22 publications

References 20 publications

A Baseline Model for Software Effort Estimation

A Baseline Model for Software Effort Estimation

Initial evaluation of data quality in a TSP software engineering project data repository

Applying a Data Quality Model to Experiments in Software Engineering

Contact Info

Product

Resources

About