Resource Estimation in Software Engineering

Abstract

This article presents a comprehensive overview of the state of the art in software resource estimation. We describe common estimation methods and also provide an evaluation framework to systematically compare and assess alternative estimation methods. Although we have tried to be as precise and objective as possible, it is inevitable that such a comparison exercise be somewhat subjective. We however, provide as much information as possible, so that the reader can form his or her own opinion on the methods to employ. We also discuss the applications of such estimation methods and provide practical guidelines. Understanding this article does not require any specific expertise in resource estimation or quantitative modeling. However, certain method descriptions are brief and the level of understanding that can be expected from such a text depends, to a certain extent, on the reader's knowledge. Our objective is to provide the reader with a comprehension of existing software resource estimation methods as well as with the tools to reason about estimation methods and how they relate to the reader's problems. The second section (on resource estimation) briefly describes the problems at hand, the history, and the current status of resource estimation in software engineering research and practice. The third section (on overview of estimation models) provides a comprehensive, although certainly not complete, overview of resource estimation methods. Project sizing, an important issue related to resource estimation, is then discussed in the fourth section (on sizing projects). The fifth section (on framework for com parison and evaluation) defines an evaluation framework that allows us to make systematic and justified comparisons in the sixth section (on evaluation and comparison effort estimation methods). The seventh section (on considerations influencing choice of estimating method) provides guidelines regarding the selection of appropriate estimation methods, in a given context. The eighth section (on typical applications) describes typical scenarios for using resource estimation methods, thereby relating them to software management practice. The ninth section (on future directions) attempts to define important research and practice directions, requiring the collaboration of academia and industry. This article then concludes by summarizing the main points made throughout the article.

A simulation study of the model evaluation criterion mmre

“…There are a number of empirical studies including studies on generic model-based methods [12] [29], Stepwise ANOVA (Analysis of Variance) [35], OLS (Ordinary Least Squares) regression (more than 30 studies, see [10] for an account), Robust regression [23] [48] and, finally, artificial neural network-based models [47] [50]. (We have adopted the classification scheme proposed in the Encyclopedia of Software Engineering [10] except possibly for neural networks. )…”

“…The most widely used evaluation criterion to assess the performance of software prediction models is the Mean Magnitude of Relative Error (MMRE) [10]. This is usually computed following standard evaluation processes such as cross-validation [6].…”

LSA-X: Exploiting Productivity Factors in Linear Size Adaptation for Analogy-Based Software Effort Estimation

“…The normalization for a feature vector i was produced by subtracting the minimum value of the vector from all the attribute values, then dividing result by value of the maximum minus the minimum values of i). This procedure is widely used in effort estimation studies such as in [4], [8], [37], [39], [40].…”

Estimating Software Development Effort Using Tabu Search

“…The validation process was performed using a hold-out validation on Desharnais and a 3-fold cross validation for NASA, since it is quite small (Briand and Wieczorek, 2002). In particular, we randomly split the Desharnais dataset into a training set of 59 observations and a test set of 18 observations.…”

“…These methods, named data-driven, exploit data from past projects, consisting of both factor values that are related to effort and the actual effort to develop the projects, in order to estimate the effort for a new project under development (Shepperd and Schofield, 2000). In this class, we can find some widely used techniques, such as Linear and Stepwise Regression, Classification and Regression Tree, and Case-Based Reasoning (Briand and Wieczorek, 2002).…”