The business case for automated software engineering

Abstract

Adoption of advanced automated SE (ASE) tools would be more favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the "local tuning" problem. Normally, predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable.This paper shows that assessing the relative merits of different SE methods need not require precise local tunings. STAR1 is a simulated annealer plus a Bayesian post-processor that explores the space of possible local tunings within software prediction models. STAR1 ranks project decisions by their effects on effort and defects and threats. In experiments with NASA systems, STAR1 found one project where ASE were essential for minimizing effort/ defect/ threats; and another project were ASE tools were merely optional.

On the Relative Merits of Software Reuse

“…year at ICSP'08 and elsewhere [6][7][8]) we showed that this methods can yield estimates close to those seem using traditional methods, without requiring a time consuming data collection exercise.…”

How to avoid drastic software process change (using stochastic stability)

“…We have conducted experiments with other approaches that allow intermediate values. On comparison with the simulated annealing method used in a prior publications [16], we found that seesawing between {Low, High} values was adequate for our purposes.…”

“…This is due to data not being collected or the business sensitivity associated with the data as well as differences in how the metrics are defined, collected and archived. For example, after two years we were only able to add 7 records to a NASA-wide software cost metrics repository [16]. Having failed to generate precise tunings, we developed SEESAW to discover what stable conclusions we could find within the space of possible tunings.…”

“…The median value of these 100 simulations is the score for the current project settings. As SEESAW executes, the ranges in Options are removed and replaced by single values (lines [16][17], thus constraining the space of possible simulations.…”

“…requirements engineering [12]) but most often in the field of software testing [1]. Harmon's writing inspired us try simulated annealing to search the what-ifs in untuned COCOMO models [16]. However, we found that SEESAW ran much faster and produced results with far less variance than simulated annealing.…”