Comparing the performance of metaheuristics for the analysis of multi-stakeholder tradeoffs in requirements optimisation

2012 First International Workshop on Realizing AI Synergies in Software Engineering (RAISE)

2012

Self Cite

Abstract-There has been a recent surge in interest in the application of Artificial Intelligence (AI) techniques to Software Engineering (SE) problems. The work is typified by recent advances in Search Based Software Engineering, but also by long established work in Probabilistic reasoning and machine learning for Software Engineering. This paper explores some of the relationships between these strands of closely related work, arguing that they have much in common and sets out some future challenges in the area of AI for SE.

Section: Giving Insight To Software Engineersmentioning

confidence: 99%

The role of Artificial Intelligence in Software Engineering

2012 First International Workshop on Realizing AI Synergies in Software Engineering (RAISE)

2012

Self Cite

“…Our experimental studies reveal that, with the aid of NSGDP, the decision maker can avoid information loss (without which we show that, for an example real world requirements problem, he or she will lose up to 99.95 percent of the optimal solutions and will make up to 36.48 percent inexact requirement selection decisions as a result). Furthermore, for the RALIC study, the execution time of NSGDP is better than NSGA-II which is currently the best performing NRP solver according to a recent empirical study [19]. However, this is only a result from a single set of requirements, and so we cannot claim overall superior performance, based on this single study.…”

Section: Introductionmentioning

confidence: 85%

The Value of Exact Analysis in Requirements Selection

IIEEE Trans. Software Eng.

et al. 2017

Self Cite

Uncertainty is characterised by incomplete understanding. It is inevitable in the early phase of requirements engineering, and can lead to unsound requirement decisions. Inappropriate requirement choices may result in products that fail to satisfy stakeholders' needs, and might cause loss of revenue. To overcome uncertainty, requirements engineering decision support needs uncertainty management. In this research, we develop a decision support framework METRO for the Next Release Problem (NRP) to manage algorithmic uncertainty and requirements uncertainty. An exact NRP solver (NSGDP) lies at the heart of METRO. NSGDP's exactness eliminates interference caused by approximate existing NRP solvers. We apply NSGDP to three NRP instances, derived from a real world NRP instance, RALIC, and compare with NSGA-II, a widely-used approximate (inexact) technique. We find the randomness of NSGA-II results in decision makers missing up to 99.95 percent of the optimal solutions and obtaining up to 36.48 percent inexact requirement selection decisions. The chance of getting an inexact decision using existing approximate approaches is negatively correlated with the implementation cost of a requirement (Spearman r up to À0:72). Compared to the inexact existing approach, NSGDP saves 15.21 percent lost revenue, on average, for the RALIC dataset.

“…Saliu and Ruhe [84] used a Pareto optimal approach to explore the balance of concerns between requirements at different levels of abstraction, while Zhang et al, showed how SBSE could be used to explore the tradeoff among the different stakeholders in requirements assignment problems [112].…”

Section: Sbse As Decision Supportmentioning

confidence: 99%

Search Based Software Engineering: Techniques, Taxonomy, Tutorial

Empirical Software Engineering and Verification

McMinn

Souza

et al. 2012

Self Cite

228

257

Abstract. The aim of Search Based Software Engineering (SBSE) research is to move software engineering problems from human-based search to machine-based search, using a variety of techniques from the metaheuristic search, operations research and evolutionary computation paradigms. The idea is to exploit humans' creativity and machines' tenacity and reliability, rather than requiring humans to perform the more tedious, error prone and thereby costly aspects of the engineering process. SBSE can also provide insights and decision support. This tutorial will present the reader with a step-by-step guide to the application of SBSE techniques to Software Engineering. It assumes neither previous knowledge nor experience with Search Based Optimisation. The intention is that the tutorial will cover sufficient material to allow the reader to become productive in successfully applying search based optimisation to a chosen Software Engineering problem of interest. IntroductionSearch Based Software Engineering (SBSE) is the name given to a body of work in which Search Based Optimisation is applied to Software Engineering. This approach to Software Engineering has proved to be very successful and generic. It has been a subfield of software engineering for ten years [45], the past five of which have been characterised by an explosion of interest and activity [48]. New application areas within Software Engineering continue to emerge and a body of empirical evidence has now accrued that demonstrates that the search based approach is definitely here to stay. SBSE seeks to reformulate Software Engineering problems as 'search problems' [45,48]. This is not to be confused with textual or hypertextual searching. Rather, for Search Based Software Engineering, a search problem is one in which optimal or near optimal solutions are sought in a search space of candidate solutions, guided by a fitness function that distinguishes between better and worse solutions. The term SBSE was coined by Harman and Jones [45] in 2001, which was the first paper to advocate Search Based Optimisation as a general approach to Software Engineering, though there were other authors who had previously applied search based optimisation to aspects of Software Engineering.SBSE has been applied to many fields within the general area of Software Engineering, some of which are already sufficiently mature to warrant their own surveys. For example, there are surveys and overviews, covering SBSE for requirements [111], design [78] and testing [3,4,65], as well as general surveys of the whole field of SBSE [21,36,48]. This paper does not seek to duplicate these surveys, though some material is repeated from them (with permission), where it is relevant and appropriate. Rather, this paper aims to provide those unfamiliar with SBSE with a tutorial and practical guide. The aim is that, having read this paper, the reader will be able to begin to develop SBSE solutions to a chosen software engineering problem and will be able to collect and analyse the results of the applicati...