Learning software configuration spaces: A systematic literature review

Pereira, Juliana Alves; Martin, Hugo; Acher, Mathieu; Jézéquel, Jean-Marc; Botterweck, Goetz; Ventresque, Anthony

doi:10.1016/j.jss.2021.111044

Cited by 43 publications

(49 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is extremely complex both in terms of kernel variants, i.e., with its thousands of configuration options and in terms of code size, i.e., millions of Lines Of Code (LOC). Other existing cases considered in the literature (e.g., see [1]) vary from 5 to a few hundreds options and from 2,595 LOC to 305,191 LOC, while, e.g., Linux version 4.13 has > 12,797 options and > 13M LOC. Thus, those subject systems exhibit far less options which question whether proposed techniques can scale and obtain accurate results for a huge configuration space like the Linux one.…”

Section: Linux Kernelmentioning

confidence: 99%

“…This is where machine learning comes in handy. Recent approaches show the usefulness of machine learning techniques for learning performance models based on a sample set of configurations [1], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. Researchers have been experimenting with different techniques, e.g., decision trees, linear regression, neural networks, etc.…”

Section: Predicting Kernel Size With Machine Learningmentioning

confidence: 99%

“…Due to the wide distribution of Linux kernel binary sizes (from 7MB to around 2GB in our dataset), relying on some metrics such as Mean Absolute Error or Mean Squared Error can be biased, as an error of a few MB can be a big error for small kernels, and negligible for the biggest kernels. As numerous existing works (e.g., [1], [4], [7], [21], [42]), we rely on a variant of Mean Absolute Error, but normalized in a percentage error, known as Mean Absolute Percentage Error (MAPE), computed as follows…”

Section: Accuracy Measurementmentioning

confidence: 99%

“…Numerous works [1], [2], [3], [4], [5] have shown that quantifying the performance influence of each individual option is not meaningful in most cases. The performance influence of n options, all jointly activated in a configuration, is not easily deducible from the performance influence of each individual option [5].…”

Section: Introductionmentioning

confidence: 99%

“…configurations not measured before). Several works, thus, follow a "sampling, measuring, learning" process (e.g., [1], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]). Obtaining a low prediction error typically requires a large sample, that should be distributed as uniformly as possible over the configuration space.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Martin

Acher

Lesoil

et al. 2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

With large scale and complex configurable systems, it is hard for users to choose the right combination of options (i.e., configurations) in order to obtain the wanted trade-off between functionality and performance goals such as speed or size. Machine learning can help in relating these goals to the configurable system options, and thus, predict the effect of options on the outcome, typically after a costly training step. However, many configurable systems evolve at such a rapid pace that it is impractical to retrain a new model from scratch for each new version. In this paper, we propose a new method to enable transfer learning of binary size predictions among versions of the same configurable system. Taking the extreme case of the Linux kernel with its ≈ 14, 500 configuration options, we first investigate how binary size predictions of kernel size degrade over successive versions. We show that the direct reuse of an accurate prediction model from 2017 quickly becomes inaccurate when Linux evolves, up to a 32% mean error by August 2020. We thus propose a new approach for transfer evolution-aware model shifting (TEAMS). It leverages the structure of a configurable system to transfer an initial predictive model towards its future versions with a minimal amount of extra processing for each version. We show that TEAMS vastly outperforms state of the art approaches over the 3 years history of Linux kernels, from 4.13 to 5.8.

show abstract

Section: Linux Kernelmentioning

confidence: 99%

Section: Predicting Kernel Size With Machine Learningmentioning

confidence: 99%

Section: Accuracy Measurementmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Martin

Acher

Lesoil

et al. 2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

Scratching the Surface of ./configure: Learning the Effects of Compile-Time Options on Binary Size and Gadgets

Tërnava

Acher

Lesoil

et al. 2022

Reuse and Software Quality

Self Cite

View full text Add to dashboard Cite

Numerous software systems are configurable through compiletime options and the widely used ./configure. However, the combined effects of these options on binary's non-functional properties (size and attack surface) are often not documented, and or not well understood, even by experts. Our goal is to provide automated support for exploring and comprehending the configuration space (a.k.a., surface) of compile-time options using statistical learning techniques. In this paper, we perform an empirical study on four C-based configurable systems. We measure the variation of binary size and attack surface (by quantifying the number of code reuse gadgets) in over 400 compile-time configurations of a subject system. We then apply statistical learning techniques on top of our build infrastructure to identify how compile-time options relate to non-functional properties. Our results show that, by changing the default configuration, the system's binary size and gadgets vary greatly (roughly −79% to 244% and −77% to 30%, respectively). Then, we found out that identifying the most influential options can be accurately learned with a small training set, while their relative importance varies across size and attack surface for the same system. Practitioners can use our approach and artifacts to explore the effects of compile-time options in order to take informed decisions when configuring a system with ./configure.

show abstract

Family-Based Fingerprint Analysis: A Position Paper

Damasceno

Strüber

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning software configuration spaces: A systematic literature review

Cited by 43 publications

References 48 publications

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Scratching the Surface of ./configure: Learning the Effects of Compile-Time Options on Binary Size and Gadgets

Family-Based Fingerprint Analysis: A Position Paper

Contact Info

Product

Resources

About