White-Box Performance-Influence Models: A Profiling and Learning Approach

Weber, Max; Apel, Sven; Siegmund, Norbert

doi:10.1109/icse43902.2021.00099

Cited by 16 publications

(23 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…White-box approaches [88], [89], [90] have been proposed to inspect the implementation of a configurable system in order to guide the performance analysis. Static dataflow analysis or profiling are typically used to help understanding options and their interactions in a fine-grained way.…”

Section: Related Workmentioning

confidence: 99%

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Martin

Acher

Lesoil

et al. 2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

With large scale and complex configurable systems, it is hard for users to choose the right combination of options (i.e., configurations) in order to obtain the wanted trade-off between functionality and performance goals such as speed or size. Machine learning can help in relating these goals to the configurable system options, and thus, predict the effect of options on the outcome, typically after a costly training step. However, many configurable systems evolve at such a rapid pace that it is impractical to retrain a new model from scratch for each new version. In this paper, we propose a new method to enable transfer learning of binary size predictions among versions of the same configurable system. Taking the extreme case of the Linux kernel with its ≈ 14, 500 configuration options, we first investigate how binary size predictions of kernel size degrade over successive versions. We show that the direct reuse of an accurate prediction model from 2017 quickly becomes inaccurate when Linux evolves, up to a 32% mean error by August 2020. We thus propose a new approach for transfer evolution-aware model shifting (TEAMS). It leverages the structure of a configurable system to transfer an initial predictive model towards its future versions with a minimal amount of extra processing for each version. We show that TEAMS vastly outperforms state of the art approaches over the 3 years history of Linux kernels, from 4.13 to 5.8.

show abstract

Section: Related Workmentioning

confidence: 99%

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Martin

Acher

Lesoil

et al. 2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…In fact, our generic definition supports a multitude of effect properties for which the only assumption is that there is an effective method to determine all configurations in which the effect property holds. Such methods also include variability-aware white-box analyses [87,90], where the source code or operational behavior of system variants is accessible, as well as black-box analyses relying on testing or sampling [43,52]. In the following paragraphs, we exemplify how to obtain effect sets from analysis results.…”

Section: Effect Properties and Setsmentioning

confidence: 99%

“…For the analysis of configurable software systems, many approaches have been presented in the last two decades [69,84]. Nowadays, there is broad tool support for variability-aware testing and sampling [10,43,51,52,79], static analysis [12,71,86,87,90], and model checking [5,19,23,25,67,85].…”

Section: Related Workmentioning

confidence: 99%

“…Other black-box approaches rely on regression trees [43], Fourier learning [95], or probabilistic programming [27]. Related white-box approaches use insights of local measurements and taint analysis information [87] or profiling information [90]. Variability-aware probabilistic model checking pursues a whitebox analysis approach on state-based operational models where effect properties are specified in quantitative variants of temporal logic [31,83].…”

Section: Effect Properties and Setsmentioning

confidence: 99%

See 1 more Smart Citation

Causality in Configurable Software Systems

Dubslaff,

Weis,

Baier

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Detecting and understanding reasons for defects and inadvertent behavior in software is challenging due to their increasing complexity. In configurable software systems, the combinatorics that arises from the multitude of features a user might select from adds a further layer of complexity. We introduce the notion of feature causality, which is based on counterfactual reasoning and inspired by the seminal definition of actual causality by Halpern and Pearl. Feature causality operates at the level of system configurations and is capable of identifying features and their interactions that are the reason for emerging functional and non-functional properties. We present various methods to explicate these reasons, in particular well-established notions of responsibility and blame that we extend to the feature-oriented setting. Establishing a close connection of feature causality to prime implicants, we provide algorithms to effectively compute feature causes and causal explications. By means of an evaluation on a wide range of configurable software systems, including community benchmarks and real-world systems, we demonstrate the feasibility of our approach: We illustrate how our notion of causality facilitates to identify root causes, estimate the effects of features, and detect feature interactions.

show abstract

“…When performance issues such as in our example occur, there are numerous techniques that developers could use to determine whether there is a performance bug or the system was misconfigured. In addition to off-the-shelf profilers [15,53,74], developers could use more targeted profiling techniques [4,10,13,83,84], visualize performance behavior [2,6,12,21,62,70], search for inefficient coding patterns [7,46,54,56,68], use information-flow analyses [43,45,48,69,79,81,88], or model the performance of the systems in terms of its options and interactions [24,38,64,71,72,76]. Likewise, developers could use established program debugging techniques, such as delta debugging [85], program slicing [3,39,77], and statistical debugging [5,67] for some part of the debugging process.…”

Section: Introductionmentioning

confidence: 99%

On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support

Vélez¹,

Jamshidi²,

Siegmund³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Determining whether a configurable software system has a performance bug or it was misconfigured is often challenging. While there are numerous debugging techniques that can support developers in this task, there is limited empirical evidence of how useful the techniques are to address the actual needs that developers have when debugging the performance of configurable software systems; most techniques are often evaluated in terms of technical accuracy instead of their usability. In this paper, we take a human-centered approach to identify, design, implement, and evaluate a solution to support developers in the process of debugging the performance of configurable software systems. We first conduct an exploratory study with 19 developers to identify the information needs that developers have during this process. Subsequently, we design and implement a tailored tool, adapting techniques from prior work, to support those needs. Two user studies, with a total of 20 developers, validate and confirm that the information that we provide helps developers debug the performance of configurable software systems.

show abstract

White-Box Performance-Influence Models: A Profiling and Learning Approach

Cited by 16 publications

References 50 publications

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Transfer Learning Across Variants and Versions: The Case of Linux Kernel Size

Causality in Configurable Software Systems

On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support

Contact Info

Product

Resources

About