Automatic non-functional testing of code generators families

Abstract

The intensive use of generative programming techniques provides an elegant engineering solution to deal with the heterogeneity of platforms and technological stacks. The use of domain-specific languages for example, leads to the creation of numerous code generators that automatically translate highlevel system specifications into multi-target executable code. Producing correct and efficient code generator is complex and error-prone. Although software designers provide generally high-level test suites to verify the functional outcome of generated code, it remains challenging and tedious to verify the behavior of produced code in terms of non-functional properties. This paper describes a practical approach based on a runtime monitoring infrastructure to automatically check the potential inefficient code generators. This infrastructure, based on system containers as execution platforms, allows code-generator developers to evaluate the generated code performance. We evaluate our approach by analyzing the performance of Haxe, a popular high-level programming language that involves a set of cross-platform code generators. Experimental results show that our approach is able to detect some performance inconsistencies that reveal real issues in Haxe code generators.

VaryMinions: leveraging RNNs to identify variants in variability-intensive systems’ logs

“…This problem is not restricted to business processes and naturally extends to variability-intensive systems, which change their behaviour in response to the (de)activation of some options. Examples of variability-intensive systems include Software Product Lines (SPLs) ; Apel et al (2013), operating systems kernels She et al (2010); Mortara and Collet (2021), code generators Boussaa et al (2016); , or web-based frameworks Halin et al (2019); Sánchez et al (2017). Validating these systems is difficult because enumerating all variants, whose number can grow exponentially with the number of options, is generally infeasible Halin et al (2019).…”