Taming Coverage Criteria Heterogeneity with LTest

Abstract

Abstract-Automated white-box testing is a major issue in software engineering. In previous work, we introduced LTest, a generic and integrated toolkit for automated white-box testing of C programs. LTest supports a broad class of coverage criteria in a unified way (through the label specification mechanism) and covers most major parts of the testing process -including coverage measurement, test generation and detection of infeasible test objectives. However, the original version of LTest was unable to handle several major classes of coverage criteria, such as MCDC or dataflow criteria. Moreover, its practical applicability remained barely assessed.In this work, we present a significantly extended version of LTest that supports almost all existing testing criteria, including MCDC and some software security properties, through a native support of recently proposed hyperlabels. We also provide a more realistic view on the practical applicability of the extended tool, with experiments assessing its efficiency and scalability on realworld programs.

Specify and measure, cover and reveal: A unified framework for automated test generation

“…An extension of labels, named hyperlabels, is presented in Section 10.1. Hyperlabels [8,9] provide combination operators over labels, yielding a very expressive framework for the specification of test objectives. While standard labels are restricted to state-reachability constraints (the test datum must reach a specific state), hyperlabels can express test objectives defined over trace reachability (the test datum must follow a particular sequence of states) or even hyperreachability (constraints are here expressed over finite sets of traces, typically pairs).…”

“…Finally, all recent extensions [8,9,19,1] , applications [35] and industrial adoption efforts [33,34] are now synthetised in Section 10.…”

Time to clean your test objectives

“…This makes it possible to detect a significant number of redundant objectives while avoiding a quadratic penalty in computation time. The LClean tool is implemented on top of the Frama-C/LTest platform [34,40], which features strong conceptual and technical foundations (Section 3). We specifically extend the Frama-C module dedicated to proving code assertions to make the proposed solution scalable and robust.…”

“…In the context of this paper, we are mainly interested in the four following (open-source) plug-ins. LAnnotate, LUncov and LReplay are part of Frama-C/LTest [7,40]. LAnnotate annotates the program with labels according to the selected criterion.…”

Detection of Polluting Test Objectives for Dataflow Criteria

“…This short paper evaluates three approaches to detecting polluting test objectives for dataflow criteria, involving dataflow analysis, (abstract interpretation based) value analysis, and weakest precondition calculus. We implement these approaches inside the LTest open-source testing toolset [12]. We evaluate and compare them by some initial experiments and analyze their detection capacities and limitations.…”