Leveraging a Commercial Mutation Analysis Tool For Research

Hampton, Mark; Petithomme, S.

doi:10.1109/taic.part.2007.39

Cited by 25 publications

(12 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An effective optimization of this category is to first determine which test covers (i.e., reaches and executes) which mutant, and then for each mutant to only execute tests that cover the mutant. This is implemented in mutation tools such as Javalanche [27], Major [19], and Certitude [11]. Implementation is straightforward; for example, mutant schemata can easily be extended to collect the necessary coverage information [18].…”

Section: Related Workmentioning

confidence: 99%

“…A lossless example is to perform a prepass to measure mutation coverage -that is, determining which mutations 1 would be reached and executed by which test. Given the mutation coverage information, a test is not executed on a mutant if it does not cover it [11,21,28]. This paper takes the dynamic prepass approach a step further and observes the execution state during the initial Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient mutation analysis by propagating and partitioning infected execution states

Just

Ernst

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Mutation analysis evaluates a testing technique by measuring how well it detects seeded faults (mutants). Mutation analysis is hampered by inherent scalability problems -a test suite is executed for each of a large number of mutants. Despite numerous optimizations presented in the literature, this scalability issue remains, and this is one of the reasons why mutation analysis is hardly used in practice.Whereas most previous optimizations attempted to statically reduce the number of executions or their computational overhead, this paper exploits information available only at run time to further reduce the number of executions.First, state infection conditions can reveal -with a single test execution of the unmutated program -which mutants would lead to a different state, thus avoiding unnecessary test executions. Second, determining whether an infected execution state propagates can further reduce the number of executions. Mutants that are embedded in compound expressions may infect the state locally without affecting the outcome of the compound expression. Third, those mutants that do infect the state can be partitioned based on the resulting infected state -if two mutants lead to the same infected state, only one needs to be executed as the result of the other can be inferred.We have implemented these optimizations in the Major mutation framework and empirically evaluated them on 14 open source programs. The optimizations reduced the mutation analysis time by 40% on average.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Efficient mutation analysis by propagating and partitioning infected execution states

Just

Ernst

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…We make use of some of these operators in our work and enhance them for SystemC. Mutation testing has recently been applied to Verilog [Hampton and Petithomme 2007] and SystemC TLM 2.0 communication interfaces [Bombieri et al 2008[Bombieri et al , 2009. Our work differs from these in that we are concerned about all concurrency constructs in SystemC, rather than only TLM communication constructs.…”

Section: Related Workmentioning

confidence: 99%

Concurrency-oriented verification and coverage of system-level designs

Şen

2011

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

Correct concurrent System-on-Chips (SoCs) are very hard to design and reason about. In this work, we develop an automated framework complete with concurrency-oriented verification and coverage techniques for system-level designs. Our techniques are different from traditional simulation-based reliability techniques, since concurrency information is often lost in traditional techniques. We preserve concurrency information to obtain unique verification techniques that allow us to predict potential errors (formulated as transactionlevel assertions) from error-free simulations. In order to do this, we exploit the inherent concurrency in the designs to generate and analyze novel partial-order simulation traces. Additionally, to evaluate the confidence on verification results and the gauge progress of verification, we develop novel mutation testing based on concurrent coverage metrics. Mutation testing is a fault insertion-based simulation technique that has been successfully applied in software testing. We present a comprehensive list of mutation operators for SystemC, similar to behavioral fault models, and show the effectiveness of these operators by relating them to actual bug patterns. We have successfully applied our verification and coverage techniques on industrial systems and demonstrated that current verification test suites need to be improved for concurrent designs, and we have found errors in systems that were tested previously.

show abstract

“…We make use of some of these operators in our work and enhance them for SystemC. Mutation testing has recently been applied to Verilog [10] and SystemC TLM 2.0 communication interfaces [3], [4]. Our work differs from these works in that we are concerned about all concurrency constructs in SystemC rather than only communication constructs.…”

Section: Related Workmentioning

confidence: 99%

Coverage metrics for verification of concurrent SystemC designs using mutation testing

Şen

Abadir²

2010

2010 IEEE International High Level Design Validation and Test Workshop (HLDVT)

View full text Add to dashboard Cite

Abstract-Design verification has grown to dominate the cost of electronic system design; however, designs continue to be released with latent bugs. A verification test suite developed for a sequential program is not adequate for a concurrent program. A major problem with design verification of concurrent systems is the lack of good coverage metrics. Coverage metrics are heuristic measures of the exhaustiveness of a test suite. High coverage, in general, implies fewer bugs. SystemC is the most popular concurrent system level modeling language used for designing SoCs in the industry. We propose to attack the verification quality problem for concurrent SystemC programs by developing novel mutation testing based coverage metrics. Mutation testing has successfully been applied in software testing and RTL designs. In this paper, we develop a comprehensive set of mutation operators for concurrency constructs in SystemC. Our approach is also unique in that we define a novel concurrent coverage metric considering multiple execution schedules that a concurrent program can generate. This metric allows us to adequately measure the coverage for concurrent programs. We performed experiments with various designs including a large industrial design and obtained favorable results on multiple applications.

show abstract

Leveraging a Commercial Mutation Analysis Tool For Research

Cited by 25 publications

References 6 publications

Efficient mutation analysis by propagating and partitioning infected execution states

Efficient mutation analysis by propagating and partitioning infected execution states

Concurrency-oriented verification and coverage of system-level designs

Coverage metrics for verification of concurrent SystemC designs using mutation testing

Contact Info

Product

Resources

About