The Impact of Concurrent Coverage Metrics on Testing Effectiveness

Hong, Shin; Staats, Matt; Ahn, Jae-Min; Kim, Moonzoo; Rothermel, Gregg

doi:10.1109/icst.2013.32

Cited by 18 publications

(20 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent empirical studies of these algorithms can be found in [35,20,19]. For example, ConTest [10] inserts delays at synchronization points to attempt to increase contention and force deadlocks during testing.…”

Section: Related Workmentioning

confidence: 98%

Assertion guided abstraction

Kusano

Wang

2014

Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

View full text Add to dashboard Cite

We propose a new method for reducing the interleaving space during stateless model checking of multithreaded C/C++ programs. The problem is challenging because of the exponential growth of possible interleavings between threads. We have developed a new method, called assertion guided abstraction, which leverages both static and dynamic program analyses in a cooperative framework to reduce the interleaving space. Unlike existing methods that consider all interleavings of all conflicting memory accesses in a program, our new method relies on a new notion of predicate dependence based on which we can soundly abstract the interleaving space to only those conflicting memory accesses that may cause assertion violations and/or deadlocks. Our experimental evaluation of assertion guided abstraction on open source benchmarks shows that it is capable of achieving a significant reduction, thereby allowing for the verification of programs that were previously too complex for existing algorithms to handle.

show abstract

Section: Related Workmentioning

confidence: 98%

Assertion guided abstraction

Kusano

Wang

2014

Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

View full text Add to dashboard Cite

show abstract

“…CCmutator is naive in that it will not ensure a mutant is faulty. The authors of [14] dealt with this problem when using mutation testing by removing mutants from their test suite that did not fail any tests, were malformed code or were killed by every test. LLVM ensures that CCmutator at least generates syntactically correct LLVM IR.…”

Section: The List Of Mutation Operatorsmentioning

confidence: 99%

“…In [14] different concurrent coverage metrics were evaluated against each other using Java concurrent mutation testing to see which could find the most bugs among many randomly generated mutants. CCmutator has the potential to do the same for C/C++.…”

Section: Related Workmentioning

confidence: 99%

CCmutator: A mutation generator for concurrency constructs in multithreaded C/C++ applications

Kusano

Wang

2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

We introduce CCmutator, a mutation generation tool for multithreaded C/C++ programs written using POSIX threads and the recently standardized C++11 concurrency constructs. CCmutator is capable of performing partial mutations and generating higher order mutants, which allow for more focused and complex combinations of elementary mutation operators leading to higher quality mutants. We have implemented CCmutator based on the popular Clang/LLVM compiler framework, which allows CCmutator to be extremely scalable and robust in handling real-world C/C++ applications. CCmutator is also designed in such a way that all mutants of the same order can be generated in parallel, which allows the tool to be easily parallelized on commodity multicore hardware to improve performance.

show abstract

“…White-box testing (often referred to also as structure-based testing) is a dynamic test design technique that relies on code coverage to systematically verify the amount of tests needed to achieve a completeness goal (although, it is not clear whether coverage metrics themselves can predict the effectiveness of the test suite [7] or not [8]). This goal is sometimes expected to be complete (that is, 100%) coverage, however in practice this high level is rarely attainable due to various reasons.…”

Section: A White-box Testingmentioning

confidence: 99%

Negative Effects of Bytecode Instrumentation on Java Source Code Coverage

Tengeri

Horváth

Beszedes

et al. 2016

2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)

View full text Add to dashboard Cite

Code coverage measurement is an important element in white-box testing, both in industrial practice and academic research. Other related areas are highly dependent on code coverage as well, including test case generation, test prioritization, fault localization, and others. Inaccuracies of a code coverage tool sometimes do not matter that much but in certain situations they can lead to serious confusion. For Java, the prevalent approach to code coverage measurement is to use bytecode instrumentation due to its various benefits over source code instrumentation. However, if the results are to be mapped back to source code this may lead to inaccuracies due to the differences between the two program representations. In this paper, we systematically investigate the amount of differences in the results of these two Java code coverage approaches, enumerate the possible reasons and discuss the implications on various applications. For this purpose, we relied on two widely used tools to represent the two approaches and a set of benchmark programs from the open source domain.

show abstract

The Impact of Concurrent Coverage Metrics on Testing Effectiveness

Cited by 18 publications

References 29 publications

Assertion guided abstraction

Assertion guided abstraction

CCmutator: A mutation generator for concurrency constructs in multithreaded C/C++ applications

Negative Effects of Bytecode Instrumentation on Java Source Code Coverage

Contact Info

Product

Resources

About