Many software testing elds, like white-box testing, test case generation, test prioritization and fault localization, depend on code coverage measurement. If used as an overall completeness measure, the minor inaccuracies of coverage data reported by a tool do not matter that much; however, in certain situations they can lead to serious confusion. For example, a code element that is falsely reported as covered can introduce false condence in the test. This work investigates code coverage measurement issues for the Java programming language. For Java, the prevalent approach to code coverage measurement is using bytecode instrumentation due to its various benets over source code instrumentation. As we have experienced, bytecode instrumentation-based code coverage tools produce dierent results than source code instrumentation-based ones in terms of the reported items as covered. We report on an empirical study to compare the code coverage results provided by tools using the dierent instrumentation types for Java coverage measurement on the method level. In particular, we want to nd out how much a bytecode instrumentation approach is inaccurate compared to a source code instrumentation method. The dierences are systematically investigated both in quantitative (how much the outputs dier) and in qualitative terms (what the causes for the dierences are). In addition, the impact on test prioritization and test suite reduction a possible application of coverage measurement is investigated in more detail as well. Keywords Code coverage • white-box testing • Java bytecode instrumentation • source code instrumentation • coverage tools • empirical study The nal publication is available at Springer via

No abstract

Assessing the overall quality (adequacy for a particular purpose) of existing test suites is a complex task. Their code coverage is a simple yet powerful attribute for this purpose, so the additional benefits of mutation analysis may not always justify the comparably much higher costs and complexity of the computation. Mutation testing methods and tools slowly start to reach a maturity level at which their use in everyday industrial practice becomes possible, yet it is still not completely clear in which situations they provide additional insights into various quality attributes of the test suites. This paper reports on an experiment conducted on four open source systems' test suites to compare them from the viewpoints of code coverage, mutation score and test suite reducibility (the amount test adequacy is degraded in a reduced test suite). The purpose of the comparison is to find out when the different attributes provide additional insights with respect to defect density, a separately computed attribute for the estimation of real faults. We demonstrate that in some situations code coverage might be a sufficient indicator of the expected defect density, but mutation and reducibility are better in most of the cases.

Data visualisation with high expressive power plays an important role in code comprehension. Recent visualisation tools try to fulfill the expectations of the users and use various analogies. For example, in an architectural metaphor, each class is represented by a building. Buildings are grouped into districts according to the structure of the namespaces. We think that these unique ways of code representation have great potential, but in our opinion they use very simple graphical techniques (shapes, figures, low resolution) to visualise the structure of the source code. On the other hand, computer games use high quality graphic and have high expressive power. A good example is Minecraft, a popular role playing game that supports both high definition, photo-realistic textures and long range 3D scene displaying. Additionally, it provides great extensibility and interactivity for third party software. In this paper, we introduce our mission to create a virtual world of source code in which developers and other stakeholders could explore and evaluate their project collaboratively in a virtual Minecraft world. Code properties are represented by graphical primitives offered by the game engine, and various interactivity features are planned. Besides challenges of the implementation there are some fundamental research issues considering the selection of a set of visual elements and mapping to source code properties. These elements have to be compatible not only with the visualisation and with the data model but also with the thinking of developers.