Impact analysis in the presence of dependence clusters using Static Execute After in WebKit

Abstract

Abstract-Impact analysis based on code dependence can be an integral part of software quality assurance by providing opportunities to identify those parts of the software system that are affected by a change. Because changes usually have far reaching effects in programs, effective and efficient impact analysis is vital, which has different applications including change propagation and regression testing. Static Execute After (SEA) is a relation on program elements (procedures) that is efficiently computable and accurate enough to be a candidate for use in impact analysis in practice. To assess the applicability of SEA in terms of capturing real defects, we present results on integrating it into the build system of WebKit, a large, open source software system, and on related experiments. We show that a large number of real defects can be captured by impact sets computed by SEA, albeit many of them are large. We demonstrate that this is not an issue in applying it to regression test prioritization, but generally it can be an obstacle in the path to efficient use of impact analysis. We believe that the main reason for large impact sets is the formation of dependence clusters in code. As apparently dependence clusters cannot be easily avoided in the majority of cases, we focus on determining the effects these clusters have on impact analysis.

Abstracting Program Dependencies Using the Method Dependence Graph

“…For NanoXML and Jaba, the precision was over 95% for 75% of queries, and for Schedule1 it was also as high as 90%. In these cases, we found many queries that share the same dependence sets, possibly due to the existence of dependence clusters as previously investigated [13]. The worst overall precision was seen by XML-security, for which the MDG gave a precision of no more than 85% for 75% of its 1,928 queries.…”

“…When such inputs are not available, impact analysis would be performed using static approaches. In the current literature, the most cost-effective method-level static impact analysis we are aware of is based on the SEA relations among methods [13]. Such analyses input a program and a query (a method for which impacts are queried), and add all methods that possibly statically execute after the query into its impact set as the output.…”

“…At method level, a main approach to such analysis is to find methods that are directly or transitively dependent on the given query. In comparison to the SEA-based impact analysis that requires a reachability algorithm [12], [13], a static impact analysis based on the MDG simply computes the transitive closure from the query. The MDG-based impact analysis also gives more information regarding how, in addition to whether, impacts propagate across methods (through the ports and edges between them), thus it tends to better support impact inspection and understanding, than the SEA-based approach.…”

“…This approach approximates dependencies among methods via control flows using the interprocedural control flow graph (ICFG) and was shown to as almost precise as static slicing based on the TSD model while coming at low costs and perfect (100%) recall. Later, the SEA was also applied to static impact analysis proven more accurate than peer techniques [12] while improving regression test selection and prioritization [13] as well.…”

A framework for cost-effective dependence-based dynamic impact analysis

“…MOTIVATION One important application of DDIA is regression testing, for which developers use the results of impact analysis to guide regression test selection and prioritization [10], [19]. For test selection, only test cases that cover at least one impacted entity need be executed; for test prioritization, test cases that cover more impacted entities can be given higher priority.…”