Directed Test Suite Augmentation

Xu, Zhihong; Rothermel, Gregg

doi:10.1109/apsec.2009.29

Cited by 48 publications

(42 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the research community has invested a lot of effort in designing techniques for improving the testing of software patches, ranging from test suite prioritisation and selection algorithms [11,30,35] to program analysis techniques for test case generation and bug finding [1,2,20,21,27,28,36,40] to methods for surviving errors introduced by patches at runtime [14]. Many of these techniques depend on the existence of a manual test suite, sometimes requiring the availability of a test exercising the patch [24,37], sometimes making assumptions about the stability of program coverage or external behaviour over time [14,29], other times using it as a starting point for exploration [10,16,22,39], and often times employing it as a baseline for comparison [3,6,9,26]. However, despite the key role that test suites play in software testing, it is surprising how few empirical studies one can find in the research literature regarding the co-evolution of test suites and code and their impact on the execution of real systems.…”

Section: Introductionmentioning

confidence: 99%

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Marinescu

Hošek

Cadar

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Software repositories provide rich information about the construction and evolution of software systems. While static data that can be mined directly from version control systems has been extensively studied, dynamic metrics concerning the execution of the software have received much less attention, due to the inherent difficulty of running and monitoring a large number of software versions.In this paper, we present Covrig, a flexible infrastructure that can be used to run each version of a system in isolation and collect static and dynamic software metrics, using a lightweight virtual machine environment that can be deployed on a cluster of local or cloud machines.We use Covrig to conduct an empirical study examining how code and tests co-evolve in six popular open-source systems. We report the main characteristics of software patches, analyse the evolution of program and patch coverage, assess the impact of nondeterminism on the execution of test suites, and investigate whether the coverage of code containing bugs and bug fixes is higher than average.

show abstract

Section: Introductionmentioning

confidence: 99%

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Marinescu

Hošek

Cadar

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…Creating new, executable test cases for affected areas of the code, which is known as a test suite augmentation problem [6], is one of the important regression testing problems, but only a few researchers have started working on this problem [6], [7], [8], [9]. While their work has made some progress in this area, they have only provided guidance for creating new tests [7], generated new test cases limited to numeric values [9], and considered only small desktop applications. However, web applications involve different challenges than desktop applications written in C or Java [10], and the majority of web applications heavily deal with strings in addition to numeric values.…”

Section: Introductionmentioning

confidence: 99%

An Effective Regression Testing Approach for PHP Web Applications

Marback

Ehresmann

2012

2012 IEEE Fifth International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Web applications change and are upgraded frequently due to security attacks, feature updates, or user preference changes. These fixes often involve small patches or revisions, but still, testers need to perform regression testing on their products to ensure that the changes have not introduced new faults. Applying regression testing to the entire product, however, can be very expensive, and often, companies cannot afford to do this because, typically, the turnaround time to release patches is expected to be short. One solution is focusing only on the areas of code that have been changed and performing regression testing on them. In this way, companies can provide quick patches more dependably whenever they encounter security breaches. In this paper, we propose a new regression testing approach that is applied to frequently patched web applications, considering security problems, and in particular, focusing on PHP programs. Our approach identifies the affected areas by code changes using impact analysis and generates new test cases for the impacted areas by changes using program slices considering both numeric and string input values. To facilitate our approach, we implemented a PHP Analysis and Regression Testing Engine (PARTE) and performed a controlled experiment using open source web applications. The results show that our approach is effective in reducing the cost of regression testing for frequently patched web applications.

show abstract

“…Other techniques have been proposed for test-suite augmentation that attempt to generate a test case that exposes a difference in the output [12] or improves program coverage after the program changes [21]. These test-generation techniques are mostly complementary to ours-they can potentially be used to create test cases that satisfy the requirements that our approach identifies.…”

Section: Related Workmentioning

confidence: 99%

Applying aggressive propagation-based strategies for testing changes

Santelices

Harrold

2011

2011 Fourth IEEE International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Test-suite augmentation for evolving softwarethe process of augmenting a test suite to adequately test software changes-is necessary for any program that undergoes modifications as part of its development and maintenance cycles. Recently, we presented a new technique for test-suite augmentation based on leveraging the propagation conditions for the effects of changes. Although empirical studies show that this technique can be quite effective for testing changes, the experiments have been limited because of the complexity of the implementation. In this paper, we present a new and more efficient approach for propagation-based testing of changes that can reach much longer propagation-distances and can focus the testing more precisely on those behaviors of changes that can actually affect the output. Using an implementation of this new approach, we performed a study on a set of changes on Java programs for which we compared, to a much larger extent, our propagation-based strategy with other existing techniques for testing changes. The results of the study not only confirm the superior effectiveness of propagation-based strategies over these other techniques for testing changes, but also quantify that superiority and clarify the conditions under which our approach is most effective.

show abstract

Directed Test Suite Augmentation

Abstract: Abstract-As

Cited by 48 publications

References 20 publications

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

An Effective Regression Testing Approach for PHP Web Applications

Applying aggressive propagation-based strategies for testing changes

Contact Info

Product

Resources

About