Evaluation of the IPO-Family algorithms for test case generation in web security testing

Abstract

Security testing of web applications remains a major problem of software engineering. In order to reveal vulnerabilities, testing approaches use different strategies for detection of certain kinds of inputs that might lead to a security breach. Such approaches depend on the corresponding test case generation technique that are executed against the system under test. In this work we examine how two of the most popular algorithms for combinatorial test case generation, namely the IPOG and IPOG-F algorithms, perform in web security testing. For generating comprehensive and sophisticated testing inputs we have used input parameter modelling which includes also constraints between the different parameter values. To handle the test execution, we make use of a recently introduced methodology which is based on model-based testing. Our evaluation indicates that both algorithms generate test inputs that succeed in revealing security leaks in web applications with IPOG-F giving overall slightly better results w.r.t. the test quality of the generated inputs. In addition, using constraints during the modelling of the attack grammars results in an increase on the number of test inputs that cause security breaches. Last but not least, a detailed analysis of our evaluation results confirms that combinatorial testing is an efficient test case generation method for web security testing as the security leaks are mainly due to the interaction of a few parameters. This statement is further supported by some combinatorial coverage measurement experiments on the successful test inputs.

A Combinatorial Approach to Analyzing Cross-Site Scripting (XSS) Vulnerabilities in Web Application Security Testing

“…-In the majority of our past security testing experiments [2], [3], [7] we have witnessed that higher strength interaction testing yields better results w.r.t. exploitation rate.…”

“…Combinatorial testing has been successfully applied for testing (critical) software systems in large organizations [11]. It is an already proven method for blackbox security testing of large-scale web software systems [7], [3], [2] where t-way testing was applied successfully to XSS detection. In this section, we review these key contributions in web security testing that are based on combinatorial methods and are used as a basis for analyzing XSS vulnerabilities via fault localization methods throughout this paper.…”

“…However, we were still able to exploit XSS vulnerabilities using a 4-way test set. -An important finding in the post-processing of 2-way test sets used in [3] was that it revealed a surprising high percentage of 3-way and 4-way combinations covered in the successful XSS attack vectors (per test set and SUT).…”

Attack Pattern-Based Combinatorial Testing with Constraints for Web Security Testing

“…The combinatorial test design process for XSS attack vector generation, has been explained in detail in [6], [10].…”

DeepSQLi: Deep Semantic Learning for Testing SQL Injection

“…The metrics and evaluation method used is a typical example of the construct threats, which concern whether the metrics/evaluation method can reflect what we intend to measure. In our work, the metrics studied are widely used in this field of research [59,60] and they serves as quality indicator for different aspects of SQLi testing. To mitigate the randomness introduced by the training, we repeat 20 experiment runs for each tool under a SUT.…”