A robustness testing approach for SOAP Web services

Abstract

The use of Web services in enterprise applications is quickly increasing. In a Web services environment, providers supply a set of services for consumers. However, although Web services are being used in business-critical environments, there are no practical means to test or compare their robustness to invalid and malicious inputs. In fact, client applications are typically developed with the assumption that the services being used are robust, which is not always the case. Robustness failures in such environments are particularly dangerous, as they may originate vulnerabilities that can be maliciously exploited, with severe consequences for the systems under attack. This paper addresses the problem of robustness testing in Web services environments. The proposed approach is based on a set of robustness tests (including both malicious and non-malicious invalid call parameters) that is used to discover programming and design errors. This approach, useful for both service providers and consumers, is demonstrated by two sets of experiments, showing, respectively, the use of Web services Robustness testing from the consumer and the provider points of view. The experiments comprise the robustness testing of 1,204 Web service operations publicly available in the Internet and of 29 home-implemented services, including two different implementations of the Web services specified by the standard TPC-App performance benchmark. Results show that many Web services are deployed with critical robustness N. Laranjeiro · M. Vieira (B) · H. Madeira DEI/CISUC, University of Coimbra, 3030-290 Coimbra, Portugal e-mail: mvieira@dei.uc.pt N. Laranjeiro e-mail: cnl@dei.uc.pt H. Madeira e-mail: henrique@dei.uc.pt problems and that robustness testing is an effective approach to improve services quality.

Testing data-centric services using poor quality data: from relational to NoSQL document databases

“…In practice, software fault-injection techniques aim to understand how well the software tolerates faults, which may include observing the behavior of failure detection and recovery mechanisms [30]. In this context, the closest studies in the domain of this paper actually come from the robustness testing area [10][11][12]. Although the perspective is quite different, in the end, the goal is to observe how the application deals with erroneous inputs, and this kind of concept also applies to other domains (e.g., fuzzing or even penetration testing).…”

“…It applies fault injection to parameters of system calls and also to the memory segments that implement the microkernel being tested. In a previous work, we defined an approach to assess the behavior of web services in the presence of tampered SOAP messages [10]. It consists of a set of robustness tests based on invalid web service call parameters.…”

“…The way systems are nowadays being developed, with frequent changes and hard time constraints, leads developers to focus on the functional properties of systems, disregarding important non-functional properties such as robustness or security [10]. With little time to apply runtime verification techniques, developers end up by deploying systems that comply with basic functionality but that may fail in presence of unexpected conditions.…”

“…The ability to gracefully handle erroneous inputs is a well-known problem in the robustness testing domain, where tests using invalid inputs applied on external interfaces of many different systems have been used with great success [10][11][12]. In the case of the data quality community, the problem of poor quality data (also known as dirty data) is also very well known, with numerous tools and techniques for handling different problems involving dirty data.…”

Fault Injection for Performance Testing of Composite Web Services

“…Fault injection injects anomalies into a system to assess the dependability of the system. Traditional fault injection approaches for web services mostly focus on functionality related issues, e.g., injecting mutated data into a system to see whether the system can handle data corruption caused by network transmission errors [4,9,11,18,21]. Besides functionality related issues, performance related issues are also very important.…”