Improving Test Case Generation for REST APIs Through Hierarchical Clustering

Abstract

With the ever-increasing use of web APIs in modernday applications, it is becoming more important to test the system as a whole. In the last decade, tools and approaches have been proposed to automate the creation of system-level test cases for these APIs using evolutionary algorithms (EAs). One of the limiting factors of EAs is that the genetic operators (crossover and mutation) are fully randomized, potentially breaking promising patterns in the sequences of API requests discovered during the search. Breaking these patterns has a negative impact on the effectiveness of the test case generation process. To address this limitation, this paper proposes a new approach that uses Agglomerative Hierarchical Clustering (AHC) to infer a linkage tree model, which captures, replicates, and preserves these patterns in new test cases. We evaluate our approach, called LT-MOSA, by performing an empirical study on 7 real-world benchmark applications w.r.t. branch coverage and real-fault detection capability. We also compare LT-MOSA with the two existing state-of-the-art white-box techniques (MIO, MOSA) for REST API testing. Our results show that LT-MOSA achieves a statistically significant increase in test target coverage (i.e., lines and branches) compared to MIO and MOSA in 4 and 5 out of 7 applications, respectively. Furthermore, LT-MOSA discovers 27 and 18 unique real-faults that are left undetected by MIO and MOSA, respectively.

SynTest-Solidity: Automated Test Case Generation and Fuzzing for Smart Contracts

“…It achieves this by automating the process of test case generation using state-of-the-art metaheuristic search algorithms. As part of our future plan, we will extend the framework with linkage learning-based evolutionary algorithms [8], MOSA [5], and sFuzz [4]. Using these additional algorithms, we plant to perform a more extensive evaluation.…”

More Effective Test Case Generation with Multiple Tribes of AI

“…I argue that detecting and preserving patterns of HTTP requests, hereafter referred to as linkage structures, improves the effectiveness of the test case generation process. I proposed a novel approach [44], named LT-MOSA that uses Agglomerative Hierarchical Clustering (AHC) to infer these linkage structures from automatically generated test cases in the context of REST API testing. Specifically, AHC generates a Linkage Tree (LT) model from the test cases that are the closest to reach uncovered test targets (i.e., lines and branches).…”

“…To evaluate the feasibility and effectiveness of this approach, I implemented this approach within EvoMaster. I performed an empirical study [44] with 7 real-world benchmark web/enterprise applications from the EvoMaster Benchmark (EMB) dataset. The study compares the proposed approach against the two state-ofthe-art algorithms for system-level test generations implemented in EvoMaster, namely Many Independent Objective (MIO) and Many Objective Search Algorithm (MOSA).…”

Testing Real-World Healthcare IoT Application: Experiences and Lessons Learned

“…Many works are available targeting different types of testing of REST APIs such as regression testing [23], model-based testing [29], specification-based testing [17], robustness testing [28], metamorphic testing [37], search-based test case improvement [40], security testing [13], and test input validation using deep learning [34]. Some studies are also conducted analyzing REST APIs testing approaches/tools in different contexts such as in [27], [30], and [11].…”