Building an open-source system test generation tool: lessons learned and empirical analyses with EvoMaster

Abstract

Research in software testing often involves the development of software prototypes. Like any piece of software, there are challenges in the development, use and verification of such tools. However, some challenges are rather specific to this problem domain. For example, often these tools are developed by PhD students straight out of bachelor/master degrees, possibly lacking any industrial experience in software development. Prototype tools are used to carry out empirical studies, possibly studying different parameters of novel designed algorithms. Software scaffolding is needed to run large sets of experiments efficiently. Furthermore, when using AI-based techniques like evolutionary algorithms, care needs to be taken to deal with their randomness, which further complicates their verification. The aforementioned represent some of the challenges we have identified for this domain. In this paper, we report on our experience in building the open-source EvoMaster tool, which aims at system-level test case generation for enterprise applications. Many of the challenges we faced would be common to any researcher needing to build software testing tool prototypes. Therefore, one goal is that our shared experience here will boost the research community, by providing concrete solutions to many development challenges in the building of such kind of research prototypes. Ultimately, this will lead to increase the impact of scientific research on industrial practice.

Testing Real-World Healthcare IoT Application: Experiences and Lessons Learned

“…Many works are available targeting different types of testing of REST APIs such as regression testing [23], model-based testing [29], specification-based testing [17], robustness testing [28], metamorphic testing [37], search-based test case improvement [40], security testing [13], and test input validation using deep learning [34]. Some studies are also conducted analyzing REST APIs testing approaches/tools in different contexts such as in [27], [30], and [11]. We however focus on the healthcare IoT application in the context of Oslo City's health department with the ultimate goal of suggesting the necessary testing tools for continuous testing of their evolving healthcare IoT applications.…”

“…Several approaches have been proposed for testing of web applications REST APIs such as RESTest [32], RESTler [12], Semanthesis [24], RESTCT [42], RestTestGen [14], and EvoMaster [9]. Some empirical studies have been conducted to compare different approaches for testing REST APIs, such as [11,27,30]. Our goal is to assess one of the latest approaches suitable for a healthcare IoT application deployed in Oslo City to provide recommendations to the Oslo City health department for testing their current and future healthcare IoT applications.…”

A Bi-Directional Engine for Transformation of Big Data Between NoSQL and SQL Data

“…The importance of data is in an actual situation that may be required in any format, volume, and size in different domains. The papers [5,7], clearly express that the customised solution is one of the important requirement at the time of practical implementation. The digital data explosion due to the generation of big data brought some essential questions regarding data storage, retrieval, and structure.…”

“…[3,4] proposed a "MigDB" mechanism to convert the relational data into the MongoDB data model [3,4]. A. Arcuri et al in their article suggested and developed a generic testing tool "EvoMaster" to address the technical challenges of the prototype model of project work [5]. The authors C. Amadi, et al proposed the mapping of data from SQL to MongoDB document model, and rewriting of the corresponding is also proposed in paper [6].…”