Automated isolation for white-box test generation

Honfi, Dávid; Micskei, Zoltán

doi:10.1016/j.infsof.2020.106319

Cited by 6 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods retained in the dataset must satisfy two criteria: i) Pex was able to start test generation (e.g., they are public, they are in a non-abstract class, etc.) and ii) our tool presented in [1] did not throw any error. Note that a large number of methods were filtered from the initial 7187 because of the current limitations of Pex.…”

Section: Data Descriptionmentioning

confidence: 94%

“…The number of generated tests by Pex fall into three categories (see [1] ): no tests were generated (864), a single test case was generated (793), multiple test cases were generated (939). The number of generated tests is an initial indicator of the success of test generation.…”

Section: Data Descriptionmentioning

confidence: 99%

“…The dataset presents metrics about the attributes of the selected projects (e.g., cyclomatic complexity or number of external method calls) and the test generation (e.g., statement and branch coverage, number of warnings). This data is compared to an automated isolation technique in the paper Automated Isolation for White-box Test Generation [1] . To the best of our knowledge, this is the largest public dataset about the test generation performance of Microsoft Pex on open source projects.…”

mentioning

confidence: 99%

“…Data were collected using scripts parsing the output reports and generated tests. Data format Raw Analyzed Parameters for data collection Test generation was executed on 10 different open source projects selected from GitHub 1) using the default parameters of Pex and then 2) with the automated isolation technique recommended in [1] . Description of data collection Test generation was executed on a cloud virtual machine running Windows 10 and having 3.5 GBs of memory along with a dedicated CPU running at 2.4 GHz.…”

mentioning

confidence: 99%

See 3 more Smart Citations

White-box software test generation with Microsoft Pex on open source C# projects: A dataset

Honfi

Micskei

2020

Data in Brief

Self Cite

View full text Add to dashboard Cite

The paper presents a dataset on software tests generated using the Microsoft Pex (IntelliTest) test generator tool for 10 open source projects. The projects were selected randomly from popular GitHub repositories written in C#. The selected projects contain 7187 methods from which Pex was able to generate tests for 2596 methods totaling 38,618 lines of code. Data collection was performed on a cloud virtual machine. The dataset presents metrics about the attributes of the selected projects (e.g., cyclomatic complexity or number of external method calls) and the test generation (e.g., statement and branch coverage, number of warnings). This data is compared to an automated isolation technique in the paper Automated Isolation for White-box Test Generation [1] . To the best of our knowledge, this is the largest public dataset about the test generation performance of Microsoft Pex on open source projects. The dataset highlights current practical challenges and can be used as a baseline for new test generation techniques.

show abstract

Section: Data Descriptionmentioning

confidence: 94%

Section: Data Descriptionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

White-box software test generation with Microsoft Pex on open source C# projects: A dataset

Honfi

Micskei

2020

Data in Brief

Self Cite

View full text Add to dashboard Cite

show abstract

“…Symbolic execution may also be used to generate mocks [7], [21], [47], to mock the file system [48], or a database for use in tests [49]. Honfi and Micskei [50] generate mocks to replace external interactions with calls to a parameterized sandbox. This sandbox receives inputs from the white-box test generator, Pex [22].…”

Section: Mock Generationmentioning

confidence: 99%

Mimicking Production Behavior with Generated Mocks

Tiwari¹,

Monperrus²,

Baudry³

2022

Preprint

View full text Add to dashboard Cite

Mocking in the context of automated software tests allows testing program units in isolation. Designing realistic interactions between a unit and its environment, and understanding the expected impact of these interactions on the behavior of the unit, are two key challenges that software testers face when developing tests with mocks. In this paper, we propose to monitor an application in production to generate tests that mimic realistic execution scenarios through mocks. Our approach operates in three phases. First, we instrument a set of target methods for which we want to generate tests, as well as the methods that they invoke, which we refer to mockable method calls. Second, in production, we collect data about the context in which target methods are invoked, as well as the parameters and the returned value for each mockable method call. Third, offline, we analyze the production data to generate test cases with realistic inputs and mock interactions. The approach is automated and implemented in an open-source tool called RICK. We evaluate our approach with three real-world, open-source Java applications. RICK monitors the invocation of 128 methods in production across the three applications and captures their behavior. Next, RICK analyzes the production observations in order to generate test cases that include rich initial states and test inputs, mocks and stubs that recreate actual interactions between the method and its environment, as well as mock-based oracles. All the test cases are executable, and 52.4% of them successfully mimic the complete execution context of the target methods observed in production. We interview 5 developers from the industry who confirm the relevance of using production observations to design mocks and stubs.

show abstract