ReduKtor: How We Stopped Worrying About Bugs in Kotlin Compiler

Stepanov, Daniil; Akhin, Marat; Belyaev, Mikhail

doi:10.1109/ase.2019.00038

Cited by 12 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further enhance mutation-based approaches, we decided to augment language-agnostic mutations with language-specific ones. These mutations consist of over 20 text-and tree-based transformation, which are based on our previous experience on Kotlin fuzzing and reduction [12].…”

Section: Discussionmentioning

confidence: 99%

“…Input reduction is implemented in order to make a small and understandable test from a possibly large and complex input leading to a compiler error. The approach we have implemented, called Reduktor, is described in details in [12]. On a very high level, it is based on a hybrid approach which combines several language-agnostic and language-specific techniques.…”

Section: Post-processingmentioning

confidence: 99%

See 1 more Smart Citation

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Stepanov¹,

Akhin²,

Belyaev³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Kotlin is a relatively new programming language from JetBrains: its development started in 2010 with release 1.0 done in early 2016. The Kotlin compiler, while slowly and steadily becoming more and more mature, still crashes from time to time on the more tricky input programs, not least because of the complexity of its features and their interactions. This makes it a great target for fuzzing, even the basic forms of which can find a significant number of Kotlin compiler crashes.There is a problem with fuzzing, however, closely related to the cause of the crashes: generating a random, non-trivial and semantically valid Kotlin program is hard. In this paper, we talk about type-centric compiler fuzzing in the form of typecentric enumeration, an approach inspired by skeletal program enumeration [1] and based on a combination of generative and mutation-based fuzzing, which solves this problem by focusing on program types. After creating the skeleton program, we fill the typed holes with fragments of suitable type, created via generation and enhanced by semantic-aware mutation.We implemented this approach in our Kotlin compiler fuzzing framework called Backend Bug Finder (BBF) and did an extensive evaluation, not only testing the real-world feasibility of our approach, but also comparing it to other compiler fuzzing techniques. The results show our approach to be significantly better compared to other fuzzing approaches at generating semantically valid Kotlin programs, while creating more interesting crash-inducing inputs at the same time. We managed to find more than 50 previously unknown compiler crashes, of which 18 were considered important after their triage by the compiler team.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Post-processingmentioning

confidence: 99%

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Stepanov¹,

Akhin²,

Belyaev³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…More details about the Kotlin program reducer and its results can be found in the article [20]. In general, a complex approach using language-agnostic methods and language-specific transformations gives a good result.…”

Section: Bug Manager Implementationmentioning

confidence: 99%

Backend Bug Finder — a platform for effective compiler fuzzing

Stepanov

Itsykson

2022

ICS

View full text Add to dashboard Cite

Introduction: The standard way to check the quality of a compiler is manual testing. However, it does not allow to cover a vast diversity of programs that can be written in a target programming language. Today, in addition to manual written tests there are many automated compiler testing methods, among which fuzzing is one of the most powerful and useful. A compiler fuzzer is a tool that generates a random program in a target language and checks how the compiler works in this language. Purpose: To develop a platform for compiler fuzzing and, based on it, to develop a tool for Kotlin compiler testing. Results: We have developed Backend Bug Finder which is a platform for compiler fuzzing is. We have chosen a mutation-based approach as a method for generating random programs. First, an existing program is entered to the mutator as the input to be then transformed in some way. Mutations can be both trivial, for example, replacing arithmetic operators with others, and complex, changing the structure of the program. Next, the resulting program is fed to the input of the compiler with the following check of its operation. The developed test oracle can detect three types of errors: crashes, miscompilations, and performance degradations. If an error is detected, the test case is fed into the post-processing module, where reduction and deduplication algorithms are applied. We have developed a tool for fuzzing the Kotlin language compiler based on the platform for its approbation, which showed the applicability of the proposed approach for finding errors in modern compilers. Practical relevance: Over a year and a half of work, our tool has found thousands of different Kotlin compiler bugs, more than 200 of which were sent to the developers, and more than 80 have been fixed.

show abstract

“…Recently a few researchers proposed modern implementations of HDD algorithm and their variants [5], [6], [7], [8]. Most of the implementations are language-agnostic and hence can reduce a variety of tests like HTML, source code in different programming languages, xml, and etc.…”

Section: Introductionmentioning

confidence: 99%

“…Most of the implementations are language-agnostic and hence can reduce a variety of tests like HTML, source code in different programming languages, xml, and etc. Stepanov et al noted the language-agnosticness of the HDD tools a major limiting factor in employing the tools efficiently for real-world, large-scale usage as the tools fail to consider and utilize the language-specific features, complexities and inter-dependence [8]. These tools rely on a generic AST or grammar in simplification process and produces many non-compilable intermediate variants before the convergence.…”

Section: Introductionmentioning

confidence: 99%

Redusharptor: A Tool to Simplify Developer-Written C# Unit Tests

Weber,

Christi

2023

IJSEA

View full text Add to dashboard Cite

Modern software systems are complex and locating, isolating and fixing a fault even with a failing test is tedious and time-consuming. Simplifying failing test(s) can significantly reduce the developer effort by reducing the irrelevant program entities that developers need to observe. Delta Debugging (DD) algorithm automatically reduces the failing tests. Hierarchical Delta Debugging (HDD) algorithm improves DD for hierarchical tests like source code and HTML files. Many modern implementations of these algorithms work on a generic tree-like structure and fail to consider complex structures, intricacies, and interdependence of program elements of a particular programming language. We propose a tool ReduSharptor to simplify C# tests that uses language-specific features and interdependence of C# program elements using Roslyn compiler APIs. We evaluate ReduSharptor on a set of 30 failing C# tests to demonstrate its applicability and accuracy.

show abstract

ReduKtor: How We Stopped Worrying About Bugs in Kotlin Compiler

Cited by 12 publications

References 22 publications

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Backend Bug Finder — a platform for effective compiler fuzzing

Redusharptor: A Tool to Simplify Developer-Written C# Unit Tests

Contact Info

Product

Resources

About