Mutation Operators for Large Scale Data Processing Programs in Spark

Abstract

This paper proposes a mutation testing approach for big data processing programs that follow a data flow model, such as those implemented on top of Apache Spark. Mutation testing is a fault-based technique that relies on fault simulation by modifying programs, to create faulty versions called mutants. Mutant creation is carried on by operators able to simulate specific and well identified faults. A testing process must be able to signal faults within mutants and thereby avoid having ill behaviours within a program. We propose a set of mutation operators designed for Spark programs characterized by a data flow and data processing operations. These operators model changes in the data flow and operations, to simulate faults that take into account Spark program characteristics. We performed manual experiments to evaluate the proposed mutation operators in terms of cost and effectiveness. Thereby, we show that mutation operators can contribute to the testing process, in the construction of reliable Spark programs.

Modeling Big Data Processing Programs

“…A natural extension to this work would be to instantiate the tool for other systems of the data flow family (DryadLINQ, Apache Beam, Apache Flink ). This can be done by adapting TRANSMUT-Spark's front and back ends so that a program originally written in any of them can be tested with the mutation testing proposed in [21].…”

“…We first applied the model to formalize the mutation operators presented in [21], where we explored the application of mutation testing in Spark programs, and in the tool TRANSMUT-Spark 4 that we developed to automate this process.…”

“…Faults are simulated by applying mutation operators, which are rules with modification patterns for programs (a modified program is called a mutant). In [21], we presented a set of mutation operators designed for Spark programs that are divided into two groups: mutation operators for the data flow and mutation operators for transformations.…”

A two-level formal model for Big Data processing programs

“…The abstract and formal concepts provided by the model make it suitable for the automation of software development processes, such as those done by IDE tools. Consequently, we first applied the model to formalize the mutation operators presented in [13], where we explored the application of mutation testing in Spark programs, and in the tool TRANSMUT-Spark 1 [25] that we devel- oped to automate this process. Mutation testing is a fault-based technique that simulates faults to design and evaluate test sets [26].…”

“…We have used the model to define mutation operators that can be instantiated for different systems. In particular, specifications in our model have been used as an intermediate representation of programs in a mutation testing tool of Apache Spark programs [13].…”