Metamorphic Testing

Abstract

Metamorphic testing is an approach to both test case generation and test result veri cation. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its rst publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identi cation, test case generation, integration with other so ware engineering techniques, and the validation and evaluation of so ware systems. In this paper, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research. CCS Concepts: •So ware and its engineering → So ware veri cation and validation; So ware testing and debugging;

Adaptive metamorphic testing with contextual bandits

“…Metamorphic Testing (MT) aims at using necessary properties of a software-under-test to either check its expected outputs or to generate new test cases [1,2]. Central to MT is the concept of Metamorphic Relations (MRs) which are high-level observable properties that must hold over inputs and outputs of the system-under-test.…”

“…In the following, we formalize the definition of a metamorphic relation, the transformation from source test cases to follow-up test cases, and metamorphic testing. Our definitions follow the formalization by Chen et al [2], except for the transformation from a source to a follow-up test case. We interpret the transformation function to apply to the whole test case, which includes the test input, and formalize it accordingly in this more general, but compatible, way:…”

“…Metamorphic Testing (MT) is a software testing paradigm which aims at using necessary properties of a software-under-test to either check its expected outputs or to generate new test cases [1,2]. More precisely, MT tackles the so-called oracle problem which occurs whenever predicting the expected outputs of a system-under-test is just too difficult or even impossible.…”

“…MT has been very successful to address testing issues in various application domains, e.g., driverless cars [7], search engines [8], or bioinformatics [9] just to name a few (see Section 2.3 for more references). However, generally speaking, applying MT in practice requires to address two issues: the MR identification and the MR selection problems [2]. The former occurs when trying to identify MRs for a specific system-under-test, i.e., to formalize input transformation properties which lead to a known transformation of the outputs.…”

Compiler Testing Approach based on Generalized Equivalent Substitution

“…The three compiler testing techniques have their advantages and disadvantages, in which RDT is the most effective technique for error detection [6] , but it relies on high-precision compilers [3] , and the impact of oracle in EMI technology is minimal , But it is less efficient [6] . The Metamorphic Testing Technique [10] is to test the program by checking the relationship between multiple execution results of the program, without constructing the expected output [4] , effectively avoiding the risk of a test oracle in software testing. In this paper, the technique of metamorphic testing is applied to the compiler testing, and a new compiler test method based on generalized equivalence substitution is proposed.…”