Automated inference of likely metamorphic relations for model transformations

Troya, Javier; Segura, Sergio; Ruiz-Corts, Antonio

doi:10.1016/j.jss.2017.05.043

Cited by 33 publications

(24 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These properties are possibly the most successful examples of patterns so far having been used by different research groups for the identification and inference of metamorphic relations in several machine learning and numerical programs, e.g., [15], [16], [22]- [24]. Troya et al [14] identified a set of domain-independent trace patterns to automatically infer likely metamorphic relations in the context of model transformations.…”

Section: Background a Metamorphic Relation Patternsmentioning

confidence: 99%

“…This is because patterns guide testers on the search for metamorphic relations with a certain structure, making the identification of the relations significantly easier than when starting from scratch. Second, the identification of patterns is a key point to enable the automated inference of likely metamorphic relations that conform to those patterns [14]. For example, papers on the automated inference of likely metamorphic relations in numerical and machine learning programs (e.g., [15], [16]) typically search for instances of the "metamorphic properties" (similar to the concept of pattern) defined by Murphy et al back in 2008 [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Metamorphic Relation Patterns for Query-Based Systems

Segura

Durán

Troya

et al. 2019

2019 IEEE/ACM 4th International Workshop on Metamorphic Testing (MET)

Self Cite

View full text Add to dashboard Cite

Searching and displaying data based on user queries is a key feature of most software applications such as information systems, web portals, web APIs, and data analytic platforms. The large volume of data managed by these types of systems, henceforth called query-based systems (QBS), makes them extremely hard to test due to the difficulty to assess whether the output of a query is correct, the so-called oracle problem. Metamorphic testing has proved to be a very effective approach to alleviate the oracle problem in QBS, enabling the detection of bugs in data repositories, large e-commerce sites, and some of the most used software applications worldwide such as Google Search and YouTube. We have observed, however, that the metamorphic relations used to test different types of QBS are very similar, regardless of their domain, since all of them exploit standard query features such as filtering and ordering. Inspired by this finding, in this paper we present a catalogue of metamorphic relation patterns to assist testers in the identification and inference of metamorphic relations in QBS. For the definition of the patterns we resorted to the root of most query languages: relational algebra. We show how the proposed patterns help in the identification of metamorphic relations in the e-commerce platform PrestaShop, the email service Gmail, and the mobile application of video streaming HBO.

show abstract

Section: Background a Metamorphic Relation Patternsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Metamorphic Relation Patterns for Query-Based Systems

Segura

Durán

Troya

et al. 2019

2019 IEEE/ACM 4th International Workshop on Metamorphic Testing (MET)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The inference process is constrained by searching for a set of predefined metamorphic relations. Javier [17] proposed an approach to infer likely metamorphic relations automatically for Atlas Transformation Language (ATL) model transformations. Liu [14] proposed a method named composition of metamorphic relations (CMR) to construct new metamorphic relations by combining several existing relations.…”

Section: Metamorphic Relation Inferencementioning

confidence: 99%

“…Some studies try to classify the classification of MRs based on machine learning methods [12,13], or attempt to construct more MRs based on information of the existence of the initial MRs [14,15]. Other recent research has yield MR identification based on the concepts of category and choice [16,17]. Liu [14] verified that the composition of metamorphic relations can improve the failure-detection capabilities rather than any particular metamorphic relations.…”

Section: Introductionmentioning

confidence: 99%

A Genetic Algorithm-Based Approach for Composite Metamorphic Relations Construction

Xiang

2019

Information

View full text Add to dashboard Cite

The test oracle problem exists widely in modern complex software testing, and metamorphic testing (MT) has become a promising testing technique to alleviate this problem. The inference of efficient metamorphic relations (MRs) is the core problem of metamorphic testing. Studies have proven that the combination of simple metamorphic relations can construct more efficient metamorphic relations. In most previous studies, metamorphic relations have been mainly manually inferred by experts with professional knowledge, which is an inefficient technique and hinders the application. In this paper, a genetic algorithm-based approach is proposed to construct composite metamorphic relations automatically for the program to be tested. We use a set of relation sequences to represent a particular class of MRs and turn the problem of inferring composite MRs into a problem of searching for suitable sequences. We then dynamically implement multiple executions of the program and use a genetic algorithm to search for the optimal set of relation sequences. We conducted empirical studies to evaluate our approach using scientific functions in the GNU scientific library (abbreviated as GSL). From the empirical results, our approach can automatically infer high-quality composite MRs, on average, five times more than basic MRs. More importantly, the inferred composite MRs can increase the fault detection capabilities by at least 30 % more than the original metamorphic relations.

show abstract

“…A concrete example of an "equivalence" MROP is that a search for YouTube videos should return the same set of videos "regardless of the ordering criteria specified (date, rating, relevance, title, or number of views)" [39]. Segura et al [39] hypothesized that their proposed patterns could also be useful for automated inferencing of likely MRs for a given API; however, they also pointed out that such research, as in [40]- [42], was challenging, and still at an early stage.…”

Section: Introductionmentioning

confidence: 99%

Metamorphic Relations for Enhancing System Understanding and Use

Zhi

Sun

Chen

et al. 2020

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Modern information technology paradigms, such as online services and off-the-shelf products, often involve a wide variety of users with different or even conflicting objectives. Every software output may satisfy some users, but may also fail to satisfy others. Furthermore, users often do not know the internal working mechanisms of the systems. This situation is quite different from bespoke software, where developers and users usually know each other. This paper proposes an approach to help users to better understand the software that they use, and thereby more easily achieve their objectives-even when they do not fully understand how the system is implemented. Our approach borrows the concept of metamorphic relations from the field of metamorphic testing (MT), using it in an innovative way that extends beyond MT. We also propose a "symmetry" metamorphic relation pattern and a "change direction" metamorphic relation input pattern that can be used to derive multiple concrete metamorphic relations. Empirical studies reveal previously unknown failures in some of the most popular applications in the world, and show how our approach can help users to better understand and better use the systems. The empirical results provide strong evidence of the simplicity, applicability, and effectiveness of our methodology.

show abstract

Automated inference of likely metamorphic relations for model transformations

Cited by 33 publications

References 80 publications

Metamorphic Relation Patterns for Query-Based Systems

Metamorphic Relation Patterns for Query-Based Systems

A Genetic Algorithm-Based Approach for Composite Metamorphic Relations Construction

Metamorphic Relations for Enhancing System Understanding and Use

Contact Info

Product

Resources

About