Metamorphic Testing in Fault Localization of Model Transformations

Du, Keke; Jiang, Mingyue; Ding, Zuohua; Huang, Hongyun; Shu, Ting

doi:10.1007/978-3-030-41418-4_20

Cited by 7 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…He et al [55] applied metamorphic testing to bidirectional MTs, but they followed a diferent approach where testers must manually identify generic metamorphic relations of the MT. Finally, Du et al [34] proposed to combine metamorphic testing and bug localization to debug MTs (cf. Section 6).…”

Section: Test Oraclementioning

confidence: 99%

“…Finally, Du et al [34] propose to apply SBFL without the need to count on an oracle. To achieve this, the oracle is obtained by applying metamorphic testing techniques [242], similar as it is done in [124] (cf.…”

Section: Dynamic Approachesmentioning

confidence: 99%

“…We consider as an important challenge the use of realistic case studies, instead of small-scale academic examples. Mutation seems to be the major evaluation technique, since at least 17 primary studies apply it in their evaluation [5,8,9,18,26,27,34,48,55,57,64,70,83,110,111,124,131].…”

Section: Challenges (Rq4)mentioning

confidence: 99%

See 2 more Smart Citations

Model Transformation Testing and Debugging: A Survey

et al. 2022

View full text Add to dashboard Cite

Model transformations are the key technique in Model-Driven Engineering (MDE) to manipulate and construct models. As a consequence, the correctness of software systems built with MDE approaches relies mainly on the correctness of model transformations, and thus, detecting and locating bugs in model transformations have been popular research topics in recent years. This surge of work has led to a vast literature on model transformation testing and debugging, which makes it challenging to gain a comprehensive view of the current state of the art. This is an obstacle for newcomers to this topic and MDE practitioners to apply these approaches. This paper presents a survey on testing and debugging model transformations based on the analysis of 140 papers on the topics. We explore the trends, advances, and evolution over the years, bringing together previously disparate streams of work and providing a comprehensive view of these thriving areas. In addition, we present a conceptual framework to understand and categorise the different proposals. Finally, we identify several open research challenges and propose specific action points for the model transformation community.

show abstract

Section: Test Oraclementioning

confidence: 99%

Section: Dynamic Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Model Transformation Testing and Debugging: A Survey

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Burgueño et al (Burgueño et al 2015) proposed a static fault localization approach based on the extraction of metamodel footprints from the contracts and the transformation implementation for locating the bug. In contrast, Spectrum-Based Fault Localization (SBFL) for model transformations (Troya et al 2018;Li et al 2020;Du et al 2020) is a dynamic approach that requires input models and to execute the model transformation for locating the bug. Both approaches rank the transformation rules according to the likelihood that they contain the bug.…”

Section: An Aito Publicationmentioning

confidence: 99%

“…This results in the creation of a program spectrum that details the execution information of the program from a certain perspective, such as branch or statement coverage (Harrold et al 2000). Troya et al presented an approach to apply SBFL to model transformations (Troya et al 2018)-other authors also focused on this technique recently (Li et al 2020;Du et al 2020). As explained in (Troya et al 2018), the approach requires test cases that comprise input models for exercising the model transformation.…”

Section: Spectrum-based Fault Localizationmentioning

confidence: 99%

Revisiting Fault Localization Techniques for Model Transformations: Towards A Hybrid Approach.

Muñoz¹,

Troya²,

Wimmer³

et al. 2022

JOT

View full text Add to dashboard Cite

The correctness of software built through model transformations highly depends on the correctness of these transformations. Different approaches have been proposed to ensure the correctness of model transformations by checking if pairs of input-output models satisfy a set of contracts. If a contract is not satisfied, at least one transformation rule must contain a bug. Localizing the rules that contain bugs is key for repairing the model transformation. Among others, Spectrum-Based Fault Localization (SBFL) is a dynamic technique to locate the faulty component of a software, and it has already been applied in the context of model transformations considering the rules as the components. As a result, this technique proposes an order (a so-called suspiciousness ranking) in which the rules should be inspected in order to locate the bug. However, SBFL relies on so-called suspiciousness formulae that were created in different domains, so none of them offers a perfect behavior in the context of model transformations. Indeed, some of the rankings for model transformations present many ties, so the tester is uncertain as of which rule to inspect first in the ties. In this paper, we explore how SBFL can be combined with static information in a hybrid approach in order to improve the results obtained from SBFL, specially in the case of ties in the rankings.Our evaluation shows the potential of the hybrid approach to improve previous SBFL results for model transformations.

show abstract

Fault localization in DSLTrans model transformations by combining symbolic execution and spectrum-based analysis

Oakes,

Troya,

Galasso

et al. 2023

Softw Syst Model

View full text Add to dashboard Cite

The verification of model transformations is important for realizing robust model-driven engineering technologies and quality-assured automation. Many approaches for checking properties of model transformations have been proposed. Most of them have focused on the effective and efficient detection of property violations by contract checking. However, there remains the fault localization step between identifying a failing contract for a transformation based on verification feedback and precisely identifying the faulty rules. While there exist fault localization approaches in the model transformation verification literature, these require the creation and maintenance of test cases, which imposes an additional burden on the developer. In this paper, we combine transformation verification based on symbolic execution with spectrum-based fault localization techniques for identifying the faulty rules in DSLTrans model transformations. This fault localization approach operates on the path condition output of symbolic transformation checkers instead of requiring a set of test input models. In particular, we introduce a workflow for running the symbolic execution of a model transformation, evaluating the defined contracts for satisfaction, and computing different measures for tracking the faulty rules. We evaluate the effectiveness of spectrum-based analysis techniques for tracking faulty rules and compare our approach to previous works. We evaluate our technique by introducing known mutations into five model transformations. Our results show that the best spectrum-based analysis techniques allow for effective fault localization, showing an average EXAM score below 0.30 (less than 30% of the transformation needs to be inspected). These techniques are also able to locate the faulty rule in the top-three ranked rules in 70% of all cases. The impact of the model transformation, the type of mutation and the type of contract on the results is discussed. Finally, we also investigate the cases where the technique does not work properly, including discussion of a potential pre-check to estimate the prospects of the technique for a certain transformation.

show abstract

Metamorphic Testing in Fault Localization of Model Transformations

Cited by 7 publications

References 17 publications

Model Transformation Testing and Debugging: A Survey

Model Transformation Testing and Debugging: A Survey

Revisiting Fault Localization Techniques for Model Transformations: Towards A Hybrid Approach.

Fault localization in DSLTrans model transformations by combining symbolic execution and spectrum-based analysis

Contact Info

Product

Resources

About