Metamorphic relation prioritization for effective regression testing

Srinivasan, Madhusudan; Kanewala, Upulee

doi:10.1002/stvr.1807

Cited by 9 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main objective of selection approaches is to increase the efficiency of metamorphic testing by finding the most useful set of MRs to be given priority in testing. Srinivasan and Kanewala 36 proposed an approach for MR prioritization based on fault detection information and code coverage. In fault‐based MR prioritization, previous test results are utilized to find the MRs that revealed the highest number of faults.…”

Section: Discussionmentioning

confidence: 99%

Metamorphic relation automation: Rationale, challenges, and solution directions

Altamimi

Elkawakjy

Çatal

2022

J Software Evolu Process

View full text Add to dashboard Cite

Metamorphic testing addresses the issue of the oracle problem by comparing results transformation from multiple test executions. The relationship that governs the output transformation is called metamorphic relation. Metamorphic relations require expert knowledge and the generation of them is considered a time-consuming task.Researchers have proposed various techniques to automate metamorphic testing, generation, and selection. Although there are several research articles on this issue, there is a lack of overview of the state-of-the-art of metamorphic relation automation. As such, we performed a systematic literature review study to collect, extract, and synthesize the required data. Based on our research questions, the literature was categorized and summarized into different categories. We found that the automation of metamorphic relation is most effective in mathematical and scientific applications.We concluded that some approaches involve analysis of different forms of softwarerelated information such as control flow graph and program dependence graph as well as an initial set of metamorphic relations. On the other hand, other methods involve analysis of executions of the software functions with random and specific inputs. The results show that this field is still in its infancy with opportunities for novel work, especially in methods utilizing machine learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Metamorphic relation automation: Rationale, challenges, and solution directions

Altamimi

Elkawakjy

Çatal

2022

J Software Evolu Process

View full text Add to dashboard Cite

show abstract

“…Continued on next page We explored various testing approaches applied at different levels of MLES, revealing a significant emphasis on either model-centric testing methods such as Search-based Testing [127,132,135,173], Metamorphic Testing [91,93,128,139,141,174], Mutation Testing [92,133,134,148], Model Coverage [131,136], and Adversarial Testing [145,146], or on input-focused strategies like Input Perturbation [129] and Out-Of-Distribution testing [130]. Which predominantly aim at evaluating the robustness, accuracy, and reliability of the ML models themselves.…”

mentioning

confidence: 99%

Sustainable Engineering of Machine Learning-Enabled Systems: A Systematic Mapping Study

Chadli,

Botterweck,

Saber

2024

Preprint

View full text Add to dashboard Cite

The widespread adoption of Machine Learning (ML) across various sectors presents unique challenges beyond the scope of conventional software engineering, especially throughout the lifecycle of ML-Enabled Systems (MLES). As ML becomes central to software operations, the substantial computational resources required for their training, testing, retraining, and maintenance underscore the urgent need for sustainable DevOps practices in AI-centric software ecosystems. Despite the critical importance of this subject, there remains a lack of a unified review that addresses these sustainability challenges within the ML lifecycle from a holistic perspective. This study aims to bridge the research gap by conducting a systematic mapping study of current practices and methodologies that promote sustainable MLOps. Additionally, we have mapped these techniques and methodologies across the MLOps pipeline and extracted lessons learned from each phase to enhance our understanding and application of sustainable practices in MLOps. By doing so, this paper seeks to offer insights into strategies that could mitigate the environmental, economic, technical, social, and individual sustainability challenges associated with MLES, thereby contributing to the development of more sustainable ML-Enabled Systems.

show abstract

Boosting Multimode Ruling in DHR Architecture With Metamorphic Relations

Li,

Kong,

Guo

et al. 2024

Software Testing Verif & Rel

View full text Add to dashboard Cite

The DHR architecture provides a revolutionary security defense structure for cyberspace. The multimode ruling in DHR is expected to alleviate the oracle problem, which still suffers from the existence of common model vulnerability. In this work, we design a test segmentation method to transform multimode ruling to a metamorphic testing problem. The text test input that causes inconsistency of heterogeneous executors is converted to a condition set, and we extract subsets of conditions based on its syntax tree. The original test can exploit a specific vulnerability, the follow‐up tests are composed by different subsets of conditions within the original test. We collect the execution matrix for the follow‐up tests to analyse the impact of each subset of conditions on ruling decision. Metamorphic relations are extracted based on the localization of independent condition, that is, the subsets of conditions that can impact ruling decision independently. The executors in an inconsistent ruling should be examined with metamorphic testing methods, rather than traditional majority voting mechanism. The proposed test segmentation and improved multimode ruling methods are evaluated on two DHR‐based cases, SQL injection in cyber‐range system and deserialization attack in ‐ project. The experimental results show that our test segmentation can help to locate malicious expressions and the metamorphic testing‐based multimode ruling can generate more correct results than majority voting mechanism with an average 15.8% performance loss.

show abstract

Metamorphic relation prioritization for effective regression testing

Cited by 9 publications

References 30 publications

Metamorphic relation automation: Rationale, challenges, and solution directions

Metamorphic relation automation: Rationale, challenges, and solution directions

Sustainable Engineering of Machine Learning-Enabled Systems: A Systematic Mapping Study

Boosting Multimode Ruling in DHR Architecture With Metamorphic Relations

Contact Info

Product

Resources

About