METRIC: METamorphic Relation Identification based on the Category-choice framework

Chen, Tsong Yueh; Poon, Pak-Lok; Xie, Xue‐Jun

doi:10.1016/j.jss.2015.07.037

Cited by 66 publications

(45 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The empirical evidence is promising; people's performance with respect to MR identification improved, and novices achieved a comparable level of performance to experts (Chen et al 2016). However, the technique has an important limitation; it can currently only support MRs that are composed of one source and follow up test case (Chen et al 2016).…”

Section: Effortmentioning

confidence: 99%

“…Therefore, if one lacks adequate domain knowledge, it is advisable to consult domain experts (Liu et al 2012). MRs are identified in booms and slumps; the SUT is investigated during a slump to develop new intuitions that can be used to identify MRs, and such MRs are defined in boom periods (Chen et al 2016). This iterative process affords further opportunities to continuously supplement one's domain knowledge.…”

Section: Prerequisite Skills and Knowledgementioning

confidence: 99%

“…For example, Chen et al (2016) found that MR identification is difficult because inputs and outputs must be considered simultaneously (Chen et al 2016). They alleviated this by automating input analysis, thereby constraining the tester's attention to outputs (Chen et al 2016). The technique specifies a set of characteristics, called "Categories"; each is associated with inputs that manipulate it.…”

Section: Effortmentioning

confidence: 99%

“…These test frames produce test cases that are executed to obtain a set of outputs, which can be manually checked for relationships. The process of automatically generating test frames and manually analysing them is iterative (Chen et al 2016); since an infeasible number of pairs typically exist, the terminating condition is the tester's satisfaction with the identified pool of MRs (Chen et al 2016).…”

Section: Effortmentioning

confidence: 99%

See 3 more Smart Citations

A mapping study on testing non-testable systems

Patel

Hierons

2017

Software Qual J

View full text Add to dashboard Cite

The terms "Oracle Problem" and "Non-testable system" interchangeably refer to programs in which the application of test oracles is infeasible. Test oracles are an integral part of conventional testing techniques; thus, such techniques are inoperable in these programs. The prevalence of the oracle problem has inspired the research community to develop several automated testing techniques that can detect functional software faults in such programs. These techniques include N-Version testing, Metamorphic Testing, Assertions, Machine Learning Oracles, and Statistical Hypothesis Testing. This paper presents a Mapping Study that covers these techniques. The Mapping Study presents a series of discussions about each technique, from different perspectives, e.g. effectiveness, efficiency, and usability. It also presents a comparative analysis of these techniques in terms of these perspectives. Finally, potential research opportunities within the non-testable systems problem domain are highlighted within the Mapping Study. We believe that the aforementioned discussions and comparative analysis will be invaluable for new researchers that are attempting to familiarise themselves with the field, and be a useful resource for practitioners that are in the process of selecting an appropriate technique for their context, or deciding how to apply their selected technique. We also believe that our own insights, which are embedded throughout these discussions and the comparative analysis, will be useful for researchers that are already accustomed to the field. It is our hope that the potential research opportunities that have been highlighted by the Mapping Study will steer the direction of future research endeavours.

show abstract

Section: Effortmentioning

confidence: 99%

Section: Prerequisite Skills and Knowledgementioning

confidence: 99%

Section: Effortmentioning

confidence: 99%

Section: Effortmentioning

confidence: 99%

See 2 more Smart Citations

A mapping study on testing non-testable systems

Patel

Hierons

2017

Software Qual J

View full text Add to dashboard Cite

show abstract

“…s i , s o and this intuition form an IOR. Tools that partially automate the exploration of relationships between inputs and outputs may simplify this task [6]. One must then leverage one's knowledge about the SUT's implementation details to identify execution trace behaviours that should manifest in executions in which this IOR is satisfied.…”

Section: Applying Imtmentioning

confidence: 99%

Resolving the Equivalent Mutant Problem in the Presence of Non-determinism and Coincidental Correctness

Patel

Hierons

2016

Testing Software and Systems

View full text Add to dashboard Cite

Abstract. In this paper, we develop a new mutation testing technique called Interlocutory Mutation Testing (IMT) that mitigates the equivalent mutant problem in the presence of coincidental correctness and non-determinism. The accuracy of IMT was evaluated; it obtained a classification accuracy of 93.33% for non-equivalent mutants and 100% for equivalent mutants in a non-deterministic system with coincidental correctness.

show abstract

Supporting maintenance and testing for AI functions of mobile apps based on user reviews: An empirical study on plant identification apps

Tao

Guo

Zhang

et al. 2022

J Software Evolu Process

View full text Add to dashboard Cite

Despite the tremendous development of artificial intelligence (AI)‐based mobile apps, they suffer from quality issues. Data‐driven AI software poses challenges for maintenance and quality assurance. Metamorphic testing has been successfully adopted to AI software. However, most previous studies require testers to manually identify metamorphic relations in an ad hoc and arbitrary manner, thereby encountering difficulties in reflecting real‐world usage scenarios. Previous work showed that information available in user reviews is effective for maintenance and testing tasks. Yet, there is a lack of studies leveraging reviews to facilitate AI function maintenance and testing activities. This paper proposes METUR, a novel approach to supporting maintenance and testing for AI functions based on reviews. Firstly, METUR automatically classifies reviews that can be exploited for supporting AI function maintenance and evolution activities. Then, it identifies test contexts from reviews in the usage scenario category. METUR instantiates the metamorphic relation pattern for deriving concrete metamorphic relations based on test contexts. The follow‐up test dataset is constructed for conducting metamorphic testing. Empirical studies on plant identification apps indicate that METUR effectively categorizes reviews that are related to AI functions. METUR is feasible and effective in detecting inconsistent behaviors by using the metamorphic relations constructed based on reviews.

show abstract

METRIC: METamorphic Relation Identification based on the Category-choice framework

Cited by 66 publications

References 26 publications

A mapping study on testing non-testable systems

A mapping study on testing non-testable systems

Resolving the Equivalent Mutant Problem in the Presence of Non-determinism and Coincidental Correctness

Supporting maintenance and testing for AI functions of mobile apps based on user reviews: An empirical study on plant identification apps

Contact Info

Product

Resources

About