A Metamorphic Approach to Integration Testing of Context-Sensitive Middleware-Based Applications

Chan, W. K.; Chen, Tsong Yueh; Lü, Haishen; Tse, T. H.; Yau, Sik-Sang

doi:10.1109/qsic.2005.3

Cited by 26 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results of the evaluation of the metamorphic relations using manual mutation testing were reported. In [9], Chan et al presented a metamorphic approach for integration testing in context-sensitive middleware-based applications. The authors identified functional relations that associate different execution sequences of a test case.…”

Section: Related Workmentioning

confidence: 99%

Automated metamorphic testing on the analyses of feature models

Segura¹,

Hierons²,

Benavides³

et al. 2011

Information and Software Technology

View full text Add to dashboard Cite

Copyright © 2010 Elsevier B.V. All rights reserved.Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem.Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach.Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively.Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency.Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.This work has been partially supported by the European Commission(FEDER)and Spanish Government under CICYT project SETI(TIN2009-07366)and the Andalusian Government project ISABEL(TIC-2533)

show abstract

Section: Related Workmentioning

confidence: 99%

Automated metamorphic testing on the analyses of feature models

Segura¹,

Hierons²,

Benavides³

et al. 2011

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…Tse et al [2004] and Chan et al [2006] explored new ways to verify test results based on metamorphic relationships derived from context-aware applications. Lu et al [2006] proposed to generalize the concept of "data definition" to "data update" to model that contexts can be sensed (updated) directly by the pervasive environment rather than just by program statements.…”

Section: Pervasive Computing and Context Awarenessmentioning

confidence: 99%

Partial constraint checking for context consistency in pervasive computing

Cheung

Chan

et al. 2010

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Pervasive computing environments typically change frequently in terms of available resources and their properties. Applications in pervasive computing use contexts to capture these changes and adapt their behaviors accordingly. However, contexts available to these applications may be abnormal or imprecise due to environmental noises. This may result in context inconsistencies, which imply that contexts conflict with each other. The inconsistencies may set such an application into a wrong state or lead the application to misadjust its behavior. It is thus desirable to detect and resolve the context inconsistencies in a timely way. One popular approach is to detect context inconsistencies when contexts breach certain consistency constraints. Existing constraint checking techniques recheck the entire expression of each affected consistency constraint upon context changes. When a changed context affects only a constraint's subexpression, rechecking the entire expression can adversely delay the detection of other context inconsistencies. This article proposes a rigorous approach to identifying the parts of previous checking results that are reusable without entire rechecking. We evaluated our work on the Cabot middleware through both simulation experiments and a case study. The experimental results reported that our approach achieved over a fifteenfold performance improvement on context inconsistency detection than conventional approaches.

show abstract

“…Noting that the output of a test case for pervasive applications can be too transient to record, Chan et al [6] advocated the use of metamorphic relations among different contexts to address the test oracle problem. Observing that layered architecture disseminated the complete application logic of pervasive software into multiple tiers, Lu et al [18] [19] proposed to assemble context-aware entities (including adaptation rules and context management components such as CIR services) into a traditional control-flow-graph model and developed new coverage-based testing criteria to dynamically verify the definition and use of variables in pervasive software.…”

Section: Related Workmentioning

confidence: 99%

Correlating Context-Awareness and Mutation Analysis for Pervasive Computing Systems

Huai

Zhai

Tse

2010

2010 10th International Conference on Quality Software

View full text Add to dashboard Cite

Pervasive computing systems often use middleware as a means to communicate with the changing environment. However, the interactions with the context-aware middleware as well as the interactions among applications sharing the same middleware may introduce faults that are difficult to reveal by existing testing techniques. Our previous work proposed the notion of context diversity as a metric to measure the degree of changes in test inputs for pervasive software. In this paper, we present a case study on how much context diversity for test cases relates to fault-based mutants in pervasive software. Our empirical results show that conventional mutation operators can generate sufficient candidate mutants to support test effectiveness evaluation of pervasive software, and test cases with higher context diversity values tend to have higher mean mutation scores. On the other hand, for test cases sharing the same context diversity, their mutation scores can vary significantly in terms of standard derivations.

show abstract

A Metamorphic Approach to Integration Testing of Context-Sensitive Middleware-Based Applications

Abstract: Abstract

Cited by 26 publications

References 10 publications

Automated metamorphic testing on the analyses of feature models

Automated metamorphic testing on the analyses of feature models

Partial constraint checking for context consistency in pervasive computing

Correlating Context-Awareness and Mutation Analysis for Pervasive Computing Systems

Contact Info

Product

Resources

About