Verification of Microservices Using Metamorphic Testing

Luo, Gang; Zheng, Xi; Liu, Huai; Xu, Rongbin; Nagumothu, Dinesh; Janapareddi, Ranjith Nagendra Kumar; Zhuang, Er; Liu, Xiao

doi:10.1007/978-3-030-38991-8_10

Cited by 2 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ChaT is motivated by MT that removes the burden of determining software oracles in various application domains [6]. Luo et al proposed simple uses of the verification technique on microservices using MT and they proved the effectiveness of the technique in detecting failures using the defined MRs [7]. ChaT enhances the proposal by not only detecting and validating the failures using MRs, but also using some anomaly detection algorithms.…”

Section: Related Workmentioning

confidence: 99%

ChaT: Evaluation of Reconfigurable Distributed Network Systems Using Metamorphic Testing

Pranata

Barais

Bourcier

et al. 2021

2021 IEEE Global Communications Conference (GLOBECOM)

View full text Add to dashboard Cite

Detecting faults in distributed network systems is challenging because of their complexity, but this is required to evaluate and improve their reliability. This paper proposes ChaT, a testing and evaluation methodology under system reconfigurations and perturbations for distributed network systems, to evaluate QoS reliability by discriminating safe and failure-prone behaviors from different testing scenarios. Motivated by metamorphic testing technique that removes the burden of defining software oracles, we propose some metamorphic relationships that correlate system inputs and outputs to find patterns in executions. Classification techniques based on machine learning (principal component analysis and support vector machine) are used to identify system states and validate the proposed metamorphic relationships. These metamorphic relationships are also used to help anomaly detection. We verify this with several anomaly detection techniques (isolation forest, one-class SVM, local outlier factor, and robust covariance) that categorize experiments belonging to either safe or failure-prone states. We apply ChaT to a video streaming application use case. The simulation results show the effectiveness of ChaT to achieve our goals: identifying execution classes and detecting failure-prone experiments based on metamorphic relationships with high level of statistical scores.

show abstract

Section: Related Workmentioning

confidence: 99%

ChaT: Evaluation of Reconfigurable Distributed Network Systems Using Metamorphic Testing

Pranata

Barais

Bourcier

et al. 2021

2021 IEEE Global Communications Conference (GLOBECOM)

View full text Add to dashboard Cite

show abstract

A declarative approach and benchmark tool for controlled evaluation of microservice resiliency patterns

Aderaldo,

Costa,

Vasconcelos

et al. 2024

Softw Pract Exp

View full text Add to dashboard Cite

Microservice developers increasingly use resiliency patterns such as Retry and Circuit Breaker to cope with remote services that are likely to fail. However, there is still little research on how the invocation delays typically introduced by those resiliency patterns may impact application performance under varying workloads and failure scenarios. This article presents a novel approach and benchmark tool for experimentally evaluating the performance impact of existing resiliency patterns in a controlled setting. The main novelty of this approach resides in the ability to declaratively specify and automatically generate multiple testing scenarios involving different resiliency patterns, which one can implement using any programming language and resilience library. The article illustrates the benefits of the proposed approach and tool by reporting on an experimental study of the performance impact of the Retry and Circuit Breaker resiliency patterns in two mainstream programming languages (C# and Java) using two popular resilience libraries (Polly and Resilience4j), under multiple service workloads and failure rates. Our results show that, under low to moderate failure rates, both resiliency patterns effectively reduce the load over the application's target service with barely any impact on the application's performance. However, as the failure rate increases, both patterns significantly degrade the application's performance, with their effect varying depending on the service's workload and the patterns' programming language and resilience library.

show abstract

Verification of Microservices Using Metamorphic Testing

Cited by 2 publications

References 27 publications

ChaT: Evaluation of Reconfigurable Distributed Network Systems Using Metamorphic Testing

ChaT: Evaluation of Reconfigurable Distributed Network Systems Using Metamorphic Testing

A declarative approach and benchmark tool for controlled evaluation of microservice resiliency patterns

Contact Info

Product

Resources

About