A Proposal to Orchestrate Test Cases

García, Boni; Lonetti, Francesca; Gallego, Micael; Miranda, Breno; Jiménez, Eduardo García; Angelis, Guglielmo De; Santos, Carlos; Marchetti, Eda

doi:10.1109/quatic.2018.00016

Cited by 7 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Unlike the previous works, our article proposes to optimize the resources not only focused on time or cost but also on other resources used during the testing based on both the test dependencies and resource usage. García et al (Garcia et al, 2018) also propose to orchestrate the test cases through a proper selection and sequencing based on the outcome of test execution (verdict-driven) or on the produced output (datadriven).…”

Section: Test Resource Optimizationmentioning

confidence: 99%

RETORCH: an approach for resource-aware orchestration of end-to-end test cases

et al. 2020

View full text Add to dashboard Cite

Continuous integration practice mandates to continuously introduce incremental changes into code, but doing so may introduce new faults too. These faults could be detected automatically through regression testing, but this practice becomes prohibitive as the cost of executing the tests grows. This problem is preponderant in End-to-End testing where the whole system is requested for test execution. However, some of these test cases could be executed with fewer resources (e.g. memory, web services, computation, Cloud instances, among others), by deploying only the subsystems needed by each test. This paper is focused on the optimization of the resources employed in End-to-End testing by means of a resource-aware test orchestration technique in the context of continuous integration practices in the Cloud. The RETORCH approach proposes a novel way to identify the resources required by End-to-End test cases and to use this information to group together those tests requiring equivalent resources. Besides, the approach proposes to deploy the grouped tests in isolated and elastic environments, so that their execution can be scheduled in parallel on several machines. RETORCH is exemplified with a real-world application and its performance evaluation shows promising savings in terms of resource usage and time.

show abstract

Section: Test Resource Optimizationmentioning

confidence: 99%

RETORCH: an approach for resource-aware orchestration of end-to-end test cases

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This work contributes to both those research directions by leveraging the concept of test case orchestration [15], which refers to all those activities that dynamically organize the execution of tests. Test case orchestration enables the declaration of flows of test cases, and provides a means for running existing test cases in different operational contexts: previous test results can dynamically instantiate current parameters or modify the execution environment.…”

Section: Regression Testing In Sossmentioning

confidence: 99%

“…As detailed in [15], verdict-driven orchestration is helpful in order to structure conditional chains of test cases executions by reasoning on the logical combination of the observed outcomes. In addition to such an aspect, data-driven orchestration also enables the possibility to interconnect test cases: the output data of each test can be used to feed the next test in the execution chain.…”

Section: Fig 1 a Framework For Governing Sos Regression Testingmentioning

confidence: 99%

See 1 more Smart Citation

Governing Regression Testing in Systems of Systems

Bertolino

Angelis

Lonetti

2019

2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)

Self Cite

View full text Add to dashboard Cite

Great advances in network technology and software engineering have triggered the development and spread of Systems of Systems (SoSs). The dynamic and evolvable nature of SoSs poses important challenges on the validation of such systems and in particular on their regression testing, aiming at assessing that run-time changes and evolutions do not introduce regression in SoS behavior. This paper outlines issues and challenges of regression testing of SoSs, identifying the main kinds of evolution that can impact on their regression testing activity. Furthermore, it presents a conceptual framework for governing the regression testing of SoSs. The proposed framework leverages the concept of an orchestration graph that describes the flow of test cases and sketches a solution for deriving a regression test plan according to test cases dependencies.

show abstract

“…Unlike those works, we understand as a resource a wide range of terms beyond time or cost, and aim to optimize the testing depending on several aspects. A similar aim is pursued by García et al [17], who propose to orchestrate test cases (essentially to find a proper selection and sequencing) depending on the outcome of test execution (verdict-driven) or on the produced output (data-driven).…”

Section: -29238-6_22mentioning

confidence: 99%

RETORCH: Resource-Aware End-to-End Test Orchestration

Augusto¹,

Morán²,

Bertolino³

et al. 2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

Continuous integration practices introduce incremental changes in the code to both improve the quality and add new functionality. These changes can introduce faults that can be timely detected through continuous testing by automating the test cases and re-executing them at each code change. However, reexecuting all test cases at each change may not be always feasible, especially for those test cases that make heavy use of resources thoroughly like End-to-End test cases that need a complex test infrastructure. This paper is focused on optimizing the usage of the resources employed during End-to-End testing (e.g., storage, memory, web servers or tables of a database, among others) through a resourceaware test orchestration technique in the context of continuous integration in the cloud. In order to optimize both the cost/usage of resources and the execution time, the approach proposes to i) identify the resources required by the End-to-End test cases, ii) group together those tests that need the same resources, iii) deploy the tests in both dependency isolated and elastic environments, and iv) schedule their parallel execution in several machines.

show abstract

A Proposal to Orchestrate Test Cases

Cited by 7 publications

References 15 publications

RETORCH: an approach for resource-aware orchestration of end-to-end test cases

RETORCH: an approach for resource-aware orchestration of end-to-end test cases

Governing Regression Testing in Systems of Systems

RETORCH: Resource-Aware End-to-End Test Orchestration

Contact Info

Product

Resources

About