Performance regression testing of concurrent classes

Pradel, Michael; Huggler, Markus; Groß, Thomas

doi:10.1145/2610384.2610393

Cited by 69 publications

(31 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, performance testing employs statistical methods to detect difference in performance of two versions of the software [9], [6], [10]. However, these techniques are solely focused on detecting performance degradations.…”

Section: Related Workmentioning

confidence: 99%

CanaryAdvisor: a statistical-based tool for canary testing (demo)

Tarvo¹,

Sweeney²,

Mitchell³

et al. 2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Canary testing is an emerging technique that offers to minimize the risk of deploying a new version of software. It does so by slowly transferring load from the current to the new ("canary") version. As this ramp-up progresses, a human compares the performance and correctness of the two versions, and assesses whether to abort the canary version. For canary testing to be effective, a plethora of metrics must be analyzed, including CPU utilization and logged errors, across hundreds to thousands of machines. Performing this analysis manually is both time consuming and error prone.In this paper, we present CanaryAdvisor, a tool for automatic canary testing of cloud-based applications. CanaryAdvisor continuously monitors the deployed versions of an application and detects degradations in correctness, performance, and/or scalability. We describe our design and implementation of the CanaryAdvisor and outline open challenges.

show abstract

Section: Related Workmentioning

confidence: 99%

CanaryAdvisor: a statistical-based tool for canary testing (demo)

Tarvo¹,

Sweeney²,

Mitchell³

et al. 2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…Algorithmic denial of service [7] is a class of attacks that attempts to reduce the performance of a server by repeatedly sending data that exposes bad performance. SpeedGun [31] is a technique for automated performance regression testing of thread-safe classes. Our work shares the idea of generating input to trigger performance problems.…”

Section: Related Workmentioning

confidence: 99%

“…Our work shares the idea of generating input to trigger performance problems. EventBreak differs from [5], [7], and [31] by generating sequences of events instead of input data. Grechanik et al [13] describe a strategy for selecting test cases that may expose performance problems, assuming to have a large set of existing test inputs.…”

Section: Related Workmentioning

confidence: 99%

EventBreak

et al. 2014

Self Cite

View full text Add to dashboard Cite

Event-driven user interface applications typically have a single thread of execution that processes event handlers in response to input events triggered by the user, the network, or other applications. Programmers must ensure that event handlers terminate after a short amount of time because otherwise, the application may become unresponsive. This paper presents EventBreak, a performance-guided test generation technique to identify and analyze event handlers whose execution time may gradually increase while using the application. The key idea is to systematically search for pairs of events where triggering one event increases the execution time of the other event. For example, this situation may happen because one event accumulates data that is processed by the other event. We implement the approach for JavaScriptbased web applications and apply it to three real-world applications. EventBreak discovers events with an execution time that gradually increases in an unbounded way, which makes the application unresponsive, and events that, if triggered repeatedly, reveal a severe scalability problem, which makes the application unusable. The approach reveals two known bugs and four previously unknown responsiveness problems. Furthermore, we show that EventBreak helps in testing that event handlers avoid such problems by bounding a handler's execution time.

show abstract

“…When concerning the performance of the concurrent program, some authors (for example, in [16]) analyze mostly the speed of execution of the program. However, there is a number of different metrics that can be used as an objective function for the evaluation of the performance of the system.…”

Section: B Aspects Of Efficiency (Main Outputs Of the Framework)mentioning

confidence: 99%

Specification of the Framework for Evaluating the Efficiency of Mechanisms for Concurrent Processes

Nazaruk

Rusakov

2014

Applied Computer Systems

View full text Add to dashboard Cite

The goal of this paper is to give a specification of the software framework that evaluates the efficiency of different mechanisms for concurrent processes, notably process synchronization mechanisms. The paper discusses the concept of this framework, the potential users of it and some necessary considerations, including assumptions. Further, it defines general requirements for the framework and presents its desired conceptual design. The conclusions and possible directions for future work end the paper.

show abstract

Performance regression testing of concurrent classes

Cited by 69 publications

References 72 publications

CanaryAdvisor: a statistical-based tool for canary testing (demo)

CanaryAdvisor: a statistical-based tool for canary testing (demo)

EventBreak

Specification of the Framework for Evaluating the Efficiency of Mechanisms for Concurrent Processes

Contact Info

Product

Resources

About