Reliability estimation for large distributed software systems

Avritzer, Alberto; Duarte, Flávio P.; Leão, Rosa M. M.; Silva, Edmundo de Souza e; Cohen, M.; Costello, David

doi:10.1145/1463788.1463804

Cited by 8 publications

(6 citation statements)

References 14 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CPU and memory usage cannot be too high during the course of a load test, otherwise the request processing can hang and system performance can be unstable [173]. Another example is the reliability requirement for safety critical and telecommunication systems [10], [85]. The reliability requirements are usually specified as "three-nines" or "five-nines", which means the system reliability cannot be lower than 99.9% (for "three-nines") and 99.999% (for "five-nines").…”

Section: Results Of a Load Test Analyzed?mentioning

confidence: 99%

“…Finally, we include relevant papers that cite these papers, based on the "Cited by" feature from Microsoft Academic Search [82], Google Scholar [81], ACM Portal [83] and IEEE Explore [84]. For example, papers like [85], [86] are included, because they cite [23] and [87], respectively. In the end, we have surveyed a total of 147 papers and tools between the year 1993 − 2013.…”

Section: Our Paper Selection Processmentioning

confidence: 99%

“…However, if the size of traces becomes large, the load driver might not be able to handle traces. Similarity, testing load derived from deterministic state testing [10], [85] is not easily realized in existing load drivers, either.…”

Section: • Encoding Testing Loads Into Testing Toolsmentioning

confidence: 99%

“…Mission critical systems usually have very strict reliability requirements. Avritzer et al [23], [85] use the Bayesian Network to estimate the system reliability from the load test data. Avritzer et al use the failure probability of each type of load (workload mix and workload intensity) and the likelihood of these types of load occurring in the field.…”

Section: Comparing Against Derived Datamentioning

confidence: 99%

See 3 more Smart Citations

A Survey on Load Testing of Large-Scale Software Systems

Jiang

Hassan²

2015

IIEEE Trans. Software Eng.

145

View full text Add to dashboard Cite

Many large-scale software systems must service thousands or millions of concurrent requests. These systems must be load tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In this paper, we survey the state of load testing research and practice. We compare and contrast current techniques that are used in the three phases of a load test: (1) designing a proper load, (2) executing a load test, and (3) analyzing the results of a load test. This survey will be useful for load testing practitioners and software engineering researchers with interest in the load testing of large-scale software systems.

show abstract

Section: Results Of a Load Test Analyzed?mentioning

confidence: 99%

Section: Our Paper Selection Processmentioning

confidence: 99%

Section: • Encoding Testing Loads Into Testing Toolsmentioning

confidence: 99%

Section: Comparing Against Derived Datamentioning

confidence: 99%

See 2 more Smart Citations

A Survey on Load Testing of Large-Scale Software Systems

Jiang

Hassan²

2015

IIEEE Trans. Software Eng.

145

View full text Add to dashboard Cite

show abstract

“…Machine learning has been widely used for the analysis of data resulting from performance testing and also for performance preservation. For example, anomaly detection through analysis of performance data, e.g., resource usage, using clustering techniques (Syer et al 2011), predicting reliability from the testing data using Bayesian Networks (Avritzer et al 2008), performance signature identification based on performance data analysis using supervised and unsupervised learning techniques (Malik et al 2013;Malik et al 2010), and also adaptive RL-driven performance in particular response time control for cloud services (Ibidunmoye et al 2017;Veni and Bhanu 2016;Jamshidi et al 2016) and also software on other execution platforms, e.g., PLCbased real-time systems (Moghadam et al 2018). Machine learning has been also applied to the generation of performance test cases in some studies.…”

Section: Related Workmentioning

confidence: 99%

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case-based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments in a simulated performance testing setup, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process and performs adaptively without access to source code and performance models.

show abstract