How is Performance Addressed in DevOps?

Abstract

DevOps is a modern so ware engineering paradigm that is gaining widespread adoption in industry. e goal of DevOps is to bring so ware changes into production with a high frequency and fast feedback cycles.is con icts with so ware quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities -such as load testing -require a considerable amount of time to get statistically signi cant results.We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps se ings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. e survey responses, which come from a wide variety of participants from di erent industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. e implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline.ACM Reference format:

Dynamically reconfiguring software microbenchmarks: reducing execution time without sacrificing result quality

“…Microbenchmarks in CI. The long benchmark execution times (see Section 3 and [24,32,45]) are a major obstacle for including microbenchmarks in CI [6]. To overcome this hurdle, a combination of our technique with benchmark selection [14], benchmark prioritization [39], and risk analysis on commits [24] would reduce the required time for microbenchmarking and potentially enable CI integration.…”

“…For functional testing, CI has been a revelation, where (unit) tests are regularly executed to detect functional regressions as early as possible [22]. However, performance testing is not yet standard CI practice, although there would be a need for it [6,36]. A major reason for not running performance tests on every commit is their long runtimes, often consuming multiple hours to days [24,26,32].…”

A Systematic Literature Review on DevOps Capabilities and Areas

“…According to Riungu-Kalliosaari et al (2016), DevOps is a set of practices intended to reduce the time between making a change to a system and this change being placed into normal production, all the while ensuring high quality. The main goal associated with this concept is to avoid common problems when operations and developers are kept as separated teams (Bezemer, Eismann, Ferme, & Grohmann, 2018).…”

A Taxonomy of Techniques for SLO Failure Prediction in Software Systems

“…A major drawback of all white-box approaches is the missing adoption in the industry, due to their inherent complexity [122]. Additionally, all approaches focusing solely on performance prediction is that they are usually able to accurately predict performance problems (i.e., time-outs or latency-spikes), but are not able to predict content failures.…”