Identifying Root Causes of Web Performance Degradation Using Changepoint Analysis

Abstract

Abstract. The large scale of the Internet has offered unique economic opportunities, that in turn introduce overwhelming challenges for development and operations to provide reliable and fast services in order to meet the high demands on the performance of online services. In this paper, we investigate how performance engineers can identify three different classes of externally-visible performance problems (global delays, partial delays, periodic delays) from concrete traces. We develop a simulation model based on a taxonomy of root causes in server performance degradation. Within an experimental setup, we obtain results through synthetic monitoring of a target Web service, and observe changes in Web performance over time through exploratory visual analysis and changepoint detection. Finally, we interpret our findings and discuss various challenges and pitfalls.

The making of cloud applications: an empirical study on software development for the cloud

“…However, our interview partners also mentioned the additional cost involved of spinning up environments as a reason for first attempting to reproduce issues locally. Some interviewees also stated that proper end-toend monitoring [27] and request tracing needs to be in place to be able to reproduce issues correctly:…”

“…However, our interview partners also mentioned the additional cost involved of spinning up environments as a reason for first attempting to reproduce issues locally. Some interviewees also stated that proper end-toend monitoring [27] and request tracing needs to be in place to be able to reproduce issues correctly: "If you missed some logs and the instance is already gone, have fun reproducing your environment" -D8P aaS Especially with public cloud providers, hardware characteristics also need to be tracked, as you never know what specific hardware configuration will be served.…”

Runtime metric meets developer - Building better cloud applications using feedback

“…The null hypothesis, H 0 , represents no changepoint and the alternative hypothesis, H 1 , represents existing changepoints. In previous work, we have already shown that changepoint analysis can be successfully employed to detect significant changes in the evolution of operations data [11]. …”

Bifrost

“…As discussed in Section 3.1, a core property of Bifrost is that release decisions are driven by runtime data. Hence, our proposed middleware can build on previous research on application performance management, such as Kieker [26] or our own previous work on the monitoring and management of Web application performance [6,7]. Bakshy and Frachtenberg [1] have recently presented work on statistical methods to identify performance regressions in scale-out cloud systems, based on their experience at Facebook.…”