Elastic Application-Level Monitoring for Large Software Landscapes in the Cloud

Abstract

Abstract. Application-level monitoring provides valuable, detailed insights into running applications. However, many approaches often only employ a single analysis application. This analysis application may become a performance bottleneck when monitoring several programs resulting in reduced monitoring quality or violated service level agreements of the monitored applications. We present an approach for elastic, distributed application-level monitoring for large software landscapes consisting of several hundreds of applications by utilizing cloud computing. Our approach dynamically inserts and removes worker levels to circumvent overloading the analysis master application without interrupting or pausing the actual live analysis of the monitored data. To evaluate our approach, we conduct an experiment in which we generate load -following a real workload pattern -on web applications in a 24 hour experiment. In our experiment, 160 monitored JPetStore instances generate roughly 20 million analyzed method calls per second in the peak. Furthermore, two worker levels are dynamically started and removed in line with the imposed workload on the monitored applications. The experiment shows that our monitoring approach is capable of live analyzing several millions of monitored method calls per second without overloading the analysis master application.

Microservices for Scalability

“…Besides Kieker [29], our ExplorViz approach [13] provides live visualization for large software landscapes introducing three hierarchical abstractions [10]. Live visualization with ExplorViz is scalable [6] and elastic in cloud environments [28]. Monitoring may provide runtime models [23] for system comprehension [9], trace visualization [4], architecture conformance checks [11], and a landscape control center [12] with performance anomaly detection [3,24].…”

ExplorViz: Research on software visualization, comprehension and collaboration

“…Besides employing ExplorViz in research on software visualization, comprehension and collaboration, ExplorViz has also been used as a research object itself. This includes scalable and elastic processing of large volumes of monitoring data [21], application discovery [22] and migrating monolithic software systems toward microservice architectures [8,[23][24][25].…”

Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures

“…Individual processing instances only receive part of the data, meaning that scalability is usually bounded by the number of different keys. An additional factor of scaling can be obtained by executing multiple stream processing operators in parallel or with multiple elasticity levels [22]. Röger and Mayer [7] present a comprehensive survey on parallelization approaches in stream processing.…”