Continuous Integration of Performance Model

Mazkatli, Manar; Koziolek, Anne

doi:10.1145/3185768.3186285

Cited by 11 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stochastic models such as queueing networks [15], layered queueing networks [14] and Petri nets [25] have been widely used to model web applications. Similarly, software architecture models are appropriate to describe changes in software components and resources [26], which can be used in conjunction with stochastic models to holistically describe the quality of service in a complex distributed system. However, choosing the right model parameters is challenging for the predictive accuracy of performance models.…”

Section: Introductionmentioning

confidence: 99%

Estimating Multiclass Service Demand Distributions Using Markovian Arrival Processes

Wang

Casale

Filieri

2023

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

Building performance models for software services in DevOps is costly and error-prone. Accurate service demand distribution estimation is critical to precisely modeling queueing behaviors and performance prediction. However, current estimation methods focus on capturing the mean service demand, disregarding higher-order moments of the distribution that still can largely affect prediction accuracy. To address this limitation, we propose to estimate higher moments of the service demand distribution for a microservice from monitoring traces. We first generate a closed queueing model to abstract software performance and use it to model the departure process of requests completed by the software service as a Markovian arrival process. This allows formulating the estimation of service demand into an optimization problem, which aims to find the first multiple moments of the service demand distribution that maximize the likelihood of the MAP using generated the measured inter-departure times. We then estimate the service demand distribution for different classes of service with a maximum likelihood algorithm and novel heuristics to mitigate the computational cost of the optimization process for scalability. We apply our method to real traces from a microservice-based application and demonstrate that its estimations lead to greater prediction accuracy than exponential distributions assumed in traditional service demand estimation approaches for software services.

show abstract

Section: Introductionmentioning

confidence: 99%

Estimating Multiclass Service Demand Distributions Using Markovian Arrival Processes

Wang

Casale

Filieri

2023

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

show abstract

“…Stochastic models such as queueing networks [14], layered queueing networks [13], Petri nets [20] are widely used to represent web applications. Instead, software architecture models are appropriate to describe changes in software structures and resources [21].…”

Section: Introductionmentioning

confidence: 99%

Service Demand Distribution Estimation for Microservices Using Markovian Arrival Processes

Wang

Casale

Filieri

2021

Quantitative Evaluation of Systems

View full text Add to dashboard Cite

Building performance models for microservices applications in De-vOps is costly and error-prone. Accurate service demand distribution estimation is critical to performance model parameterization. However, traditional service demand estimation methods focus on capturing the mean service demand, disregarding higher-order moments of the distribution. To address this limitation, we propose to estimate higher moments of the service demand distribution for a microservice from monitoring traces. We first generate a closed queueing model to abstract a microservice and model the departure process at the queue node as a Markovian arrival process. This allows formulating the estimation of service demand as an optimization problem, which aims to find the optimal parameters of the first multiple moments of the service demand distribution based on the interdeparture times. We then estimate the service demand distribution with a novel maximum likelihood algorithm, and heuristics to mitigate the computational cost of the optimization process for scalability. We apply our method to real traces from a microservice-based application and demonstrate that its estimations lead to greater prediction accuracy than exponential distributions assumed in traditional service demand estimation approaches.

show abstract

“…The goal of the approach is to keep the PM up-to-date automatically to allow AbPP. An initial version of this idea has been presented in a workshop publication [37] without evaluation.…”

Section: Introductionmentioning

confidence: 99%

Incremental Calibration of Architectural Performance Models with Parametric Dependencies

Mazkatli

Monschein

Grohmann

et al. 2020

2020 IEEE International Conference on Software Architecture (ICSA)

Self Cite

View full text Add to dashboard Cite

Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary.Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating.In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly.For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.

show abstract

Continuous Integration of Performance Model

Cited by 11 publications

References 11 publications

Estimating Multiclass Service Demand Distributions Using Markovian Arrival Processes

Estimating Multiclass Service Demand Distributions Using Markovian Arrival Processes

Service Demand Distribution Estimation for Microservices Using Markovian Arrival Processes

Incremental Calibration of Architectural Performance Models with Parametric Dependencies

Contact Info

Product

Resources

About