Estimating the Performance of Cloud-Based Systems Using Benchmarking and Simulation in a Complementary Manner

Johng, Haan; Kim, Doo-Hwan; Hill, Tom; Chung, Lawrence

doi:10.1007/978-3-030-03596-9_42

Cited by 4 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To validate simulated results, complementary approaches like benchmarking and monitoring a system emerged [52,111]. JOHNG and others [111], for example, developed an ontology based methodology where a mapping function between the different ontologies tries to compare the environments to achieve a closer relation between development and production environments. Execution of benchmarks is necessary for their approach to calibrate the simulation.…”

Section: Cloud Simulationmentioning

confidence: 99%

“…To achieve dev-prod parity in FaaS, a different approach is necessary. The work on a combined benchmark and simulation approach [111] inspired some aspects of this work. Their use case was on finding proper VM configurations by extending CloudSim [36].…”

Section: Competing Approachesmentioning

confidence: 99%

“…The restrictions on some target settings motivated us to collect calibration data and compute linear regression models to calculate resource configurations for arbitrary settings. Nevertheless, JOHNG and others [111] are still capable of running simulations for different influential settings to find a proper VM configuration based on their requirements. This approach is the most closely related one from a conceptual point of view, but deals with applications, not solely functions, and does still not provide proper configurations for arbitrary simulation settings.…”

Section: Competing Approachesmentioning

confidence: 99%

“…Especially at early publications, the serverless framework was often used also by other papers included in this SLR, like [6,139,167,189]. Nevertheless, the benefits over generic IaC tools like Terraform 111 or provider specific ones like AWS CloudFormation 112 are not clear in the aforementioned papers. Noteworthy in this last group of papers is the effort of USTIUGOV and others [270].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Simulation Framework for Function as a Service

Manner

2024

Schriften Aus Der Fakultät Wirtschaftsinformatik Und Angewandte Informatik Der Otto-Friedrich-Universität Bamberg

View full text Add to dashboard Cite

Serverless Computing is seen as a game changer in operating large-scale applications. While practitioners and researches often use this term, the concept they actually want to refer to is Function as a Service (FaaS). In this new service model, a user deploys only single functions to cloud platforms where the cloud provider deals with all operational concerns – this creates the notion of server-less computing for the user. Nonetheless, a few configurations for the cloud function are necessary for most commercial FaaS platforms as they influence the resource assignments like CPU time and memory. Due to these options, there is still an abstracted perception of servers for the FaaS user. The resource assignment and the different strategies to scale resources for public cloud offerings and on-premise hosted open-source platforms determine the runtime characteristics of cloud functions and are in the focus of this work. Compared to cloud offerings like Platform as a Service, two out of the five cloud computing characteristics improved. These two are rapid elasticity and measured service. FaaS is the first computational cloud model to scale functions only on demand. Due to an independent scaling and a strong isolation via virtualized environments, functions can be considered independent of other cloud functions. Therefore, noisy neighbor problems do not occur. The second characteristic, measured service, targets billing. FaaS platforms measure execution time on a millisecond basis and bill users accordingly based on the function configuration. This leads to new performance and cost trade-offs. Therefore, this thesis proposes a simulation approach to investigate this tradeoff in an early development phase. The alternative would be to deploy functions with varying configurations, analyze the execution data from several FaaS platforms and adjust the configuration. However, this alternative is time-consuming, tedious and costly. To provide a proper simulation, the development and production environment should be as similar as possible. This similarity is also known as dev-prod parity. Based on a new methodology to compare different virtualized environments, users of our simulation framework are able to execute functions on their machines and investigate the runtime characteristics for different function configurations at several cloud platforms without running their functions on the cloud platform at all. A visualization of the local simulations guide the user to choose an appropriate function configuration to resolve the mentioned trade-off dependent on their requirements.

show abstract

Section: Cloud Simulationmentioning

confidence: 99%

Section: Competing Approachesmentioning

confidence: 99%

Section: Competing Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Simulation Framework for Function as a Service

Manner

2024

Schriften Aus Der Fakultät Wirtschaftsinformatik Und Angewandte Informatik Der Otto-Friedrich-Universität Bamberg

View full text Add to dashboard Cite

show abstract

“…Besides monitoring individual service calls, it is important to predict the runtime performance of distributed systems. Johng et al [11] show that two techniques, benchmarking and simulation, have shortcomings if they are used sep-arately and introduce and validate a complementary approach. Their approach presents a process which maps benchmark ontologies of simulations.…”

Section: Related Workmentioning

confidence: 99%

On Observability and Monitoring of Distributed Systems – An Industry Interview Study

Niedermaier

Koetter

Freymann

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Business success of companies heavily depends on the availability and performance of their client applications. Due to modern development paradigms such as DevOps and microservice architectural styles, applications are decoupled into services with complex interactions and dependencies. Although these paradigms enable individual development cycles with reduced delivery times, they cause several challenges to manage the services in distributed systems. One major challenge is to observe and monitor such distributed systems. This paper provides a qualitative study to understand the challenges and good practices in the field of observability and monitoring of distributed systems. In 28 semi-structured interviews with software professionals we discovered increasing complexity and dynamics in that field. Especially observability becomes an essential prerequisite to ensure stable services and further development of client applications. However, the participants mentioned a discrepancy in the awareness regarding the importance of the topic, both from the management as well as from the developer perspective. Besides technical challenges, we identified a strong need for an organizational concept including strategy, roles and responsibilities. Our results support practitioners in developing and implementing systematic observability and monitoring for distributed systems.

show abstract