Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud

Gunasekaran, Jashwant Raj; Thinakaran, Prashanth; Kandemir, Mahmut; Urgaonkar, Bhuvan; Kesidis, George; Das, Chita R.

doi:10.1109/cloud.2019.00043

Cited by 55 publications

(41 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The optimal deployment scenario is therefore highly dependent on the expected load pattern. The option of a hybrid deployment requires the function to be developed in a way that it can be executed both on FaaS and on VMaaS and an appropriate load balancer has to be in place, as discussed for example in [8]. Running the respective simulation in the FAASSIMULATOR can provide an insight into the possible cost savings which can facilitate the decision if a hybrid deployment is worth the additional effort of implementing the hybrid deployment option, or if a pure VMaaS or pure FaaS deployment is the better choice.…”

Section: Discussionmentioning

confidence: 99%

“…As the related work has demonstrated, e.g. [8], [9], and we discuss further in the following section, this is relatively easily achievable by means of existing tooling.…”

Section: A Operational Principlementioning

confidence: 98%

“…This work aims to investigate this proposition and provide evidence towards the cost efficiency of such hybrid, that is mixed FaaS and VMaaS, deployment scenarios for different kinds of load patterns. The realizability and efficacy of this position is already demonstrated by recent works like [8] and [9]. However, in contrast to these approaches, we focus on providing system owners with the means to decide whether and under which conditions such a deployment scenario is appropriate, rather than starting with the assumption that this is so.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cost efficiency under mixed serverless and serverful deployments

Reuter

Back

Andrikopoulos

2020

2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

“…As the related work has demonstrated, e.g. [8], [9], and we discuss further in the following section, this is relatively easily achievable by means of existing tooling.…”

Section: A Operational Principlementioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cost efficiency under mixed serverless and serverful deployments

Reuter

Back

Andrikopoulos

2020

2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)

View full text Add to dashboard Cite

show abstract

“…Much research has been done for designing and improving FaaS (Functions as a Service) platforms, in terms of performance [97]- [101], and security [102]- [104]. However, these works focus on the server side, instead of the client side.…”

Section: Re L a T E D Wo R Kmentioning

confidence: 99%

Are Machine Learning Cloud APIs Used Correctly?

Wan

Liu

Hoffmann

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

University o f Chicago {cw an, s h ic h e n g 2 0 0 0 , h a n k h o ffm a n n , m m a ire , s h a n lu } @ u c h ic a g o .e d u Abstract-Machine learning (ML) cloudAPIs enable developers to easily incorporate learning solutions into software systems. Unfortunately, ML APIs are challenging to use correctly and efficiently, given their unique semantics, data requirements, and accuracy-performance tradeoffs. Much prior work has studied how to develop ML APIs or ML cloud services, but not how open-source applications are using ML APIs. In this paper, we manually studied 360 representative open-source applications that use Google or AWS cloud-based m L APIs, and found 70% of these applications contain API misuses in their latest versions that degrade functional, performance, or economical quality of the software. We have generalized 8 anti-patterns based on our manual study and developed automated checkers that identify hundreds of more applications that contain ML API misuses. I. In t r o d u c t io n A. MotivationMachine learning (ML) provides efficient solutions for a number of problems that were difficult to solve with traditional computing techniques; e.g., object detection and language translation. ML cloud APIs allow programmers to incorporate these learning solutions into software systems without designing and training the learning model themselves [1], and hence put these powerful techniques into the hands of non-experts. Indeed, there are more than 35,000 open-source projects on GitHub that use Google or Amazon ML Cloud APIs to solve a wide variety of problems, among which more than 14,000 were created within the last 1 2 months.While these APIs make it easy for non-experts to incorporate learning into software systems, there are still a number of challenges that must be addressed to ensure that the resulting applications are both correct and efficient. While certain challenges come with the use of any third-party API, this paper focuses on unique challenges for ML APIs that arise due to the nature of learning itself. Complicated data requirements. Machine learning tech-niques are used to process digitalized real-world visual, audio and text content. Although such content can be generated by a huge variety of devices and encoding software, the suitable input content and format (encoding, resolution, size, etc.) for ML APIs are rather limited and often uniquely defined by the DNN-training process. For example, cameras can produce images in many formats, but the image sets on which ML models are trained have a relatively small variety [2 ]- [8 ]. Thus, it is up to the API user to select the input or convert the input into what the API can accept and effectively process.

show abstract

“…This function represents data processing pipelines that leverage massive parallelism by partitioning large data sets into multiple smaller chunks and invoking one function per chunk [26,42]. Classify(image): We run an image classification workload given the current active research [31,37,40,52] and industry [4,21] [25]. The idea is to split a video file into multiple small chunks and process each chunk in parallel.…”

Section: Rest Api (Http Request)mentioning

confidence: 99%

Photons

Dukic

Bruno

Singla

et al. 2020

Proceedings of the 11th ACM Symposium on Cloud Computing

View full text Add to dashboard Cite

Serverless computing allows users to create short, stateless functions and invoke hundreds of them concurrently to tackle massively parallel workloads. We observe that even though most of the footprint of a serverless function is fixed across its invocations-language runtime, libraries, and other application state-today's serverless platforms do not exploit this redundancy. Such an inefficiency has cascading negative impacts: longer startup times, lower throughput, higher latency, and higher cost. To mitigate these problems, we have built Photons, a framework leveraging workload parallelism to co-locate multiple instances of the same function within the same runtime. Concurrent invocations can then share the runtime and application state transparently, without compromising execution safety. Photons reduce function's memory consumption by 25% to 98% per invocation, with no performance degradation compared to today's serverless platforms. We also show that our approach can reduce the overall memory utilization by 30%, and the total number of cold starts by 52%. CCS Concepts • Computer systems organization → Cloud computing; • Software and its engineering → Virtual machines.

show abstract

Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud

Cited by 55 publications

References 38 publications

Cost efficiency under mixed serverless and serverful deployments

Cost efficiency under mixed serverless and serverful deployments

Are Machine Learning Cloud APIs Used Correctly?

Photons

Contact Info

Product

Resources

About