High Performance Serverless Architecture for Deep Learning Workflows

Chahal, Dheeraj; Ramesh, Manju; Ojha, Ravi; Singhal, Rekha

doi:10.1109/ccgrid51090.2021.00096

Cited by 8 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The machine learning inference services are commonly latency critical, and the auto-scaling ability of serverless computing could deal with bursty workloads well [8]. Yang et al [68] presented a solution called INFLess that reduces the allocation of resources for each serverless instance to reach optimal performance in inference services.…”

Section: ) Ensure Resource Scalability and Predictive Scalingmentioning

confidence: 99%

“…For smaller batch sizes, the processing time increases linearly, but for larger batch sizes, it increases exponentially in an on-premise environment [8]. Deese [22] used the batch mode (maximum size) read and write requests within AWS and found a significant speed increase from batch write operations, but a relatively small benefit from batch reads.…”

Section: ) Batchingmentioning

confidence: 99%

“…Thus, Serverless computing [4] is an interesting option regarding the resolution of small tasks, mainly when companies cannot estimate the traffic of their ML applications, scalability and cost accurately [5]. Furthermore, several studies are exploiting serverless computing to accomplish tasks of the ML pipeline, such as training [6], hyperparameter tuning [7], and model deployment [8].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Serverless on Machine Learning: A Systematic Mapping Study

2022

View full text Add to dashboard Cite

Machine Learning Operations (MLOps) is an approach to managing the entire lifecycle of a machine learning model. It has evolved over the last years and has started attracting many people in research and businesses in the industry. It supports the development of machine learning (ML) pipelines typical in the phases of data collection, data pre-processing, building datasets, model training, hyperparameters refinement, testing, and deployment to production. This complex pipeline workflow is a tedious process of iterative experimentation. Moreover, cloud computing services provide advanced features for managing ML stages and deploying them efficiently to production. Specifically, serverless computing has been applied in different stages of the machine learning pipeline. However, to the best of our knowledge, it is missing to know the serverless suitability and benefits it can provide to the ML pipeline. In this paper, we provide a systematic mapping study of machine learning systems applied on serverless architecture that include 53 relevant studies. During this study, we focused on (1) exploring the evolution trend and the main venues; (2) determining the researchers' focus and interest in using serverless on machine learning;(3) discussing solutions that serverless computing provides to machine learning. Our results show that serverless usage is growing, and several venues are interested in the topic. In addition, we found that the most widely used serverless provider is AWS Lambda, where the primary application was used in the deployment of the ML model. Additionally, several challenges were explored, such as reducing cost, resource scalability, and reducing latency. We moreover discuss the potential challenges of adopting ML on serverless, such as respecting service level agreement, the cold start problem, security, and privacy. Finally, our contribution provides foundations for future research and applications implying machine learning in serverless computing.

show abstract

Section: ) Ensure Resource Scalability and Predictive Scalingmentioning

confidence: 99%

Section: ) Batchingmentioning

confidence: 99%