On Continuous Integration / Continuous Delivery for Automated Deployment of Machine Learning Models using MLOps

Abstract

Model deployment in machine learning has emerged as an intriguing field of research in recent years. It is comparable to the procedure defined for conventional software development. Continuous Integration and Continuous Delivery (CI/CD) have been shown to smooth down software advancement and speed up businesses when used in conjunction with development and operations (DevOps). Using CI/CD pipelines in an application that includes Machine Learning Operations (MLOps) components, on the other hand, has difficult difficulties, and pioneers in the area solve them by using unique tools, which is typically provided by cloud providers. This research provides a more in-depth look at the machine learning lifecycle and the key distinctions between DevOps and MLOps. In the MLOps approach, we discuss tools and approaches for executing the CI/CD pipeline of machine learning frameworks. Following that, we take a deep look into push and pull-based deployments in Github Operations (GitOps). Open exploration issues are also identified and added, which may guide future study.

"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning

“…Similarly, the field of MLOps, or DevOps principles applied to machine learning, has emerged from the rise of machine learning (ML) application development in software organizations. MLOps is a nascent field, where most existing papers give definitions and overviews of MLOps, as well as its relation to ML, software engineering, DevOps, and data engineering [22,35,44,58,81,89,91,[98][99][100]. Some work in MLOps attempts to characterize a production ML lifecycle; however, there is little consensus.…”

“…Some work in MLOps attempts to characterize a production ML lifecycle; however, there is little consensus. Symeonidis et al [98] discuss a lifecycle of data preparation, model selection, and model productionization, but other literature reviews [22,53] and guides on best practices drawing from authors' experiences [59] conclude that, compared to software engineering, there is not yet a standard ML lifecycle, with consensus from researchers and industry professionals. While standardizing an ML lifecycle across different roles (e.g., scientists, researchers, business leaders, engineers) might be challenging, characterizing a workflow specific to a certain role could be more tractable.…”

“…For example, adhering to data governance standards and regulation is difficult, as model training is data-hungry by nature [5,25]. Garg et al [22] discuss issues in continuous end-to-end testing (i.e., continuous integration) because ML development involves changes to datasets and model parameters in addition to code. To address such challenges, other MLOps papers have surveyed the proliferating number of industry-originated tools in MLOps [53,80,83,98].…”

Angler: Helping Machine Translation Practitioners Prioritize Model Improvements

“…There are many different ways to apply unit tests to an ML system. For example, practitioners can write unit tests to validate the data quality [88,110], verify a model's behavior [68,104], and maintain ML operations (MLOps) [36,47]. In Angler, we design simple rule-based unit tests, such as if the source does not contain offensive words, then the translation should not either.…”

Structure Learning and Hyperparameter Optimization Using an Automated Machine Learning (AutoML) Pipeline

“…CI/CD mechanisms are very essential in MLOps' development because they establish real-time model testing and deployment [34]. These mechanisms involve the design of different versions of software frameworks to handle code changes and to create automated testing tools for quality assessment and deploy software automation tools to manage the rollout of the above code changes [35,36], with the ultimate purpose being the automatic production of software applications in a continuous and iterative manner, utilizing less time and effort [34][35][36].…”