Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools

Ruf, Philipp; Madan, Manav; Reich, Christoph; Abdeslam, Djaffar Ould

doi:10.3390/app11198861

Cited by 74 publications

(25 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, not having in-built notebooks and not maintaining notebook versioning to be used as IDE for the development are limitations in this tool. In addition, MLFlow does not maintain user management and does not offer full customizability like grouping experiments [20].…”

Section: Mlflowmentioning

confidence: 99%

Machine Learning Operations: A Survey on MLOps Tool Support

Hewage¹,

Meedeniya²

2022

Preprint

View full text Add to dashboard Cite

Machine Learning (ML) has become a fast-growing, trending approach in solution development in practice. Deep Learning (DL) which is a subset of ML, learns using deep neural networks to simulate the human brain. It trains machines to learn techniques and processes individually using computer algorithms, which is also considered to be a role of Artificial Intelligence (AI). In this paper, we study current technical issues related to software development and delivery in organizations that work on ML projects. Therefore, the importance of the Machine Learning Operations (MLOps) concept, which can deliver appropriate solutions for such concerns, is discussed. We investigate commercially available MLOps tool support in software development. The comparison between MLOps tools analyzes the performance of each system and its use cases. Moreover, we examine the features and usability of MLOps tools to identify the most appropriate tool support for given scenarios. Finally, we recognize that there is a shortage in the availability of a fully functional MLOps platform on which processes can be automated by reducing human intervention.

show abstract

Section: Mlflowmentioning

confidence: 99%

Machine Learning Operations: A Survey on MLOps Tool Support

Hewage¹,

Meedeniya²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Replicable results are also important when considering the development of ML models. The emerging field of Machine Learning Ops (MLOps) tackles the automation, provenance, performance, and other aspects of ML in a workflow-based form [92]. The ZenML Python library 16 provides a high-level API to machine learning tasks and tools, while offering workflow management features such as versioning, scheduling, and visualisation.…”

Section: Text-basedmentioning

confidence: 99%

“…Recently there have been articles in multiple domains walking a domain expert through the best tools and techniques available to employ ML [52,75,111,66,92]. For example, Nakhle and Harfouche provide four detailed Jupyter notebooks 52 walking domain experts in phenomics (plant sciences) through four steps of a ML task [82].…”

Section: Guiding the Domain Expertmentioning

confidence: 99%

Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State-of-the-Practice

Oakes¹,

Famelis²,

Sahraoui³

2022

Preprint

View full text Add to dashboard Cite

Domain experts are increasingly employing machine learning to solve their domain-specific problems. This article presents six key challenges that a domain expert faces in transforming their problem into a computational workflow, and then into an executable implementation. These challenges arise out of our conceptual framework which presents the "route" of options that a domain expert may choose to take while developing their solution.To ground our conceptual framework in the stateof-the-practice, this article discusses a selection of available textual and graphical workflow systems and their support for these six challenges. Case studies from the literature in various domains are also examined to highlight the tools used by the domain experts as well as a classification of the domain-specificity and machine learning usage of their problem, workflow, and implementation.The state-of-the-practice informs our discussion of the six key challenges, where we identify which challenges are not sufficiently addressed by available tools. We also suggest possible research directions for software engineering researchers to increase the automation of these tools and disseminate best-practice techniques between software engineering and various scientific domains.

show abstract

“…MLOps, an ML-oriented version of DevOps, is concerned with supporting an entire data science life cycle, from data acquisition to deployment of a production model. Many of the same challenges are present, reproducibility and provenance are crucial in both production and research workflows [RMRO21]. Infrastructure, tools, and practices developed for MLOps may also hold value in the scientific community.…”

Section: Related Workmentioning

confidence: 99%

Design of a Scientific Data Analysis Support Platform

Martindale¹,

Hite²,

Stewart³

et al. 2022

Proceedings of the Python in Science Conference

View full text Add to dashboard Cite

Software data analytic workflows are a critical aspect of modern scientific research and play a crucial role in testing scientific hypotheses. A typical scientific data analysis life cycle in a research project must include several steps that may not be fundamental to testing the hypothesis, but are essential for reproducibility. This includes tasks that have analogs to software engineering practices such as versioning code, sharing code among research team members, maintaining a structured codebase, and tracking associated resources such as software environments. Tasks unique to scientific research include designing, implementing, and modifying code that tests a hypothesis. This work refers to this code as an experiment, which is defined as a software analog to physical experiments.A software experiment manager should support tracking and reproducing individual experiment runs, organizing and presenting results, and storing and reloading intermediate data on long-running computations. A software experiment manager with these features would reduce the time a researcher spends on tedious busywork and would enable more effective collaboration. This work discusses the necessary design features in more depth, some of the existing software packages that support this workflow, and a custom developed opensource solution to address these needs.

show abstract

Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools

Cited by 74 publications

References 43 publications

Machine Learning Operations: A Survey on MLOps Tool Support

Machine Learning Operations: A Survey on MLOps Tool Support

Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State-of-the-Practice

Design of a Scientific Data Analysis Support Platform

Contact Info

Product

Resources

About