On Scalability of Distributed Machine Learning with Big Data on Apache Spark

Hai, Ameen Abdel; Forouraghi, Babak

doi:10.1007/978-3-319-94301-5_16

Cited by 1 publication

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data in the persistent storage layer are secured and protected through security services offered by the respective cloud vendors [8,21]. Researchers [22] pointed out that Apache Spark is the best-in-class in-memory data distributed framework. Databricks has optimized a version of Apache Spark that helps it process large datasets.…”

Section: Cloud-agnostic ML Platformmentioning

confidence: 99%

Low-Code Machine Learning Platforms: A Fastlane to Digitalization

Raghavendran,

Elragal

2023

Informatics

View full text Add to dashboard Cite

In the context of developing machine learning models, until and unless we have the required data engineering and machine learning development competencies as well as the time to train and test different machine learning models and tune their hyperparameters, it is worth trying out the automatic machine learning features provided by several cloud-based and cloud-agnostic platforms. This paper explores the possibility of generating automatic machine learning models with low-code experience. We developed criteria to compare different machine learning platforms for generating automatic machine learning models and presenting their results. Thereafter, lessons learned by developing automatic machine learning models from a sample dataset across four different machine learning platforms were elucidated. We also interviewed machine learning experts to conceptualize their domain-specific problems that automatic machine learning platforms can address. Results showed that automatic machine learning platforms can provide a fast track for organizations seeking the digitalization of their businesses. Automatic machine learning platforms help produce results, especially for time-constrained projects where resources are lacking. The contribution of this paper is in the form of a lab experiment in which we demonstrate how low-code platforms can provide a viable option to many business cases and, henceforth, provide a lane that is faster than the usual hiring and training of already scarce data scientists and to analytics projects that suffer from overruns.

show abstract

Section: Cloud-agnostic ML Platformmentioning

confidence: 99%