MLI: An API for Distributed Machine Learning

Sparks, Evan; Talwalkar, Ameet; Smith, Virginia; Kottalam, Jey; Pan, Xinghao; Gonzalez, Joseph E.; Franklin, Michael J.; Jordan, Michael I.; Kraska, Tim

doi:10.1109/icdm.2013.158

Cited by 133 publications

(90 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They scale well to tens of nodes, but at large scale, this synchrony creates challenges as the chance of a node operating slowly increases. Mahout [4], based on Hadoop [18] and MLI [44], based on Spark [50], both adopt the iterative MapReduce [14] framework. A key insight of Spark and MLI is preserving state between iterations, which is a core goal of the parameter server.…”

Section: Related Workmentioning

confidence: 99%

Scaling Distributed Machine Learning with the Parameter Server

2014

Proceedings of the 2014 International Conference on Big Data Science and Computing

1,176

1,146

View full text Add to dashboard Cite

We propose a parameter server framework for distributed machine learning problems. Both data and workloads are distributed over worker nodes, while the server nodes maintain globally shared parameters, represented as dense or sparse vectors and matrices. The framework manages asynchronous data communication between nodes, and supports flexible consistency models, elastic scalability, and continuous fault tolerance.To demonstrate the scalability of the proposed framework, we show experimental results on petabytes of real data with billions of examples and parameters on problems ranging from Sparse Logistic Regression to Latent Dirichlet Allocation and Distributed Sketching.

show abstract

Section: Related Workmentioning

confidence: 99%

Scaling Distributed Machine Learning with the Parameter Server

2014

Proceedings of the 2014 International Conference on Big Data Science and Computing

1,176

1,146

View full text Add to dashboard Cite

show abstract

“…Similar techniques, including batching data within Spark records, indexing it, and optimizing partitioning, have been used in GraphX [112], MLlib [96], MLI [98] and other projects. Together, these techniques have allowed RDD-based systems to achieve similar performance to specialized systems in each domain, while providing much higher performance in applications that combine processing types, and fault tolerance across these types of computations.…”

Section: Discussionmentioning

confidence: 99%

An Architecture for Fast and General Data Processing on Large Clusters

Zaharia

2016

View full text Add to dashboard Cite

The past few years have seen a major change in computing systems, as growing data volumes and stalling processor speeds require more and more applications to scale out to distributed systems. Today, a myriad data sources, from the Internet to business operations to scientific instruments, produce large and valuable data streams. However, the processing capabilities of single machines have not kept up with the size of data, making it harder and harder to put to use. As a result, a growing number of organizations-not just web companies, but traditional enterprises and research labs-need to scale out their most important computations to clusters of hundreds of machines.At the same time, the speed and sophistication required of data processing have grown. In addition to simple queries, complex algorithms like machine learning and graph analysis are becoming common in many domains. And in addition to batch processing, streaming analysis of new real-time data sources is required to let organizations take timely action. Future computing platforms will need to not only scale out traditional workloads, but support these new applications as well.This dissertation proposes an architecture for cluster computing systems that can tackle emerging data processing workloads while coping with larger and larger scales. Whereas early cluster computing systems, like MapReduce, handled batch processing, our architecture also enables streaming and interactive queries, while keeping the scalability and fault tolerance of previous systems. And whereas most deployed systems only support simple one-pass computations (e.g., aggregation or SQL queries), ours also extends to the multi-pass algorithms required for more complex analytics (e.g., iterative algorithms for machine learning). Finally, unlike the specialized systems proposed for some of these workloads, our architecture allows these computations to be combined, enabling rich new applications that intermix, for example, streaming and batch processing, or SQL and complex analytics. We achieve these results through a simple extension to MapReduce that adds primitives for data sharing, called Resilient Distributed Datasets (RDDs). We show that this is enough to efficiently capture a wide range of workloads. We implement RDDs in the open source Spark system, which we evaluate using both synthetic 1 benchmarks and real user applications. Spark matches or exceeds the performance of specialized systems in many application domains, while offering stronger fault tolerance guarantees and allowing these workloads to be combined. We explore the generality of RDDs from both a theoretical modeling perspective and a practical perspective to see why this extension can capture a wide range of previously disparate workloads. 2To my family i

show abstract

“…Some of these works include performing analytics over Twitter [12], computing k-means clustering over big data in the cloud [13], providing recommendations [14] [15], studying the behavior of tourists [16], performing sentiment analysis [17], minimizing product escapes in aerospace test environments [18], improving a predictive model in a healthcare domain [19], detecting astrophysical objects [20], discovering communities in social networks [21] and many more. Also, recent works have provided detailed studies of technologies for batch processing techniques over big data [22], as well as current applications and systems for this purpose [23] and proposed APIs for distributed machine learning [24].…”

Section: State Of the Artmentioning

confidence: 99%

A scalable machine learning online service for big data real-time analysis

Baldominos

Albacete

Sáez

et al. 2014

2014 IEEE Symposium on Computational Intelligence in Big Data (CIBD)

View full text Add to dashboard Cite

Abstract-This work describes a proposal for developing and testing a scalable machine learning architecture able to provide real-time predictions or analytics as a service over domain-independent big data, working on top of the Hadoop ecosystem and providing real-time analytics as a service through a RESTful API. Systems implementing this architecture could provide companies with on-demand tools facilitating the tasks of storing, analyzing, understanding and reacting to their data, either in batch or stream fashion; and could turn into a valuable asset for improving the business performance and be a key market differentiator in this fast pace environment. In order to validate the proposed architecture, two systems are developed, each one providing classical machine-learning services in different domains: the first one involves a recommender system for web advertising, while the second consists in a prediction system which learns from gamers' behavior and tries to predict future events such as purchases or churning. An evaluation is carried out on these systems, and results show how both services are able to provide fast responses even when a number of concurrent requests are made, and in the particular case of the second system, results clearly prove that computed predictions significantly outperform those obtained if random guess was used.

show abstract

MLI: An API for Distributed Machine Learning

Cited by 133 publications

References 7 publications

Scaling Distributed Machine Learning with the Parameter Server

Scaling Distributed Machine Learning with the Parameter Server

An Architecture for Fast and General Data Processing on Large Clusters

A scalable machine learning online service for big data real-time analysis

Contact Info

Product

Resources

About