Resource Elasticity for Large-Scale Machine Learning

Huang, Botong; Böehm, Matthias; Tian, Yuanyuan; Reinwald, Berthold; Tatikonda, Shirish; Reiss, Frederick

doi:10.1145/2723372.2749432

Cited by 48 publications

(24 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to loop fusion, the dependencies are implicitly given by the data flow graph and operation semantics [9]. SystemML uses rewrites to identify special operator patterns, and replaces them with hand-coded local or distributed fused operators [7,13,37]. Other systems like Cumulon [36] and MatFast [92] use more generic masked and folded binary operators to exploit sparsity across matrix multiplications and element-wise operations.…”

Section: Related Workmentioning

confidence: 99%

On optimizing operator fusion plans for large-scale machine learning in systemML

et al. 2018

Self Cite

View full text Add to dashboard Cite

Many large-scale machine learning (ML) systems allow specifying custom ML algorithms by means of linear algebra programs, and then automatically generate efficient execution plans. In this context, optimization opportunities for fused operators-in terms of fused chains of basic operators-are ubiquitous. These opportunities include (1) fewer materialized intermediates, (2) fewer scans of input data, and (3) the exploitation of sparsity across chains of operators. Automatic operator fusion eliminates the need for hand-written fused operators and significantly improves performance for complex or previously unseen chains of operations. However, existing fusion heuristics struggle to find good fusion plans for complex DAGs or hybrid plans of local and distributed operations. In this paper, we introduce an optimization framework for systematically reason about fusion plans that considers materialization points in DAGs, sparsity exploitation, different fusion template types, as well as local and distributed operations. In detail, we contribute algorithms for (1) candidate exploration of valid fusion plans, (2) cost-based candidate selection, and (3) code generation of local and distributed operations over dense, sparse, and compressed data. Our experiments in SystemML show end-toend performance improvements with optimized fusion plans of up to 21x compared to hand-written fused operators, with negligible optimization and code generation overhead.

show abstract

Section: Related Workmentioning

confidence: 99%

On optimizing operator fusion plans for large-scale machine learning in systemML

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…This engine is responsible to configure execution settings, such as memory allocation and number of map and reduce tasks, by answering questions on real and hypothetical input parameters using a random search algorithm. What-if analysis is also employed by [37] for optimally configuring memory configurations. The distinctive feature of this proposal is that it is dynamic in the sense that it can take decisions at runtime leading to task migrations.…”

Section: Execution Engine Configurationmentioning

confidence: 99%

The many faces of data-centric workflow optimization: a survey

Gounaris

Simitsis

2018

Int J Data Sci Anal

View full text Add to dashboard Cite

Workflow technology is rapidly evolving and, rather than being limited to modeling the control flow in business processes, is becoming a key mechanism to perform advanced data management, such as big data analytics. This survey focuses on data-centric workflows (or workflows for data analytics or data flows), where a key aspect is data passing through and getting manipulated by a sequence of steps. The large volume and variety of data, the complexity of operations performed, and the long time such workflows take to compute give rise to the need for optimization. In general, data-centric workflow optimization is a technology in evolution. This survey focuses on techniques applicable to workflows comprising arbitrary types of data manipulation steps and semantic inter-dependencies between such steps. Further, it serves a twofold purpose. Firstly, to present the main dimensions of the relevant optimization problems and the types of optimizations that occur before flow execution. Secondly, to provide a concise overview of the existing approaches with a view to highlighting key observations and areas deserving more attention from the community.

show abstract

“…[28] developed an approach to find a near-optimal memory configuration to finish building a machine learning model as soon as possible. The approach offers no guarantee on model building time.…”

Section: Related Workmentioning

confidence: 99%

Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

Luo

2017

SIGKDD Explor. Newsl.

View full text Add to dashboard Cite

For user-friendliness, many software systems offer progress indicators for long-duration tasks. A typical progress indicator continuously estimates the remaining task execution time as well as the portion of the task that has been finished. Building a machine learning model often takes a long time, but no existing machine learning software supplies a non-trivial progress indicator. Similarly, running a data mining algorithm often takes a long time, but no existing data mining software provides a nontrivial progress indicator. In this article, we consider the problem of offering progress indicators for machine learning model building and data mining algorithm execution. We discuss the goals and challenges intrinsic to this problem. Then we describe an initial framework for implementing such progress indicators and two advanced, potential uses of them, with the goal of inspiring future research on this topic.

show abstract

Resource Elasticity for Large-Scale Machine Learning

Cited by 48 publications

References 39 publications

On optimizing operator fusion plans for large-scale machine learning in systemML

On optimizing operator fusion plans for large-scale machine learning in systemML

The many faces of data-centric workflow optimization: a survey

Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

Contact Info

Product

Resources

About