Implicit Parallelism through Deep Language Embedding

Alexandrov, A.; Kunft, Andreas; Katsifodimos, Asterios; Schüler, Felix; Thamsen, Lauritz; Kao, Odej; Herb, Tobias; Markl, Volker

doi:10.1145/2723372.2750543

Cited by 42 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Fuse-no-redundancy takes another extreme of fusion without redundant compute, which leads to materializing all intermediates with multiple consumers. This heuristic is similar to caching policies in Emma [2].…”

Section: Problem Formulation and Heuristicsmentioning

confidence: 99%

On optimizing operator fusion plans for large-scale machine learning in systemML

et al. 2018

View full text Add to dashboard Cite

Many large-scale machine learning (ML) systems allow specifying custom ML algorithms by means of linear algebra programs, and then automatically generate efficient execution plans. In this context, optimization opportunities for fused operators-in terms of fused chains of basic operators-are ubiquitous. These opportunities include (1) fewer materialized intermediates, (2) fewer scans of input data, and (3) the exploitation of sparsity across chains of operators. Automatic operator fusion eliminates the need for hand-written fused operators and significantly improves performance for complex or previously unseen chains of operations. However, existing fusion heuristics struggle to find good fusion plans for complex DAGs or hybrid plans of local and distributed operations. In this paper, we introduce an optimization framework for systematically reason about fusion plans that considers materialization points in DAGs, sparsity exploitation, different fusion template types, as well as local and distributed operations. In detail, we contribute algorithms for (1) candidate exploration of valid fusion plans, (2) cost-based candidate selection, and (3) code generation of local and distributed operations over dense, sparse, and compressed data. Our experiments in SystemML show end-toend performance improvements with optimized fusion plans of up to 21x compared to hand-written fused operators, with negligible optimization and code generation overhead.

show abstract

Section: Problem Formulation and Heuristicsmentioning

confidence: 99%

On optimizing operator fusion plans for large-scale machine learning in systemML

et al. 2018

View full text Add to dashboard Cite

show abstract

“…(1) Variables in scope at the beginning of the input code (2) Variables that are modified within the input code (3) The operators and library methods used The code analyzer extracts these properties using standard program analyses. It computes (1) and (2) using live variable and dataflow analysis [1], and it computes (3) by scanning functions that are invoked in the input code.…”

Section: Defining the Search Spacementioning

confidence: 99%

“…Figure 6 shows sample grammars that are generated for the code shown in Figure 1(a). 2 The input code uses addition and division; hence, the grammar includes addition and division in its production rules for λ m and λ r . Furthermore, Casper also uses type information of variables to prune invalid production rules in the grammar.…”

Section: Defining the Search Spacementioning

confidence: 99%

“…Query Optimizers and IRs. Modern frameworks usually ship with sophisticated query optimizers [2,9,18,29,30] for generating efficient execution plans. However, these tools make users express their queries in the provided APIs.…”

Section: Related Workmentioning

confidence: 99%

“…Thus, if variable handles v1 and v2 are both inputs into the same code fragment, Casper wraps the translated code as: if (v1 != v2) { [Casper translated code] } else { [original code] }. Computing precise alias information requires more engineering[43] and does not impact our approach 2. Refer to Appendix D to see how a grammar can be encoded in our IR.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications

Ahmad

Cheung

2018

Proceedings of the 2018 International Conference on Management of Data

View full text Add to dashboard Cite

MapReduce is a popular programming paradigm for developing large-scale, data-intensive computation. Many frameworks that implement this paradigm have recently been developed. To leverage these frameworks, however, developers must become familiar with their APIs and rewrite existing code. We present Casper, a new tool that automatically translates sequential Java programs into the MapReduce paradigm. Casper identifies potential code fragments to rewrite and translates them in two steps: (1) Casper uses program synthesis to search for a program summary (i.e., a functional specification) of each code fragment. The summary is expressed using a high-level intermediate language resembling the MapReduce paradigm and verified to be semantically equivalent to the original using a theorem prover. (2) Casper generates executable code from the summary, using either the Hadoop, Spark, or Flink API. We evaluated Casper by automatically converting realworld, sequential Java benchmarks to MapReduce. The resulting benchmarks perform up to 48.2× faster compared to the original.We implemented Casper using the Polyglot framework [37] to parse Java code into an abstract syntax tree (AST). Casper traverses the program AST to identify candidate code fragments, performs program analysis, and generates target code. We now describe the Java features supported by our compiler front-end. We also discuss how Casper identifies code fragments for translation and generates executable code from the verified program summary.

show abstract