Generating custom code for efficient query execution on heterogeneous processors

Breβ, Sebastian; Köcher, Bastian; Funke, Henning; Zeuch, Steffen; Rabl, Tilmann; Markl, Volker

doi:10.1007/s00778-018-0512-y

Cited by 40 publications

(25 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lara currently does not apply fusion of linear algebra operators and UDFs applications, as our current dense (BLAS) and sparse (Breeze) backends do not support fused operators. Future work could extend our optimizations on data layout access patterns to generate kernels for sparse linear algebra operations with UDF support and hardware-efficient code by integrating ideas from recent work [37,12,43,16]. Furthermore, one could extend the combinator view by integrating more data representations (e.g., block-wise or compressed [24]).…”

Section: Resultsmentioning

confidence: 99%

An intermediate representation for optimizing machine learning pipelines

et al. 2019

Self Cite

View full text Add to dashboard Cite

Machine learning (ML) pipelines for model training and validation typically include preprocessing, such as data cleaning and feature engineering, prior to training an ML model. Preprocessing combines relational algebra and user-defined functions (UDFs), while model training uses iterations and linear algebra. Current systems are tailored to either of the two. As a consequence, preprocessing and ML steps are optimized in isolation. To enable holistic optimization of ML training pipelines, we present Lara, a declarative domainspecific language for collections and matrices. Lara's intermediate representation (IR) reflects on the complete program, i.e., UDFs, control flow, and both data types. Two views on the IR enable diverse optimizations. Monads enable operator pushdown and fusion across type and loop boundaries. Combinators provide the semantics of domainspecific operators and optimize data access and cross-validation of ML algorithms. Our experiments on preprocessing pipelines and selected ML algorithms show the effects of our proposed optimizations on dense and sparse data, which achieve speedups of up to an order of magnitude.

show abstract

Section: Resultsmentioning

confidence: 99%

An intermediate representation for optimizing machine learning pipelines

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Very low-level LOLEPOPs would be similar to VOILA which requires state management/update. Needless to say,this also holds LOLEPOP-based representations such as Hawk [10]. Comprehensions describe enumerations as composition of scalar operations.…”

Section: Languages and Representationsmentioning

confidence: 99%

Charting the design space of query execution using VOILA

Gubner

Boncz

2021

Proc. VLDB Endow.

View full text Add to dashboard Cite

Database architecture, while having been studied for four decades now, has delivered only a few designs with well-understood properties. These few are followed by most actual systems. Acquiring more knowledge about the design space is a very time-consuming processes that requires manually crafting prototypes with a low chance of generating material insight. We propose a framework that aims to accelerate this exploration process significantly. Our framework enables synthesizing many different engines from a description in a carefully designed domain-specific language (VOILA). We explain basic concepts and formally define the semantics of VOILA. We demonstrate VOILA's flexibility by presenting translation back-ends that allow the synthesis of state-of-the-art paradigms (data-centric compilation, vectorized execution, AVX-512), mutations and mixes thereof. We show-case VOILA's flexibility by exploring the query engine design space in an automated fashion. We generated thousands of query engines and report our findings. Queries generated by VOILA achieve similar performance as state-of-the-art hand-optimized implementations and are up to 35.5X faster than well-known systems.

show abstract

“…In addition to that, it is not yet clear how to combine novel research suggestions in a unified system, and how such suggestions may affect or benefit from each other. In particular, our research community shows opportunities and challenges of modern hardware in database systems in isolation, among them the need for analysis of novel adaptive data layouts and data structures for operational and analytical systems [6,7,9,55], novel processing, storage and federation approaches on non-relational data models [11,18,51,52,62], benefits and drawbacks of porting to new compute platforms [12,16,33,63], opportunities and limitations of GPUs and other co-processors as building blocks for storage and querying purposes [8,14,33], novel proposals for main memory databases on modern hardware [3,15,21,53,56], adaptive optimization, and first attempts towards self-managing database systems [17,36,43,46].…”

Section: Introductionmentioning

confidence: 99%

GridTables: A One-Size-Fits-Most H2TAP Data Store

et al. 2020

View full text Add to dashboard Cite

Heterogeneous Hybrid Transactional Analytical Processing (H 2 TAP) database systems have been developed to match the requirements for low latency analysis of real-time operational data. Due to technical challenges, these systems are hard to architect, non-trivial to engineer, and complex to administrate. Current research has proposed excellent solutions to many of those challenges in isolation -a unified engine enabling to optimize performance by combining these solutions is still missing. In this concept paper, we suggest a highly flexible and adaptive data structure (called GRIDTABLE) to physically organize sparse but structured records in the context of H 2 TAP. For this, we focus on the design of an efficient highly-flexible storage layout that is built from scratch for mixed query workloads. The key challenges we address are:(1) partial storage in different memory locations, and (2) the ability to optimize for mixed OLTP-/OLAP access patterns. To guarantee safe and well-specified data definition or manipulation, as well as fast querying with no compromises on performance, we propose two dedicated access paths to the storage. In this paper, we explore the architecture and internals of GRIDTABLES showing design goals, concepts and trade-offs. We close this paper with open research questions and challenges that must be addressed in order to take advantage of the flexibility of our solution.

show abstract

Generating custom code for efficient query execution on heterogeneous processors

Cited by 40 publications

References 57 publications

An intermediate representation for optimizing machine learning pipelines

An intermediate representation for optimizing machine learning pipelines

Charting the design space of query execution using VOILA

GridTables: A One-Size-Fits-Most H2TAP Data Store

Contact Info

Product

Resources

About