Block oriented processing of relational database operations in modern computer architectures

Padmanabhan, Sriram; Malkemus, Timothy; Jhingran, Anant; Agarwal, R.

doi:10.1109/icde.2001.914871

Cited by 47 publications

(44 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As long as query processing was dominated by disk I/O the iterator model worked fine. However, as the CPU consumption became an issue, some systems tried to reduce the high calling costs of the iterator model by passing blocks of tuples between operators [11]. This greatly reduces the number of function invocations, but causes additional materialization costs.…”

Section: Related Workmentioning

confidence: 99%

“…These observations have led some modern systems to a departure from this pure iterator model, either internally (e.g., by internally decompressing a number of tuples at once and then only iterating over the decompressed data), or externally by producing more than one tuple during each next call [11] or even producing all tuples at once [1]. This blockoriented processing amortizes the costs of calling another operator over the large number of produced tuples, such that the invocation costs become negligible.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficiently compiling efficient query plans for modern hardware

2011

View full text Add to dashboard Cite

As main memory grows, query performance is more and more determined by the raw CPU costs of query processing itself. The classical iterator style query processing technique is very simple and flexible, but shows poor performance on modern CPUs due to lack of locality and frequent instruction mispredictions. Several techniques like batch oriented processing or vectorized tuple processing have been proposed in the past to improve this situation, but even these techniques are frequently out-performed by hand-written execution plans.In this work we present a novel compilation strategy that translates a query into compact and efficient machine code using the LLVM compiler framework. By aiming at good code and data locality and predictable branch layout the resulting code frequently rivals the performance of handwritten C++ code. We integrated these techniques into the HyPer main memory database system and show that this results in excellent query performance while requiring only modest compilation time.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Efficiently compiling efficient query plans for modern hardware

2011

View full text Add to dashboard Cite

show abstract

“…Most of these inefficiencies are inherent to LINQ's execution paradigm and are also found in other implementations (i.e., in Mono). Execution paradigm LINQ's execution paradigm resembles the tuple-at-a-time paradigm in database systems, and so shares some of its drawbacks, i.e., mainly a high per-element overhead [2,17]. The enumerable returned by iterator methods is auto-generated by the compiler.…”

Section: Inefficiencies In Linq-to-objectsmentioning

confidence: 99%

Code generation for efficient query processing in managed runtimes

2014

View full text Add to dashboard Cite

In this paper we examine opportunities arising from the convergence of two trends in data management: in-memory database systems (IMDBs), which have received renewed attention following the availability of affordable, very large main memory systems; and language-integrated query, which transparently integrates database queries with programming languages (thus addressing the famous 'impedance mismatch' problem). Language-integrated query not only gives application developers a more convenient way to query external data sources like IMDBs, but also to use the same querying language to query an application's in-memory collections. The latter offers further transparency to developers as the query language and all data is represented in the data model of the host programming language. However, compared to IMDBs, this additional freedom comes at a higher cost for query evaluation. Our vision is to improve in-memory query processing of application objects by introducing database technologies to managed runtimes.We focus on querying and we leverage query compilation to improve query processing on application objects. We explore different query compilation strategies and study how they improve the performance of query processing over application data. We take C as the host programming language as it supports languageintegrated query through the LINQ framework. Our techniques deliver significant performance improvements over the default LINQ implementation. Our work makes important first steps towards a future where data processing applications will commonly run on machines that can store their entire datasets in-memory, and will be written in a single programming language employing languageintegrated query and IMDB-inspired runtimes to provide transparent and highly efficient querying.

show abstract

“…Blink uses batched, multi-row execution at each stage of query processing as in [7,3], to exploit ILP and SIMD. Short-circuiting for non-qualifying tuples only occurs between stages.…”

Section: Policy For Invoking Rewritesmentioning

confidence: 99%

Row-wise parallel predicate evaluation

Johnson¹,

Raman

Sidle

et al. 2008

Proc. VLDB Endow.

View full text Add to dashboard Cite

Table scans have become more interesting recently due to greater use of ad-hoc queries and greater availability of multicore, vector-enabled hardware. Table scan performance is limited by value representation, table layout, and processing techniques. In this paper we propose a new layout and processing technique for efficient one-pass predicate evaluation. Starting with a set of rows with a fixed number of bits per column, we append columns to form a set of banks and then pad each bank to a supported machine word length, typically 16, 32, or 64 bits. We then evaluate partial predicates on the columns of each bank, using a novel evaluation strategy that evaluates column level equality, range tests, IN-list predicates, and conjuncts of these predicates, simultaneously on multiple columns within a bank, and on multiple rows within a machine register. This approach outperforms pure column stores, which must evaluate the partial predicates one column at a time. We evaluate and compare the performance and representation overhead of this new approach and several proposed alternatives.

show abstract

Block oriented processing of relational database operations in modern computer architectures

Cited by 47 publications

References 10 publications

Efficiently compiling efficient query plans for modern hardware

Efficiently compiling efficient query plans for modern hardware

Code generation for efficient query processing in managed runtimes

Row-wise parallel predicate evaluation

Contact Info

Product

Resources

About