Improving hash join performance through prefetching

Chen, S.; Ailamaki, Anastassia; Gibbons, Phillip B.; Mowry, Todd C.

doi:10.1109/icde.2004.1319989

Cited by 89 publications

(97 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This join method improves cache locality by continuously partitioning into ever smaller chunks that ultimately t into the cache. Ailamaki et al [8] improved cache locality during the probing phase of the hash join using software controlled prefetching.…”

Section: Related Workmentioning

confidence: 99%

Massively Parallel NUMA-Aware Hash Joins

Lang

Leis

Albutiu

et al. 2015

In Memory Data Management and Analysis

View full text Add to dashboard Cite

Abstract. Driven by the two main hardware trends increasing main memory and massively parallel multi-core processing in the past few years, there has been much research eort in parallelizing well-known join algorithms. However, the non-uniform memory access (NUMA) of these architectures to main memory has only gained limited attention in the design of these algorithms. We study recent proposals of main memory hash join implementations and identify their major performance problems on NUMA architectures. We then develop a NUMA-aware hash join for massively parallel environments, and show how the specic implementation details aect the performance on a NUMA system. Our experimental evaluation shows that a carefully engineered hash join implementation outperforms previous high performance hash joins by a factor of more than two, resulting in an unprecedented throughput of 3/4 billion join argument tuples per second.

show abstract

Section: Related Workmentioning

confidence: 99%

Massively Parallel NUMA-Aware Hash Joins

Lang

Leis

Albutiu

et al. 2015

In Memory Data Management and Analysis

View full text Add to dashboard Cite

show abstract

“…In the literature, we find prefetching techniques that implement restricted forms of instruction stream interleaving. Chen et al [6] proposed to exploit instruction stream parallelism across subsequent tuples in hash joins by manually applying well-known loop transformations that a general-purpose compiler cannot consider due to lack of dependency information. They proposed group prefetching (GP) and software-pipelined prefetching (SPP), two techniques that transform a fixed chain of N memory accesses inside a loop into sequences of N+1 computation stages separated by prefetches.…”

Section: Interleaved Executionmentioning

confidence: 99%

“…Prior works propose two forms of such instruction stream interleaving (ISI): static, like group prefetching (GP) [6] and software pipelined prefetching (SPP) [6], and dynamic, like the state-of-the-art asynchronous memory access chaining (AMAC) [15]. Static interleaving has negligible overhead for instruction streams with identical control flow, whereas dynamic interleaving efficiently supports a wider range of use cases, allowing instruction streams to diverge, e.g., with early returns.…”

Section: Introductionmentioning

confidence: 99%

Interleaving with coroutines

et al. 2017

View full text Add to dashboard Cite

Index join performance is determined by the efficiency of the lookup operation on the involved index. Although database indexes are highly optimized to leverage processor caches, main memory accesses inevitably increase lookup runtime when the index outsizes the last-level cache; hence, index join performance drops. Still, robust index join performance becomes possible with instruction stream interleaving: given a group of lookups, we can hide cache misses in one lookup with instructions from other lookups by switching among their respective instruction streams upon a cache miss.In this paper, we propose interleaving with coroutines for any type of index join. We showcase our proposal on SAP HANA by implementing binary search and CSB + -tree traversal for an instance of index join related to dictionary compression. Coroutine implementations not only perform similarly to prior interleaving techniques, but also resemble the original code closely, while supporting both interleaved and non-interleaved execution. Thus, we claim that coroutines make interleaving practical for use in real DBMS codebases.

show abstract

“…There is rich recent literature on hardware-conscious joins [18,9,11,8,4,6,5] . [4] proposes a partitioned join that minimizes random inter-socket reads, and [15] improves upon that with a NUMA-aware data shuffling stage.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, inspired by the trend of cheaper and larger main memories, there has been a surge of advances on in-memory joins, e.g., [18,9,11,8,4,6,5] . An "in-memory join" typically means one in which the input tables, plus any intermediate data structures, completely fit in memory.…”

Section: Introductionmentioning

confidence: 99%

Memory-efficient hash joins

et al. 2014

View full text Add to dashboard Cite

We present new hash tables for joins, and a hash join based on them, that consumes far less memory and is usually faster than recently published in-memory joins. Our hash join is not restricted to outer tables that fit wholly in memory. Key to this hash join is a new concise hash table (CHT), a linear probing hash table that has 100% fill factor, and uses a sparse bitmap with embedded population counts to almost entirely avoid collisions. This bitmap also serves as a Bloom filter for use in multi-table joins.We study the random access characteristics of hash joins, and renew the case for non-partitioned hash joins. We introduce a variant of partitioned joins in which only the build is partitioned, but the probe is not, as this is more efficient for large outer tables than traditional partitioned joins. This also avoids partitioning costs during the probe, while at the same time allowing parallel build without latching overheads. Additionally, we present a variant of CHT, called a concise array table (CAT), that can be used when the key domain is moderately dense. CAT is collision-free and avoids storing join keys in the hash table.We perform a detailed comparison of CHT and CAT against leading in-memory hash joins. Our experiments show that we can reduce the memory usage by one to three orders of magnitude, while also being competitive in performance.

show abstract

Improving hash join performance through prefetching

Abstract: Abstract

Cited by 89 publications

References 19 publications

Massively Parallel NUMA-Aware Hash Joins

Massively Parallel NUMA-Aware Hash Joins

Interleaving with coroutines

Memory-efficient hash joins

Contact Info

Product

Resources

About