Morsel-driven parallelism

Abstract

With modern computer architecture evolving, two problems conspire against the state-of-the-art approaches in parallel query execution: (i) to take advantage of many-cores, all query work must be distributed evenly among (soon) hundreds of threads in order to achieve good speedup, yet (ii) dividing the work evenly is difficult even with accurate data statistics due to the complexity of modern out-of-order cores. As a result, the existing approaches for "plandriven" parallelism run into load balancing and context-switching bottlenecks, and therefore no longer scale. A third problem faced by many-core architectures is the decentralization of memory controllers, which leads to Non-Uniform Memory Access (NUMA).In response, we present the "morsel-driven" query execution framework, where scheduling becomes a fine-grained run-time task that is NUMA-aware. Morsel-driven query processing takes small fragments of input data ("morsels") and schedules these to worker threads that run entire operator pipelines until the next pipeline breaker. The degree of parallelism is not baked into the plan but can elastically change during query execution, so the dispatcher can react to execution speed of different morsels but also adjust resources dynamically in response to newly arriving queries in the workload. Further, the dispatcher is aware of data locality of the NUMA-local morsels and operator state, such that the great majority of executions takes place on NUMA-local memory. Our evaluation on the TPC-H and SSB benchmarks shows extremely high absolute performance and an average speedup of over 30 with 32 cores.

WindFlow: High-Speed Continuous Stream Processing With Parallel Building Blocks

“…Although the use of a system-level programming language like C++ brings a certain performance improvement per se, this comparison is useful to assess the potential of WINDFLOW with respect to existing (and in some cases) widely utilized tools. However, to make the performance evaluation stronger, we also compare with STREAMBOX [10], a C++-based SPS that uses a different streaming model (morsel-driven parallelism [9]).…”

“…In this final part, we compare with the C++-based SPS STREAMBOX [10]. STREAMBOX uses the morsel-driven parallelism model [9], where inputs are buffered in batches of records (called bundles), and dynamically scheduled for processing on a pool of threads. In STREAMBOX, the scheduling activity of bundles is performed in a centralized fashion, by leveraging lock-based primitives.…”

“…An alternative trend for designing new SPSs for scale-up servers is to change the runtime system design to enhance the throughput by fitting at best the memory bandwidth of the machine. Some new prototypes implement this approach by adopting the discretized streaming model (inspired by the morsel-driven parallelism in [9]), where inputs are buffered in batches that are then scheduled to a pool of available threads. Threads execute entire pipelines of operators as a tight loop on the batch elements, until a pipeline breaker (e.g., a keyby distribution) is reached.…”

Make the most out of your SIMD investments: counter control flow divergence in compiled query pipelines

“…It remains an open question how the optimal threshold can be predicted at query compilation time, as it depends on hardware, refilling costs, the costs incurred by underutilized lanes, and the actual input data. A possible approach to address this issue is to adaptively adjust the threshold parameter at runtime (per batch or per morsel [14]). Nevertheless, divergence handling cannot fully be disabled once the pipeline has been compiled.…”

Accelerating In-Memory Database Selections Using Latency Masking Hardware Threads

“…Multithreaded architectures seamlessly exploit thread-level parallelism and improve the performance of any of the aforementioned algorithms by partitioning the input over multiple threads [51]. Most modern processors support simultaneous multithreading (SMT) that allows multiple instructions from different threads to execute in the same cycle.…”