Reactive and proactive sharing across concurrent analytical queries

Abstract

Today an ever increasing amount of data is collected and analyzed by researchers, businesses, and scientists in data warehouses (DW). In addition to the data size, the number of users and applications querying data grows exponentially. The increasing concurrency is itself a challenge in query execution, but also introduces an opportunity favoring synergy between concurrent queries. Traditional execution engines of DW follows a query-centric approach, where each query is optimized and executed independently. On the other hand, workloads with increased concurrency have several queries with common parts of data and work, creating the opportunity for sharing among concurrent queries. Sharing can be reactive to the inherently existing sharing opportunities, or proactive by redesigning query operators to maximize the sharing opportunities.This demonstration showcases the impact of proactive and reactive sharing by comparing and integrating representative stateof-the-art techniques: Simultaneous Pipelining (SP), for reactive sharing, which shares intermediate results of common sub-plans, and Global Query Plans (GQP) for proactive sharing, which build and evaluate a single query plan with shared operators. We visually demonstrate, in an interactive interface, the behavior of both sharing approaches on top of a state-of-the-art storage engine using the original prototypes. We show that pull-based sharing for SP eliminates the serialization point imposed by the original push-based approach. Then, we compare, through a sensitivity analysis, the performance of SP and GQP. Finally, we show that SP can improve the performance of GQP for a query mix with common sub-plans.

How to stop under-utilization and love multicores

“…[SIGMOD14b] i  multicores [VLDB07a,PVLDB13b] reactive sharing: how to react? reactive sharing can improve proactive sharing [PVLDB13b] i  multicores sharing in practice 116 QPipe [SIGMOD05] CJOIN [VLDB09a] DataPath [SIGMOD10a] SharedDB [PVLDB12b,PVLDB14b] sharing type reactive proactive (global query plan) balances memory and interconnect traffic [CIDR13b] radix hash join 123 [VLDB09c] cache-efficient but not NUMA-aware partitions (by key) are small enough to fit into cache figure courtesy of Kim et al…”