Scaling up analytical queries with column-stores

Abstract

As data analytics is used by an increasing number of applications, data analytics engines are required to execute workloads with increased concurrency, i.e., an increasing number of clients submitting queries. Data management systems designed for data analytics -a market dominated by column-stores -however, were initially optimized for single query execution, minimizing its response time. Hence, they do not treat concurrency as a first class citizen.In this paper, we experiment with one open-source and two commercial column-stores using the TPC-H and SSB benchmarks in a setup with an increasing number of concurrent clients submitting queries, focusing on whether the tested systems can scale up in a single node instance. The tested systems for in-memory workloads scale up, to some degree; however, when the server is saturated they fail to fully exploit the available parallelism. Further, we highlight the unpredictable response times for high concurrency.

Multi-core column-store parallelization under concurrent workload

“…In [5] the authors compare the behavior of three columnar systems under concurrent TPC-H and SSB queries, while scaling up the concurrent clients. It shows the response time increases linearly as the concurrent clients increase.…”