Design and Evaluation of a Configurable Query Processing Hardware for Data Streams

Abstract

Yasin OGE†a) , Student Member, Masato YOSHIMI †b) , Takefumi MIYOSHI † †c) , Hideyuki KAWASHIMA † † †d) , Hidetsugu IRIE † † † †e) , Members, and Tsutomu YOSHINAGA †f) , Senior Member SUMMARYIn this paper, we propose Configurable Query Processing Hardware (CQPH), an FPGA-based accelerator for continuous query processing over data streams. CQPH is a highly optimized and minimaloverhead execution engine designed to deliver real-time response for highvolume data streams. Unlike most of the other FPGA-based approaches, CQPH provides on-the-fly configurability for multiple queries with its own dynamic configuration mechanism. With a dedicated query compiler, SQLlike queries can be easily configured into CQPH at run time. CQPH supports continuous queries including selection, group-by operation and sliding-window aggregation with a large number of overlapping sliding windows. As a proof of concept, a prototype of CQPH is implemented on an FPGA platform for a case study. Evaluation results indicate that a given query can be configured within just a few microseconds, and the prototype implementation of CQPH can process over 150 million tuples per second with a latency of less than a microsecond. Results also indicate that CQPH provides linear scalability to increase its flexibility (i.e., on-the-fly configurability) without sacrificing performance (i.e., maximum allowable clock speed). key words: FPGA, query processing, data stream, sliding-window aggregation, configurable hardware architecture IntroductionAn important and growing class of applications requires the ability to process online data streams on the fly in order to identify emerging trends in a timely manner. Data Stream Management Systems (DSMSs) [1] deal with potentially infinite streams of data that should be processed for real-time applications, executing SQL-like continuous queries [2] over data streams. It is essential for DSMSs that incoming data be processed in real time, or at least near realtime, depending on the applications' requirements. In particular, low-latency and high-throughput processing are key given the dynamic environment of data streams, queries can join and leave a streaming system at any time. It is therefore imperative for a query processing accelerator to support on-the-fly configurability for easy adaptation to the dynamic environment. To address the problem, Najafi et al. propose Flexible Query Processor (FQP) [11] for sliding-window join queries. To the best of our knowledge, however, slidingwindow aggregate queries-an important class of slidingwindow queries-are not discussed in their work [11]. Our Contribution. This paper presents Configurable Query Processing Hardware (CQPH), a highly optimized and minimal-overhead query processing engine, especially designed for sliding-window aggregate queries. CQPH is an FPGA-based query processor that contains a collection of configurable hardware modules, each of which supports (i) filtering, (ii) grouping, and (iii) aggregation. CQPH is highly optimized for performance with a fully pi...