No abstract

No abstract

No abstract

We describe an approach for a custom complex event processing engine using Message Passing Interface (MPI) in C++ programming language. Our approach utilizes a multi-processor infrastructure and distributes its load on multiple processes, expecting each process to run on one processor. A dispatching process receives events and distributes them on several query processes which are responsible for updating the actual queries. Query processes forwards any updates to a presentation process that output the results in an appropriate format. The distribution of roles among processes allows better scalability since further query processes can be added dynamically to handle more queries. In our evaluation we measured event processing up to a throughput of 12k events/sec using 4 processor cores.

New memory technologies are blurring the previously distinctive performance characteristics of adjacent layers in the memory hierarchy. No longer are such layers orders of magnitude different in request latency or capacity. Beyond the traditional single-layer view of caching, we now must re-cast the problem as a data placement challenge: which data should be cached in faster memory if it could instead be served directly from slower memory? We present Chopt, an offline algorithm for data placement across multiple tiers of memory with asymmetric read and write costs. We show that Chopt is optimal and can therefore serve as the upper bound of performance gain for any data placement algorithm. We also demonstrate an approximation of Chopt which makes its execution time for long traces practical using spatial sampling of requests incurring a small 0.2% average error on representative workloads at a sampling ratio of 1%. Our evaluation of Chopt on more than 30 production traces and benchmarks shows that optimal data placement decisions could improve average request latency by 8.2%-44.8% when compared with the long-established gold standard: Belady and Mattson's offline, evict-farthest-in-the-future optimal algorithms. Our results identify substantial improvement opportunities for future online memory management research.