DEMB: Cache-Aware Scheduling for Distributed Query Processing

Abstract

Abstract. Leveraging data in distributed caches for large scale query processing applications is becoming more important, given current trends toward building large scalable distributed systems by connecting multiple heterogeneous less powerful machines rather than purchasing expensive homogeneous and very powerful machines. As more servers are added to such clusters, more memory is available for caching data objects across the distributed machines. However the cached objects are dispersed and traditional query scheduling policies that take into account only load balancing do not effectively utilize the increased cache space. We propose a new multi-dimensional range query scheduling policy for distributed query processing frameworks, called DEMB, that employs a probability distribution estimation derived from recent queries. DEMB accounts for both load balancing and the availability of distributed cached objects to both improve the cache hit rate for queries and thereby decrease query turnaround time and throughput. We experimentally demonstrate that DEMB produces better query plans and lower query response times than other query scheduling policies.

Location-Dependent Query Processing: Semantic Cache for Real-Time Smart City Analytics

“…Grid-partition index has also been used to answer the nearest neighbor query using semantic and hybrid caches [14]. Some of the benefits of using cache in distributed systems are such that they are transparent for the application and do not affect the functionality of the application that utilizes a cache, as described in [13][14][15][16]. Unlike centralized systems, data is stored in the cache in webpages.…”