Cost-based query scrambling for initial delays

Abstract

Remote data access from disparate sources across a wide-area network such as the Internet is problematic due to the unpredictable nature of the communications medium and the lack of knowledge about the load and potential delays at remote sites. Traditional, static, query processing approaches break down in this environment because they are unable to adapt in response to unexpected delays. Query scrambling has been proposed to address this problem. Scrambling modifies query execution plans on-the-fly when delays are encountered during runtime. In its original formulation, scrambling was based on simple heuristics, which although providing good performance in many cases, were also shown to be susceptible to problems resulting from bad scrambling decisions. In this paper we address these shortcomings by investigating ways to exploit query optimization technology to aid in making intelligent scrambling choices. We propose three different approaches to using query optimization for scrambling. These approaches vary, for example, in whether they optimize for total work or response-time, and whether they construct partial or complete alternative plans. Using a two-phase randomized query optimizer, a distributed query processing simulator, and a workload derived from queries of the TPC-D benchmark, we evaluate these different approaches and compare their ability to cope with initial delays in accessingremote sources. The results show that cost-based scrambling can effectively hide initial delays, but that in the absence of good predictions of expected delay durations, there are fundamental tradeoffs between risk aversion and effectiveness.

Daimon

“…Adaptive/dynamic query processing has also drawn the attention of researchers [4][8] [20]. The main target is to provide flexible mechanisms for adapting to the unstable nature of the underlying network during query processing.…”

Eddies

“…A key requirement of a large-scale system like Telegraph is that it function robustly in an unpredictable and constantly fluctuating environment. This unpredictability is endemic in large-scale systems, because of increased complexity in a number of dimensions: Hardware and Workload Complexity: In wide-area environments, variabilities are commonly observable in the bursty performance of servers and networks [UFA98]. These systems often serve large communities of users whose aggregate behavior can be hard to predict, and the hardware mix in the wide area is quite heterogeneous.…”

“…When a producer and consumer run at differing rates, the faster thread may block on the queue waiting for the slower thread to catch up. As in [UFA98], River is multi-threaded and can exploit barrierfree algorithms by reading from various inputs at independent rates. The River implementation we used derives from the work on Now-Sort [AAC1 97], and features efficient I/O mechanisms including pre-fetching scans, avoidance of operating system buffering, and high-performance user-level networking.…”

Efficient XML query processing in mediators

“…The execution plan may be blocked for arbitrarily long time only because one operator does not get its data. To cope with this situation, a dynamic query plan scheduling idea was presented in ( [22], [23], [2]). To improve responsiveness, caching ([ 1], [24]) and materializing schemes [9] can be used.…”

“…This leads to a mediated query system first proposed by Wiederhold [25]. Since then, research work and prototyping has been going on, such as DISCO [22], GARLIC [6], H E M E S [ 13,and TSIMMIS[lB], to name a few. With the advent of XML as the de facto standard for data representation and exchange over Internet, there are ever-increasing data integration systems using XML as a middleware model.…”