An adaptive parallel query processing middleware for the Grid

Abstract

SUMMARYGrid services provide an important abstract layer on top of heterogeneous components (hardware and software) that take part in a Grid environment. In this scenario, applications such as scientific visualization require access to data of non-conventional data types, such as fluid path geometry, and the evaluation of special user programs and algebraic operators, such as spatial hash-join, on these data. In order to support such applications we are developing a Configurable Data Integration Middleware System for the Grid (CoDIMS-G). CoDIMS-G provides a query execution environment adapted to the heterogeneity and variations found in a Grid environment by offering a node scheduling algorithm and an adaptive query execution strategy. The latter both adapts to performance variations in a scheduled node and deals efficiently with repetitive evaluation of a query execution plan fragment, as needed for computing a particle's, trajectory.

QEF - Supporting Complex Query Applications

“…We extended QEF to support a preprocessing stage of a scientific visualization application [6]. The application simulates a fluid path flow by computing the trajectory of virtual particles in a geometry based representation of a path.…”

“…The scientific visualization application [6] brought requirements for extensions on both QEF perspectives: data and task modeling and execution semantics. The second QEF instance, the semantic web services search engine [7], adopts a query processing strategy for coping with the web service discovery problem.…”

Resource Allocation for Query Optimization in Data Grid Systems: Static Load Balancing Strategies

“…The overloading problem caused by the above constraints is currently addressed by adding a resource reallocation phase (which is called dynamic resource allocation) during the query execution [10][11][12][13][14][15], that is, to move part of the work from overloaded resources to less loaded resources. The dynamic phase is very important, not only because the static allocation result may be under optimal, but also because in the data grid system, nodes may leave or enter at any time.…”

“…There exist two main approaches: centralized and decentralized. In the central-ized approach, the workload status of the nodes is monitored by a dedicated resource broker [10][11][12][13][14]. In the decentralized approach, each node detects if it is over-loaded and makes autonomously the decision of moving operations from it to other nodes [15].…”

Resource allocation algorithm for a relational join operator in grid systems

“…Although these studies have common objectives, they provide different solutions for different types of parallelism for query processing. The studies [2,16,17,20,22] consider three types of parallelism, independent, pipelined and partitioned parallelism in allocating resources; whereas, the studies [10][11][12][13] propose resource allocation algorithms addressing only the partitioned parallelism.…”