Resource allocation algorithm for a relational join operator in grid systems

Abstract

Grid systems become very popular during the last decade because of their rapidly increasing computational capabilities. On the other hand, the advances on different domains cause enormous increase in the scale of the manipulated data. This issue augments the importance of distributed query processing and causes researchers to port their underlying environment onto the grid systems. However the dynamicity, heterogeneity and large scale characteristics of grid systems pose new problems for the distributed query processing domain. Resource allocation for query processing in grid systems is one of these problems, which attracts many researchers' attention. In this paper, we propose a new resource allocation algorithm for one relational join operator in a query considering characteristics of the grid systems. We provide theoretical analyses of the proposed algorithm and we consolidate analyses with the simulations.

Fault-tolerant resource allocation for query processing in grid environments

“…In this section, we propose an algorithm for FTRA for query processing in grid environments. To realise this, we first propose an IRA algorithm by extending the SJORA algorithm which is presented by Cokuslu et al (2012b). Then, after setting the IRA, we propose FTRA algorithm by introducing a fault-tolerance module that realises passive replication of stateful operators in the queries.…”

“…In SJORA algorithm, Cokuslu et al (2012b) find and allocate suitable resources for queries consisting of a single join operator assuming scan tasks are already allocated in the nodes in which the base relations reside. Since queries may consist of more than one join operator, it is necessary to extend the SJORA algorithm by materialising the allocation of all tasks in the entire query.…”

“…For simplicity, in the rest of this paper, these pairs of build and probe tasks are named as join tasks. In SJORA algorithm, Cokuslu et al (2012b) assumed that the partitions of base relations are known at the end of the resource discovery stage. However, for the temporary relations, it is not possible to find out their physical locations before precedent join tasks are allocated.…”

“…The IRA algorithm uses the SJORA algorithm (Cokuslu et al, 2012b) for each join operator in the query tree. Since the message complexity of the SJORA algorithm is O(nN 2 ), the total message complexity of the IRA algorithm is O(jnN 2 ) for a query that consists j join operators.…”

“…The contribution of this section is therefore twofold. First, we propose the IRA algorithm by taking advantages of Single Join Operator Resource Allocation (SJORA) algorithm which is presented by Cokuslu et al (2012b). Second, we propose the FTRA algorithm that presents fault-tolerance for stateful query operators in grid environments.…”