Implementation of a Cyber Transformer for Parallel Download in Co-Allocation Data Grid Environments

Yang, Chao‐Tung; Yang, Ming-Feng; Chiang, Wen-Chung

doi:10.1109/gcc.2008.111

Cited by 11 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If an attempt is made to replicate the same file to a third site in the same region, BHR will see that there is already a duplicate file in the region and terminate. Thus, files will have at most two copies in each region, which means only two links will be available for parallel data downloading [1,2,[30][31][32][33][34] in any one region. This limitation leads to high time costs, thus reducing the effectiveness of an important grid computing feature and adversely affecting overall performance.…”

Section: Replica Managementmentioning

confidence: 99%

“…However, data grid technologies, developed to solve these kinds of problems, offer an effective alternative means of utilizing largescale computing power and storage capacities to compute and store data. Grids [11,12,20,21,27,[29][30][31][32][33][34] enable sharing of computing power and storage capacities geographically distributed around the world such that they work together as tremendous virtual computers [1,2] on experiments and simulations. The Globus Toolkit [13,27] is open-source software for building data grid environments.…”

mentioning

confidence: 99%

See 1 more Smart Citation

File replication, maintenance, and consistency management services in data grids

Yang

Hsu

2009

J Supercomput

Self Cite

View full text Add to dashboard Cite

Data replication and consistency refer to the same data being stored in distributed sites, and kept consistent when one or more copies are modified. A good file maintenance and consistency strategy can reduce file access times and access latencies, and increase download speeds, thus reducing overall computing times. In this paper, we propose dynamic services for replicating and maintaining data in grid environments, and directing replicas to appropriate locations for use. To address a problem with the Bandwidth Hierarchy-based Replication (BHR) algorithm, a strategy for maintaining replicas dynamically, we propose the Dynamic Maintenance Service (DMS). We also propose a One-way Replica Consistency Service (ORCS) for data grid environments, a positive approach to resolving consistency maintenance issues we hope will strike a balance between improving data access performance and replica consistency. Experimental results show that our services are more efficient than other strategies.

show abstract

Section: Replica Managementmentioning

confidence: 99%

mentioning

confidence: 99%

File replication, maintenance, and consistency management services in data grids

Yang

Hsu

2009

J Supercomput

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors in [24][25][26][27][28] propose a recursive-adjustment co-allocation scheme, to reduce the idle time of faster servers waiting for the slowest server to finish the final blocks. The proposed scheme continuously adjusts the replica server's workload to affect its real-time network bandwidth.…”

Section: Related Workmentioning

confidence: 99%

“…In the case of downloading from dynamic servers, dynamic co-allocation technique is used to dynamically select source servers according to the current network situation and the server loads. The dynamic co-allocation problem has been addressed in many researches [7,8,[24][25][26][27][28][29][30][31][32]. All these researches focus on finding a perfect dynamic co-allocation algorithm that can achieve high download speed from multiple source servers in a dynamic changing environment where server may be dead or slow down over time.…”

Section: Related Workmentioning

confidence: 99%

Improving job scheduling performance with parallel access to replicas in Data Grid environment

et al. 2009

View full text Add to dashboard Cite

Data Grid has evolved to be the solution for data-intensive applications, such as High Energy Physics (HEP), astrophysics, and computational genomics. These applications usually have large input of data to be analyzed and these input data are widely replicated across Data Grid to improve the performance. The job scheduling performance on traditional computing jobs can be studied using queuing theory. However, with the addition of data transfer, the job scheduling performance is too complex to be modeled. In this research, we study the impact of data transfer on the performance of job scheduling in the Data Grid environment. We have proposed a parallel downloading system that supports replicating data fragments and parallel downloading of replicated data fragments, to improve the job scheduling performance. The performance of the parallel downloading system is compared with non-parallel downloading system, using three scheduling heuristics: Shortest Turnaround Time (STT), Least Relative Load (LRL) and Data Present (DP). Our simulation results show that the proposed parallel download approach greatly improves the Data Grid performance for all three scheduling algorithms, in terms of the geometric mean of job turnaround time. The advantage of parallel downloading system is most evident when the Data Grid has relatively low network bandwidth and relatively high computing power.

show abstract

“…One method for improving download speeds uses replica selection techniques to determine the best replica locations (Chervenak et al, 2001Czajkowski et al, 1999Czajkowski et al, , 2001Foster and Kesselman, 1997;Yang et al, , 2008Zhang et al, 2003;Vazhkudai andSchopf, 2002, 2003;Yang et al, 2006). However, downloading data sets from single best servers often results in ordinary transfer rates because bandwidth quality varies unpredictably due to the shared nature of the Internet.…”

Section: Introductionmentioning

confidence: 97%

Enhancement of anticipative recursively adjusting mechanism for redundant parallel file transfer in data grids

Yang

Chiang

2009

Journal of Network and Computer Applications

Self Cite

View full text Add to dashboard Cite

Implementation of a Cyber Transformer for Parallel Download in Co-Allocation Data Grid Environments

Cited by 11 publications

References 24 publications

File replication, maintenance, and consistency management services in data grids

File replication, maintenance, and consistency management services in data grids

Improving job scheduling performance with parallel access to replicas in Data Grid environment

Enhancement of anticipative recursively adjusting mechanism for redundant parallel file transfer in data grids

Contact Info

Product

Resources

About