Comparative Performance Evaluation of High-performance Data Transfer Tools

Nadig, Deepak; Jung, Eun-Sung; Kettimuthu, Rajkumar; Fosterz, Ian; Rao, S. V. Nageswara; Ramamurthy, Byrav

doi:10.1109/ants.2018.8710071

Cited by 4 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The existence of Globus deployment at the HPC facilities also allows for the easy assembly of data sets published on data portals using the Modern Research Data Portal (MRDP) [19] design pattern, such as the NCAR/UCAR Research Data Archive [20], which also use Globus. A variety of other data transfer tools and platforms also exist in the broader scientific community -an in-depth evaluation of several such tools is provided by [34].…”

Section: Data Transfer Softwarementioning

confidence: 99%

The Petascale DTN Project: High Performance Data Transfer for HPC Facilities

Dart,

Allcock,

Bhimji

et al. 2021

Preprint

View full text Add to dashboard Cite

The movement of large-scale (tens of Terabytes and larger) data sets between high performance computing (HPC) facilities is an important and increasingly critical capability. A growing number of scientific collaborations rely on HPC facilities for tasks which either require large-scale data sets as input or produce large-scale data sets as output. In order to enable the transfer of these data sets as needed by the scientific community, HPC facilities must design and deploy the appropriate data transfer capabilities to allow users to do data placement at scale. This paper describes the Petascale DTN Project, an effort undertaken by four HPC facilities, which succeeded in achieving routine data transfer rates of over 1PB/week between the

show abstract

Section: Data Transfer Softwarementioning

confidence: 99%