Energy-efficient data transfers in radio astronomy with software UDP RDMA

Abstract

Modern radio astronomy relies on very large amounts of data that need to be transferred between various parts of astronomical instruments, over distances that are often in the range of tens or hundreds of kilometres. The Square Kilometre Array (SKA) will be the world's largest radio telescope, data rates between its components will exceed Terabits per second. This will impose a huge challenge on its data transport system, especially with regard to power consumption. High-speed data transfers using modern off-the-shelf hardware may impose a significant load on the receiving system with respect to CPU and DRAM usage. The SKA has a strict energy budget which demands a new, custom-designed data transport solution. In this paper we present SoftiWARP UDP, an unreliable datagram-based Remote Direct Memory Access (RDMA) protocol, which can significantly increase the energy-efficiency of high-speed data transfers for radio astronomy. We have implemented a fully functional software prototype of such a protocol, supporting RDMA Read and Write operations and zero-copy capabilities. We present mea- * Preprint submitted to Future Generation Computer Systems surements of power consumption and achieved bandwidth and investigate the behaviour of all examined protocols when subjected to packet loss.

Evaluation of HPC Acceleration and Interconnect Technologies for High-Throughput Data Acquisition

“…Efficient data movement in multi-node systems is a crucial issue at the crossroads of related areas such as scientific data acquisition and computing, big data, and highperformance computing (HPC). This is strikingly confirmed by a plethora of previous works in the technical literature which point out the key role of high-performance interconnect technologies, e.g., for data acquisition, in demanding applications ranging from high-energy physics [1][2][3] to astronomy [4,5]. At the same time, these applications pose unprecedented processing requirements for the vast amount of data being generated and transferred, increasingly leading to the adoption of accelerator-based platforms, e.g., using graphics processing units (GPUs) or field-programmable gate arrays (FPGAs) in order to meet stringent energy-efficiency requirements through some form of customized computing.…”

“…In fact, the development of an efficient infrastructure for data communication is a common problem addressed by the work plans of all major facilities and international projects supporting large-scale scientific experiments. For example, several works that have appeared in the literature in recent years involve interconnects for data communication in high-energy physics (HEP), e.g., for the Large Hadron Collider (LHC) at CERN [1,3], data acquisition from 2D X-ray detectors in the RASHPA platform at the European Synchrotron Radiation Facility (ESRF) [6], from gain adaptive detectors used for macromolecular crystallography [8], or from superheated emulsion detectors, e.g., based on FPGA multi-channel acquisition systems [9], as well as projects dealing with astronomic data, including the Square Kilometer Area (SKA) telescope [4] or specific applications such as adaptive optics [5].…”

RDMA data transfer and GPU acceleration methods for high-throughput online processing of serial crystallography images

“…We did not evaluate this solution as it relies on the TCP stack, which is incompatible with detector electronics. Interestingly, there are preliminary studies (Grant et al, 2015;Lenkiewicz et al, 2018) of a UDP/iWARP implementation which deserve further investigation.…”