Integration of FPGA RDMA into the ATLAS readout with FELIX in High Luminosity LHC

Vasile, M. E.; Martoiu, Sorin; Boukhadida, N.; Stoicea, G.; Micu, P.; Dumitru, A.; Ulmamei, A.; Hobincu, Radu; Iordache, Cosmin

doi:10.1088/1748-0221/18/01/c01025

Cited by 4 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Continuous flow transfers can not be safely implemented when using RDMA Write, unless clients account for the consumption of received data. First, as seen from the results in [2], the FPGA RDMA implementation can reach bandwidths close to the theoretical maximum of a 100 Gb/s link. In GB/s, that is 12.5 GB/s, which would mean that 125 GB of memory could be filled in 10 seconds.…”

Section: Jinst 19 C03034mentioning

confidence: 75%

See 1 more Smart Citation

Performance profiling and design choices of an RDMA implementation using FPGA devices

Vasile,

Iancu,

Martoiu

et al. 2024

J. Inst.

Self Cite

View full text Add to dashboard Cite

RDMA communication is an efficient choice for many applications, such as data acquisition systems, data center networking and any other networking application, where high bandwidth and low latency are necessary. RDMA can be implemented using a large array of options, which need to be tailored to the needed use case, in order to get optimal results. Aspects such as the effects of using multiple simultaneous connections, using various transport functions such as RDMA Write and RDMA Send and communication models such as sending individual bursts or continuous streams of data will be investigated for implementing RDMA on FPGA devices.

show abstract

Section: Jinst 19 C03034mentioning

confidence: 75%

“…In previous work [1,2], all the tests have been run only on individual bursts of data having size equal to message size * message count. However, in a production setting, this is not enough.…”

mentioning

confidence: 99%

Performance profiling and design choices of an RDMA implementation using FPGA devices

Vasile,

Iancu,

Martoiu

et al. 2024

J. Inst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the Phase II upgrade, research work has started in ATLAS to study the usage of RDMA inside the FPGA. Promising first studies have been performed, getting close to the theoretical bandwidth limit of the communication link [5].…”

Section: The Atlas Daq Upgradesmentioning

confidence: 90%

A novel approach for FPGA-to-server data transmission over an Ethernet-based network using the eXpress Data Path technology

et al. 2023

View full text Add to dashboard Cite

In the context of the upgrade of the Large Hadron Collider at CERN for high-luminosity operation, the particle detectors have to cope with much higher data rates and therefore need to upgrade their data acquisition systems. This upgrade is taken as an opportunity to exchange the currently used highly customized hardware by commercial solutions. Nevertheless, some part of the data processing still needs to be done within Field Programmable Gate Arrays (FPGA), requiring the transfer of data between the FPGAs and the commercial servers. This paper reports on a study of direct data transmission from FPGAs to servers via a commercial network. Large data buffers as required for reliable data-transmission protocols are avoided by using an emerging technique named eXpress Data Path (XDP). Based on XDP, the transmission of 5.2 PB (i.e. 2.92 × 1012 packets) was achieved within 168 h without a single missing packet.

show abstract

“…In addition to the studies mentioned above, there are other studies [13][14][15][16][17][18] that have also compared TCP with RDMA technology. It can also be seen from the research conclusions of these documents that the RDMA protocol significantly reduces the number of memory copies due to its zero-copy and kernel bypass characteristics.…”

Section: Comparison Of Rdma Technology and Tcpmentioning

confidence: 99%

“…It reduces computing resource consumption while guaranteeing extremely low transmission latency. However, most of the current optimization work of RDMA for distributed file systems only uses RDMA libraries or plug-ins as communication submodules of the system, and does not make full use of the excellent features of RDMA [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27]. Accelio, the RDMA middleware used in CephFS, encapsulates the underlying RDMA network functions and only provides an abstract call interface [28,29].…”

Section: Introductionmentioning

confidence: 99%

Design and Optimization of a Distributed File System Based on RDMA

Gao

Zhang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

With the rapid development of computer and network technology, to deal with explosively growing mass data correctly, distributed file systems have come into being, and they have been applied in many enterprises. However, in the packet storage process, the traditional TCP/IP transmission protocol has to pass through the operating system and application software layer. It takes up many server resources and memory bus bandwidth, which places a heavy burden on the server’s CPU and memory. In this paper, we analyze the research status of the distributed file system based on analyzing and comparing the advantages of RDMA over traditional TCP Ethernet communication architecture. We propose an optimization scheme for the network transmission performance of the original FastDFS to replace the original TCP module with an RDMA network communication module, and we design a lightweight distributed file system with higher performance than the original FastDFS, which is suitable for small file storage. Using the zero-copy technology and kernel bypass technology of soft RDMA, the network communication mechanism in FastDFS is optimized to free the server’s CPU from heavy data processing work. In this paper, the RDMA-based FastDFS is compared with the original FastDFS. The test results show that the network transmission performance of the optimized FastDFS is improved compared with the original FastDFS, which achieves the expected effect and can better meet the requirements of mass small file storage for today’s Internet applications.

show abstract

Integration of FPGA RDMA into the ATLAS readout with FELIX in High Luminosity LHC

Cited by 4 publications

References 5 publications

Performance profiling and design choices of an RDMA implementation using FPGA devices

Performance profiling and design choices of an RDMA implementation using FPGA devices

A novel approach for FPGA-to-server data transmission over an Ethernet-based network using the eXpress Data Path technology

Design and Optimization of a Distributed File System Based on RDMA

Contact Info

Product

Resources

About