RDMA-accelerated data transport in ALFA

Klein, Dennis; Rybalchenko, A.; Al-Turany, M.; Kollegger, T.

doi:10.1051/epjconf/201921405022

Cited by 2 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FairMQ offers multiple implementations of its abstract data transport interface in order to integrate existing data transport technologies. Currently, implementations for ZeroMQ [4], nanomsg[5], shared memory [6] and RDMA based transport [7] are available. The user of FairMQ can select a transport implementation at run-time per input/output channel.…”

Section: Transport Layermentioning

confidence: 99%

ALFA: A framework for building distributed applications

Al-Turany¹,

Rybalchenko²,

Klein³

et al. 2020

EPJ Web Conf.

Self Cite

View full text Add to dashboard Cite

The ALFA framework is a joint development between ALICE Online-Offline and FairRoot teams. ALFA has a distributed architecture, i.e. a collection of highly maintainable, testable, loosely coupled, independently deployable processes. ALFA allows the developer to focus on building singlefunction modules with well-defined interfaces and operations. The communication between the independent processes is handled by FairMQ transport layer. FairMQ offers multiple implementations of its abstract data transport interface, it integrates some popular data transport technologies like ZeroMQ and nanomsg. Furthermore it also provides shared memory and RDMA transport (based on libfabric) for high throughput, low latency applications. Moreover, FairMQ allows the single process to use multiple and different transports at the same time. FairMQ based processes can be controlled and orchestrated via different systems by implementing the corresponding plugin. However, ALFA delivers also the Dynamic Deployment System (DDS) as an independent set of utilities and interfaces, providing a dynamic distribution of different user processes on any Resource Management System (RMS) or a laptop. ALFA is already being tested and used by different experiments in different stages of data processing as it offers an easy integration of heterogeneous hardware and software.

show abstract

Section: Transport Layermentioning

confidence: 99%

ALFA: A framework for building distributed applications

Al-Turany¹,

Rybalchenko²,

Klein³

et al. 2020

EPJ Web Conf.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Network transports based on ZeroMQ[5] and nanomsg[6] involve at least one copy of the data for inter-process transfer. Also the new OFI transport [7] that exploits RDMA techniques for network transfers will copy the data between peers by design.…”

Section: Introductionmentioning

confidence: 99%

Shared Memory Transport for ALFA

et al. 2019

Self Cite

View full text Add to dashboard Cite

The high data rates expected for the next generation of particle physics experiments (e.g.: new experiments at FAIR/GSI and the upgrade of CERN experiments) call for dedicated attention with respect to design of the needed computing infrastructure. The common ALICE-FAIR framework ALFA is a modern software layer, that serves as a platform for simulation, reconstruction and analysis of particle physics experiments. Beside standard services needed for simulation and reconstruction of particle physics experiments, ALFA also provides tools for data transport, configuration and deployment. The FairMQ module in ALFA offers building blocks for creating distributed software components (processes) that communicate between each other via message passing. The abstract "message passing" interface in FairMQ has at the moment three implementations: ZeroMQ, nanomsg and shared memory. The newly developed shared memory transport will be presented, that provides significant performance benefits for transferring large data chunks between components on the same node. The implementation in FairMQ allows users to switch between the different transports via a trivial configuration change. The design decisions, implementation details and performance numbers of the shared memory transport in FairMQ/ALFA will be highlighted. *

show abstract

RDMA-accelerated data transport in ALFA

Cited by 2 publications

References 3 publications

ALFA: A framework for building distributed applications

ALFA: A framework for building distributed applications

Shared Memory Transport for ALFA

Contact Info

Product

Resources

About