A high-throughput readout architecture based on PCI-Express Gen3 and DirectGMA technology

Rota, Lorenzo; Vogelgesang, Matthias; Perez, L.E. Ardila; Caselle, M.; Chilingaryan, S.; Dritschler, T.; Zilio, N.; Kopmann, A.; Balzer, M.; Weber, M.

doi:10.1088/1748-0221/11/02/p02007

Cited by 8 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The two PCIe endpoints can be directly configured as a single physical x16 lane with data throughput up to 120 Gb/s. The firmware architecture consists of "KIT DMA" logic [6] coupled to the PCIe [7] core and the DDR3 memory controller. The firmware architecture is shown in figure 3.…”

Section: Readout Card and Direct Memory Access Architecturementioning

confidence: 99%

A high-speed DAQ framework for future high-level trigger and event building clusters

et al. 2017

Self Cite

View full text Add to dashboard Cite

Modern data acquisition and trigger systems require a throughput of several GB/s and latencies of the order of microseconds. To satisfy such requirements, a heterogeneous readout system based on FPGA readout cards and GPU-based computing nodes coupled by InfiniBand has been developed. The incoming data from the back-end electronics is delivered directly into the internal memory of GPUs through a dedicated peer-to-peer PCIe communication. High performance DMA engines have been developed for direct communication between FPGAs and GPUs using “DirectGMA (AMD)” and “GPUDirect (NVIDIA)” technologies. The proposed infrastructure is a candidate for future generations of event building clusters, high-level trigger filter farms and low-level trigger system. In this paper the heterogeneous FPGA-GPU architecture will be presented and its performance be discussed.

show abstract

Section: Readout Card and Direct Memory Access Architecturementioning

confidence: 99%

A high-speed DAQ framework for future high-level trigger and event building clusters

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use an FPGA-based data transfer infrastructure based on the Direct Memory Access (DMA) architecture described in [8]. The FPGA gets loaded with our test-data in advance and is programmed with custom DMA firmware that enables low latency data transfer from the FPGA directly into the GPU's memory [9,10]. The data is assumed to be compressed, making use of the detector's finite resolution.…”

Section: Implementation Of the Gpu Track Triggermentioning

confidence: 99%

Evaluation of GPUs as a level-1 track trigger for the High-Luminosity LHC

et al. 2017

Self Cite

View full text Add to dashboard Cite

In this work, we investigate the use of GPUs as a way of realizing a low-latency, high-throughput track trigger, using CMS as a showcase example. The CMS detector at the Large Hadron Collider (LHC) will undergo a major upgrade after the long shutdown from 2024 to 2026 when it will enter the high luminosity era. During this upgrade, the silicon tracker will have to be completely replaced. In the High Luminosity operation mode, luminosities of 5 -7 × 10 34 cm −2 s −1 and pileups averaging at 140 events, with a maximum of up to 200 events, will be reached. These changes will require a major update of the triggering system. The demonstrated systems rely on dedicated hardware such as associative memory ASICs and FPGAs. We investigate the use of GPUs as an alternative way of realizing the requirements of the L1 track trigger. To this end we implemeted a Hough transformation track finding step on GPUs and established a low-latency RDMA connection using the PCIe bus. To showcase the benefits of floating point operations, made possible by the use of GPUs, we present a modified algorithm. It uses hexagonal bins for the parameter space and leads to a more truthful representation of the possible track parameters of the individual hits in Hough space. This leads to fewer duplicate candidates and reduces fake track candidates compared to the regular approach. With data-transfer latencies of 2 µs and processing times for the Hough transformation as low as 3.6 µs, we can show that latencies are not as critical as expected. However, computing throughput proves to be challenging due to hardware limitations. K : Trigger concepts and systems (hardware and software), Trigger algorithms, Computing (architecture, farms, GRID for recording, storage, archiving, and distribution of data), Data processing methods 1Corresponding author.

show abstract

“…The UFO framework is a solid starting point for a board support package as the hardware of the camera platform is also used for very different embedded applications like beam monitoring systems. The integration of microcontroller-based systems offers a good enhancement of the existing functionality and provides the option to deploy the entire system to other hardware platforms [5,6].…”

Section: Selected Elements Of the Dts Platformmentioning

confidence: 99%

The Common Data Acquisition Platform in the Helmholtz Association

et al. 2017

Self Cite

View full text Add to dashboard Cite

Various centres of the German Helmholtz Association (HGF) started in 2012 to develop a modular data acquisition (DAQ) platform, covering the entire range from detector readout to data transfer into parallel computing environments. This platform integrates generic hardware components like the multi-purpose HGF-Advanced Mezzanine Card or a smart scientific camera framework, adding user value with Linux drivers and board support packages. Technically the scope comprises the DAQ-chain from FPGA-modules to computing servers, notably frontend-electronicsinterfaces, microcontrollers and GPUs with their software plus high-performance data transmission links. The core idea is a generic and component-based approach, enabling the implementation of specific experiment requirements with low effort. This so called DTS-platform will support standards like MTCA.4 in hard-and software to ensure compatibility with commercial components. Its capability to deploy on other crate standards or FPGA-boards with PCI express or Ethernet interfaces remains an essential feature.Competences of the participating centres are coordinated in order to provide a solid technological basis for both research topics in the Helmholtz Programme "Matter and Technology": "Detector Technology and Systems" and "Accelerator Research and Development". The DTSplatform aims at reducing costs and development time and will ensure access to latest technologies for the collaboration. Due to its flexible approach, it has the potential to be applied in other scientific programs. K: Data acquisition concepts; Software architectures (event data models, frameworks and databases); Detector control systems (detector and experiment monitoring and slow-control systems, architecture, hardware, algorithms, databases); Image reconstruction in medical imaging 1Corresponding author.

show abstract

A high-throughput readout architecture based on PCI-Express Gen3 and DirectGMA technology

Cited by 8 publications

References 13 publications

A high-speed DAQ framework for future high-level trigger and event building clusters

A high-speed DAQ framework for future high-level trigger and event building clusters

Evaluation of GPUs as a level-1 track trigger for the High-Luminosity LHC

The Common Data Acquisition Platform in the Helmholtz Association

Contact Info

Product

Resources

About