A Scalable Software Framework for Stateful Stream Data Processing on Multiple GPUs and Applications

Alghabi, F.; Schipper, U.; Kolb, Andreas

doi:10.1007/978-981-287-134-3_7

Cited by 5 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GStream supports stream processing applications in the form of a C++ library; it uses MPI to implement the data communication between different nodes and uses CUDA to conduct stream operations on GPUs. Alghabi et al [10] first introduced the concept of stateful stream data processing on a node with multiple GPUs. Nguyen et al [63] considered the scalability with the number of GPUs on a single node, and developed a GPU performance model for stream workload partitioning in multi-GPU platforms with high scalability.…”

Section: Gpu-enabled Stream Processingmentioning

confidence: 99%

Hardware-Conscious Stream Processing

Zhang

et al. 2020

SIGMOD Rec.

View full text Add to dashboard Cite

Data stream processing systems (DSPSs) enable users to express and run stream applications to continuously process data streams. To achieve realtime data analytics, recent researches keep focusing on optimizing the system latency and throughput. Witnessing the recent great achievements in the computer architecture community, researchers and practitioners have investigated the potential of adoption hardware-conscious stream processing by better utilizing modern hardware capacity in DSPSs. In this paper, we conduct a systematic survey of recent work in the field, particularly along with the following three directions: 1) computation optimization, 2) stream I/O optimization, and 3) query deployment. Finally, we advise on potential future research directions.

show abstract

Section: Gpu-enabled Stream Processingmentioning

confidence: 99%

Hardware-Conscious Stream Processing

Zhang

et al. 2020

SIGMOD Rec.

View full text Add to dashboard Cite

show abstract

“…As an application example in the area of crystallography, the framework has already been successfully applied for correcting pnCCD images [22]. See [20,23] for more information on the scope of problems that can be addressed by this kind of (multi-)GPU software framework.…”

Section: Gpu K+1mentioning

confidence: 99%

Fast GPU-based spot extraction for energy-dispersive X-ray Laue diffraction

et al. 2014

View full text Add to dashboard Cite

This paper describes a novel method for fast online analysis of X-ray Laue spots taken by means of an energy-dispersive X-ray 2D detector. Current pnCCD detectors typically operate at some 100 Hz (up to a maximum of 400 Hz) and have a resolution of 384 × 384 pixels, future devices head for even higher pixel counts and frame rates.The proposed online data analysis is based on a computer utilizing multiple Graphics Processing Units (GPUs), which allow for fast and parallel data processing. Our multi-GPU based algorithm is compliant with the rules of stream-based data processing, for which GPUs are optimized. The paper's main contribution is therefore an alternative algorithm for the determination of spot positions and energies over the full sequence of pnCCD data frames. Furthermore, an improved background suppression algorithm is presented.The resulting system is able to process data at the maximum acquisition rate of 400 Hz. We present a detailed analysis of the spot positions and energies deduced from a prior (single-core) CPU-based and the novel GPU-based data processing, showing that the parallel computed results using the GPU implementation are at least of the same quality as prior CPU-based results. Furthermore, the GPU-based algorithm is able to speed up the data processing by a factor of 7 (in comparison to single-core CPU-based algorithm) which effectively makes the detector system more suitable for online data processing.

show abstract

“…Areas such as medical physics [7,8], high-energy physics [9][10][11] and lattice quantum chromodynamics [12,13] have also witnessed increasing use of GPUs for acceleration of scientific computing. Recently, a GPU data processing scheme was successfully applied to accelerate the extraction of the Laue spots' positions and energies from a pnCCD data set for energy-dispersive Laue diffraction of hen egg-white lysozyme, resulting in a 7 times faster processing time [14][15][16].…”

Section: Introductionmentioning

confidence: 99%

Fast GPU-based absolute intensity determination for energy-dispersive X-ray Laue diffraction

et al. 2016

View full text Add to dashboard Cite

A: This paper presents a novel method for fast determination of absolute intensities in the sites of Laue spots generated by a tetragonal hen egg-white lysozyme crystal after exposure to white synchrotron radiation during an energy-dispersive X-ray Laue diffraction experiment. The Laue spots are taken by means of an energy-dispersive X-ray 2D pnCCD detector. Current pnCCD detectors have a spatial resolution of 384 × 384 pixels of size 75 × 75 µm 2 each and operate at a maximum of 400 Hz. Future devices are going to have higher spatial resolution and frame rates.The proposed method runs on a computer equipped with multiple Graphics Processing Units (GPUs) which provide fast and parallel processing capabilities. Accordingly, our GPU-based algorithm exploits these capabilities to further analyse the Laue spots of the sample. The main contribution of the paper is therefore an alternative algorithm for determining absolute intensities of Laue spots which are themselves computed from a sequence of pnCCD frames. Moreover, a new method for integrating spectral peak intensities and improved background correction, a different way of calculating mean count rate of the background signal and also a new method for n-dimensional Poisson fitting are presented.We present a comparison of the quality of results from the GPU-based algorithm with the quality of results from a prior (base) algorithm running on CPU. This comparison shows that our algorithm is able to produce results with at least the same quality as the base algorithm. Furthermore, the GPU-based algorithm is able to speed up one of the most time-consuming parts of the base algorithm, which is n-dimensional Poisson fitting, by a factor of more than 3. Also, the entire procedure of extracting Laue spots' positions, energies and absolute intensities from a raw dataset of pnCCD frames is accelerated by a factor of more than 3.

show abstract

A Scalable Software Framework for Stateful Stream Data Processing on Multiple GPUs and Applications

Cited by 5 publications

References 16 publications

Hardware-Conscious Stream Processing

Hardware-Conscious Stream Processing

Fast GPU-based spot extraction for energy-dispersive X-ray Laue diffraction

Fast GPU-based absolute intensity determination for energy-dispersive X-ray Laue diffraction

Contact Info

Product

Resources

About