Data Flow Computing in Geoscience Applications

Gan, Lin; Fu, Haohuan; Mencer, Oskar; Luk, Wayne; Yang, Guangwen

doi:10.1016/bs.adcom.2016.09.005

Cited by 10 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using reconfigurable hardware accelerators rather than CPUs helps to deal iteratively with large volumes of data. DFEs have recently been successfully applied to a wide range of scientific problems, including geoscience (Gan et al, 2017), fluid-dynamics (Düben et al, 2015), artificial neural networks (Liang et al, 2018), quantum chemistry (Cooper et al, 2017), and genomics (Arram et al, 2015).…”

Section: Data Modelmentioning

confidence: 99%

See 1 more Smart Citation

Fast sampling from Wiener posteriors for image data with dataflow engines

Jeffrey

Heavens

Fortio

2018

Astronomy and Computing

View full text Add to dashboard Cite

We use Dataflow Engines (DFE) to construct an efficient Wiener filter of noisy and incomplete image data, and to quickly draw probabilistic samples of the compatible true underlying images from the Wiener posterior. Dataflow computing is a powerful approach using reconfigurable hardware, which can be deeply pipelined and is intrinsically parallel. The unique Wiener-filtered image is the minimumvariance linear estimate of the true image (if the signal and noise covariances are known) and the most probable true image (if the signal and noise are Gaussian distributed). However, many images are compatible with the data with different probabilities, given by the analytic posterior probability distribution referred to as the Wiener posterior. The DFE code also draws large numbers of samples of true images from this posterior, which allows for further statistical analysis. Naive computation of the Wiener-filtered image is impractical for large datasets, as it scales as n 3 , where n is the number of pixels. We use a messenger field algorithm, which is well suited to a DFE implementation, to draw samples from the Wiener posterior, that is, with the correct probability we draw samples of noiseless images that are compatible with the observed noisy image. The Wiener-filtered image can be obtained by a trivial modification of the algorithm. We demonstrate a lower bound on the speed-up, from drawing 10 5 samples of a 128 2 image, of 11.3 ± 0.8 with 8 DFEs in a 1U MPC-X box when compared with a 1U server presenting 32 CPU threads. We also discuss a potential application in astronomy, to provide better dark matter maps and improved determination of the parameters of the Universe.

show abstract

Section: Data Modelmentioning

confidence: 99%

“…Time to complete a task is also only one metric of performance among other metrics. Lower clock frequencies mean that DFEs use less power than conventional CPU machines (Gan et al, 2017).…”

Section: Dfe Systemmentioning

confidence: 99%

Fast sampling from Wiener posteriors for image data with dataflow engines

Jeffrey

Heavens

Fortio

2018

Astronomy and Computing

View full text Add to dashboard Cite

show abstract

“…The Multiscale Dataflow computing paradigm developed by Maxeler Technologies as a combination of hardware and software components has been successfully demonstrated to provide substantial acceleration of real applications in a variety of domains. These include computational finance, accelerating stochastic algorithms for Monte Carlo simulations for credit models and interest rate derivative payoff evaluations based on the Heath–Jarrow–Morton model, computational biology, accelerating short read alignment, , geoscience, accelerating the finite difference algorithm for modeling wave propagation, − atmospheric modeling, solving Euler atmospheric equations, , and more.…”

Section: Dataflow Computingmentioning

confidence: 99%

Quantum Chemistry in Dataflow: Density-Fitting MP2

Cooper

Girdlestone

Burovskiy

et al. 2017

J. Chem. Theory Comput.

View full text Add to dashboard Cite

We demonstrate the use of dataflow technology in the computation of the correlation energy in molecules at the Møller-Plesset perturbation theory (MP2) level. Specifically, we benchmark density fitting (DF)-MP2 for as many as 168 atoms (in valinomycin) and show that speed-ups between 3 and 3.8 times can be achieved when compared to the MOLPRO package run on a single CPU. Acceleration is achieved by offloading the matrix multiplications steps in DF-MP2 to Dataflow Engines (DFEs). We project that the acceleration factor could be as much as 24 with the next generation of DFEs.

show abstract

“…Since then, the increase of computing power is largely from an increased density of computing units in the processors. As a result, for the leading-edge supercomputers that were developed after 2010, a major part of computing power is provided by many-core accelerators, such as NVIDIA GPUs (Vazhkudai et al, 2018), Intel Xeon Phi MICs (Liao et al, 2014), and even reconfigurable FPGAs (Gan et al, 2017;Chen, 2019). As a result, recent machines contain two major architectural changes.…”

Section: Introductionmentioning

confidence: 99%

Optimizing High-Resolution Community Earth System Model on a Heterogeneous Many-Core Supercomputing Platform (CESM-HR_sw1.0)

Zhang¹,

Fu²,

Wu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Abstract. With the semi-conductor technology gradually approaching its physical and heat limits, recent supercomputers have adopted major architectural changes to continue increasing the performance through more power-efficient heterogeneous many-core systems. Examples include Sunway TaihuLight that has four Management Processing Element (MPE) and 256 Computing Processing Element (CPE) inside one processor and Summit that has two central processing units (CPUs) and 6 graphics processing units (GPUs) inside one node. Meanwhile, current high-resolution Earth system models that desperately require more computing power, generally consist of millions of lines of legacy codes developed for traditional homogeneous multi-core processors and cannot automatically benefit from the advancement of supercomputer hardware. As a result, refactoring and optimizing the legacy models for new architectures become a key challenge along the road of taking advantage of greener and faster supercomputers, providing better support for the global climate research community and contributing to the long-lasting society task of addressing long-term climate change. This article reports the efforts of a large group in the International Laboratory for High-Resolution Earth System Prediction (iHESP) established by the cooperation of Qingdao Pilot National Laboratory for Marine Science and Technology (QNLM), Texas A & M University and the National Center for Atmospheric Research (NCAR), with the goal of enabling highly efficient simulations of the high-resolution (25-km atmosphere and 10-km ocean) Community Earth System Model (CESM-HR) on Sunway TaihuLight. The refactoring and optimizing efforts have improved the simulation speed of CESM-HR from 1 SYPD (simulation years per day) to 3.4 SYPD (with output disabled), and supported several hundred years of pre-industrial control simulations. With further strategies on deeper refactoring and optimizing for a few remaining computing hot spots, we expect an equivalent or even better efficiency than homogeneous CPU platforms. The refactoring and optimizing processes detailed in this paper on the Sunway system should have implications to similar efforts on other heterogeneous many-core systems such as GPU-based high-performance computing (HPC) systems.

show abstract

Data Flow Computing in Geoscience Applications

Cited by 10 publications

References 16 publications

Fast sampling from Wiener posteriors for image data with dataflow engines

Fast sampling from Wiener posteriors for image data with dataflow engines

Quantum Chemistry in Dataflow: Density-Fitting MP2

Optimizing High-Resolution Community Earth System Model on a Heterogeneous Many-Core Supercomputing Platform (CESM-HR_sw1.0)

Contact Info

Product

Resources

About