First Steps in Porting the LFRic Weather and Climate Model to the FPGAs of the EuroExa Architecture

Abstract

In recent years, there has been renewed interest in the use of field-programmable gate arrays (FPGAs) for high-performance computing (HPC). In this paper, we explore the techniques required by traditional HPC programmers in porting HPC applications to FPGAs, using as an example the LFRic weather and climate model. We report on the first steps in porting LFRic to the FPGAs of the EuroExa architecture. We have used Vivado High-Level Syntheusywwi to implement a matrix-vector kernel from the LFRic code on a Xilinx UltraScale+ development board containing an XCZU9EG multiprocessor system-on-chip. We describe the porting of the code, discuss the optimization decisions, and report performance of 5.34 Gflop/s with double precision and 5.58 Gflop/s with single precision. We discuss sources of inefficiencies, comparisons with peak performance, comparisons with CPU and GPU performance (taking into account power and price), comparisons with published techniques, and comparisons with published performance, and we conclude with some comments on the prospects for future progress with FPGA acceleration of the weather forecast model. The realization of practical exascale-class high-performance computinems requires significant improvements in the energy efficiency of such systems and their components. This has generated interest in computer architectures which utilize accelerators alongside traditional CPUs. FPGAs offer huge potential as an accelerator which can deliver performance for scientific applications at high levels of energy efficiency. The EuroExa project is developing and building a high-performance architecture based upon ARM CPUs with FPGA acceleration targeting exascale-class performance within a realistic power budget.

Exploring the Acceleration of the Met Office NERC Cloud Model Using FPGAs

“…In [11] the author's matrix-vector kernel involved looping over two double precision floating point operations, whereas in comparison the kernel we are offloading to the FPGA comprises of fifty three double precision floating point operations, twenty one double precision additions or subtractions, and thirty two double precision multiplications. We are also running on much larger grid sizes, and whereas in [11] the authors were limited to a maximum data size of 17MB due to keeping within the BRAM on the Zynq, in the work detailed in this paper we consider grid sizes resulting in 6.44GB of prognostic field data (and a further 6.44GB for the field source terms), necessitating the use of external SDRAM on the PCIe card. Figure 2 illustrates the performance of our HLS PW advection kernel over numerous steps that we applied one after another, for an experiment of x=512, y=512, z=64 (16.7 million grid cells).…”

“…There have been a number of previous activities investigating the role that FPGAs can play in accelerating HPC codes. One such example is [11], where the authors investigated using the high-level productivity design methodology to accelerate solving of the Helmholtz equation. They offloaded the matrix-vector updates requires as part of this solver onto a Zynq Ultrascale, however the performance they observed was around half of that when the code was run on a twelve core Broadwell CPU.…”

“…In terms of FLOPs, at 268 million grid cells our HLS kernel is providing 14.36 GFLOP/s (in comparison to 12 cores of Broadwell at 17.75 GFLOPs/), however when one includes the DMA transfer time this drops down to 4.2 GFLOP/s and so illustrates the very significant impact that DMA transfer time has on our results. The limit with some other investigations such as [11], is that they focus on the embedded CPU-FPGA Zynq chip, and limit their system size very severely to the BRAM on that chip. As such they don't encounter this transfer time overhead, but this is crucially important to bear in mind for processing realistic problems that are of interest to scientists.…”

It's All About Data Movement: Optimising FPGA Data Access to Boost Performance

“…There have been a number of previous activities investigating the role that FPGAs can play in accelerating HPC codes. One such example is [11], where the authors investigated using the high-level productivity design methodology via HLS to accelerate solving the Helmholtz equation. They offloaded the matrix-vector updates which are required as part of this solver onto a Zynq Ultrascale, however the performance obtained was around half of that when the code was run on a twelve core Broadwell CPU.…”

“…They offloaded the matrix-vector updates which are required as part of this solver onto a Zynq Ultrascale, however the performance obtained was around half of that when the code was run on a twelve core Broadwell CPU. In [11] the author's matrix-vector kernel involved looping over two double precision floating point operations, whereas in this work we are focused on accelerating a much more complicated kernel comprising of fifty three double precision floating point operations per grid cell. These double precision operations involve twenty one additions or subtractions, and thirty two double precision multiplications.…”