Beyond Peak Performance: Comparing the Real Performance of AI-Optimized FPGAs and GPUs

Boutros, Andrew; Nurvitadhi, Eriko; Ma, Rui; Gribok, Sergey; Zhao, Zhipeng; Hoe, James C.; Betz, Vaughn; Langhammer, Martin

doi:10.1109/icfpt51103.2020.00011

Cited by 54 publications

(27 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig. 8 illustrates the exploration results of an LSTM layer with (Lx, Lh) = (32,32) and different values of reuse factors, which are from 1 to 10. The red line represents the cases with the same R x and R h .…”

Section: Of An Lstm Layermentioning

confidence: 99%

“…The authors in [10] put each LSTM layer on each multi-core to achieve coarse grained pipelining. In [30,31,32,33], the batching technique is used to improve the hardware throughput and utilization for LSTM inferences. However, latency can suffer since different inputs may not come at the same time, meaning that a newly arrived request has to wait until the batch is formed, which imposes a significant latency penalty.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Accelerating Recurrent Neural Networks for Gravitational Wave Experiments

Que,

Wang,

Marikar

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper presents novel reconfigurable architectures for reducing the latency of recurrent neural networks (RNNs) that are used for detecting gravitational waves. Gravitational interferometers such as the LIGO detectors capture cosmic events such as black hole mergers which happen at unknown times and of varying durations, producing time-series data. We have developed a new architecture capable of accelerating RNN inference for analyzing time-series data from LIGO detectors. This architecture is based on optimizing the initiation intervals (II) in a multi-layer LSTM (Long Short-Term Memory) network, by identifying appropriate reuse factors for each layer. A customizable template for this architecture has been designed, which enables the generation of low-latency FPGA designs with efficient resource utilization using high-level synthesis tools. The proposed approach has been evaluated based on two LSTM models, targeting a ZYNQ 7045 FPGA and a U250 FPGA. Experimental results show that with balanced II, the number of DSPs can be reduced up to 42% while achieving the same IIs. When compared to other FPGA-based LSTM designs, our design can achieve about 4.92 to 12.4 times lower latency.Unlike CNN inference designs [8,9] which only have forward datapaths and can be fully pipelined, there are feedback datapaths in RNN inference and data dependencies exist between the current timestep and the next timestep. Unrolling the timesteps fully may help, however the sequence length

show abstract

Section: Of An Lstm Layermentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Accelerating Recurrent Neural Networks for Gravitational Wave Experiments

Que,

Wang,

Marikar

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…With compute and data intensive deep learning (DL) becoming a major component of many applications, specialized hardware acceleration of such workloads has become a commonplace. More recently, field-programmable gate arrays (FP-GAs) have been shown to deliver state-of-the-art performance when accelerating different DL workloads because of their massive parallelism, flexibility and energy efficiency [1], [2]. With new DL use cases emerging faster than ever, FPGAs are also starting to adapt.…”

Section: Introductionmentioning

confidence: 99%

“…In general, the development of novel FPGA architectures and CAD algorithms depends mainly on a versatile framework that consists of three main components: (1) a set of benchmarks written in a hardware description language or synthesized using high-level synthesis, (2) an architecture model that captures the organization of FPGA blocks and routing architecture as well as area/timing/power models from circuit-level implementations, and (3) a CAD flow that synthesizes the given benchmarks then implements them on a given FPGA architecture [7]. Although most research efforts in the FPGA community are focused on architecture and CAD, benchmarks actually play a crucial role in this flow.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Koios: A Deep Learning Benchmark Suite for FPGA Architecture and CAD Research

Arora¹,

Boutros²,

Rauch³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the prevalence of deep learning (DL) in many applications, researchers are investigating different ways of optimizing FPGA architecture and CAD to achieve better qualityof-results (QoR) on DL-based workloads. In this optimization process, benchmark circuits are an essential component; the QoR achieved on a set of benchmarks is the main driver for architecture and CAD design choices. However, current academic benchmark suites are inadequate, as they do not capture any designs from the DL domain. This work presents a new suite of DL acceleration benchmark circuits for FPGA architecture and CAD research, called Koios. This suite of 19 circuits covers a wide variety of accelerated neural networks, design sizes, implementation styles, abstraction levels, and numerical precisions. These designs are larger, more data parallel, more heterogeneous, more deeply pipelined, and utilize more FPGA architectural features compared to existing open-source benchmarks. This enables researchers to pin-point architectural inefficiencies for this class of workloads and optimize CAD tools on more realistic benchmarks that stress the CAD algorithms in different ways. In this paper, we describe the designs in our benchmark suite, present results of running them through the Verilog-to-Routing (VTR) flow using a recent FPGA architecture model, and identify key insights from the resulting metrics. On average, our benchmarks have 3.7× more netlist primitives, 1.8× and 4.7× higher DSP and BRAM densities, and 1.7× higher frequency with 1.9× more near-critical paths compared to the widely-used VTR suite. Finally, we present two example case studies showing how architectural exploration for DL-optimized FPGAs can be performed using our new benchmark suite.

show abstract