The Difficult Balance Between Modern Hardware and Conventional CPUs

Abstract

Research has demonstrated the potential of accelerators in a wide range of use cases. However, there is a growing imbalance between modern hardware and the CPUs that submit the workload. Recent studies of GPUs on real systems have shown that many servers are often needed per accelerator to generate a high enough load so the computing power is leveraged. This fact is often ignored in research, although it often determines the actual feasibility and overall efficiency of a deployment. In this paper, we conduct a detailed study of the possible configurations and overall cost efficiency of deploying an FPGA-based accelerator on a commercial search engine. First, we show that there are many possible configurations balancing the upstream system and the way the accelerator is configured. Of these configurations, not all of them are suitable in practice, even if they provide some of the highest throughput. Second, we analyse the cost of a deployment capable of sustaining the required workload of the commercial search engine. We examine deployments both on-premises and in the cloud with and without FPGAs and with different board models. The results show that, while FPGAs have the potential to significantly improve overall performance, the performance imbalance between their host CPUs and the FPGAs can make the deployments economically unattractive. These findings are intended to inform the development and deployment of accelerators by showing what is needed on the CPU side to make them effective and also to provide important insights into their end-to-end integration within existing systems. CCS CONCEPTS• Hardware → Hardware accelerators; • Computer systems organization → Cloud computing; Client-server architectures; Heterogeneous (hybrid) systems.

Strega : An HTTP Server for FPGAs

“…The PCIe bus, as evident from both Figure 8 and Figure 11, does not perform optimally when the ratio between computation and data transfer is high, meaning that there is insufficient data to maximise its throughput. To overcome this limitation, the conventional approach has been to batch process multiple individual invocations into a single operation, which trades higher throughput for significantly increased individual latency [39]. The introduction of Strega, which provides an even higher level of abstraction for FPGA-based kernels compared to OpenCL without compromising performance, marks a significant milestone in the integration of heterogeneous hardware into distributed systems.…”

“…Both communicate trough the PCIe bus, much like GPU kernels, as illustrated in Figure 1: (i) memory is first allocated on the device; (ii) data is transferred from the host to the device; (iii) the kernel is executed; and (iv) the CPU fetches the result data from the device memory. In the context of distributed systems deployed in the cloud, the consequence of this flow is that client requests must navigate through the CPU in order to be accelerated by the FPGA, imposing not only considerable communication overhead [41], but more importantly a very tight coupling between both the host and the accelerator [39].…”

“…Amazon Web Services offers FPGA boards connected to servers with specific numbers of vCPUs [57], but there is no option to connect a powerful CPU to an FPGA device. This limitation makes the use of FPGAs too expensive when a single FPGA can consume the workload produced by several CPUs [39].…”