Performance portability in a real world application: PHAST applied to Caffe

Abstract

This work covers the PHAST Library’s employment, a hardware-agnostic programming library, to a real-world application like the Caffe framework. The original implementation of Caffe consists of two different versions of the source code: one to run on CPU platforms and another one to run on the GPU side. With PHAST, we aim to develop a single-source code implementation capable of running efficiently on CPU and GPU. In this paper, we start by carrying out a basic Caffe implementation performance analysis using PHAST. Then, we detail possible performance upgrades. We find that the overall performance is dominated by few ‘heavy’ layers. In refining the inefficient parts of this version, we find two different approaches: improvements to the Caffe source code and improvements to the PHAST Library itself, which ultimately translates into improved performance in the PHAST version of Caffe. We demonstrate that our PHAST implementation achieves performance portability on CPUs and GPUs. With a single source, the PHAST version of Caffe provides the same or even better performance than the original version of Caffe built from two different codebases. For the MNIST database, the PHAST implementation takes an equivalent amount of time as native code in CPU and GPU. Furthermore, PHAST achieves a speedup of 51% and a 49% with the CIFAR-10 database against native code in CPU and GPU, respectively. These results provide a new horizon for software development in the upcoming heterogeneous computing era.

Analysis and Optimization of Direct Convolution Execution on Multi-Core Processors

“…Where possible, nowadays the trend is to use GPUs, FPGAs, NPUs, and other ad-hoc accelerators for seeking higher performance/efficiency than CPUs [15]. GPUs' massively parallel hardware has been successfully employed in the im2col+gemm convolution implementation, but also in direct convolution [16], [17] recently. State-of-the-art convolutional accelerators (e.g., [18], [19]) use specific dataflow structures that can be seen as portions of direct convolution algorithm mapped in hardware, since tensors are processed spatially without applying any transformations.…”